[ovs-discuss] Best Practices for Configuring a Bonded Interface with Open vSwitch?

Gurucharan Shetty shettyg at nicira.com
Tue Jul 7 14:50:57 UTC 2015


I tried to follow the emails, but got lost a bit.
Fwiw,
 If you get bitten more, do have a look at
https://github.com/openvswitch/ovs/blob/master/rhel/etc_sysconfig_network-scripts_ifup-ovs
to know how the information in ifcfg-* gets translated into OVS
objects.

On Mon, Jul 6, 2015 at 1:22 PM, Eric <epretorious at yahoo.com> wrote:
>> e.g., When restarting the host's networking stack, the system complains:
>
> After destroying the bond and disbling the bond's ifcfg file:
>
>> [root at xen-2 ~]# grep ONBOOT
>> /etc/sysconfig/network-scripts/ifcfg-{bond2,eth1,eth5}
>> /etc/sysconfig/network-scripts/ifcfg-bond2:ONBOOT=no
>> /etc/sysconfig/network-scripts/ifcfg-eth1:ONBOOT=no
>> /etc/sysconfig/network-scripts/ifcfg-eth5:ONBOOT=no
>>
>> [root at xen-2 ~]# ovs-vsctl show
>> 0dfeba4c-bcae-4773-8bd9-582a9ebbd01a
>>     Bridge EXT
>>         Port "bond3"
>>             Interface "eth5"
>>             Interface "eth1"
>>         Port EXT
>>             Interface EXT
>>                 type: internal
>>     ovs_version: "2.3.2"
>>
>> [root at xen-2 ~]# ovs-appctl bond/show
>> ---- bond3 ----
>> bond_mode: active-backup
>> bond may use recirculation: no, Recirc-ID : -1
>> bond-hash-basis: 0
>> updelay: 0 ms
>> downdelay: 0 ms
>> lacp_status: off
>> active slave mac: 00:1f:29:69:6e:f0(eth1)
>>
>> slave eth1: enabled
>>         active slave
>>         may_enable: true
>>
>> slave eth5: enabled
>>         may_enable: true
>
> ...things are working much more reliably. (I've noticed that, after creating
> the virtual bridge's bond *from*the*CLI*, things are a bit unstable.
> Creating the virtual bridge's bond *and*then* rebooting [to allow the init
> scripts to configure everything] seems to work much more reliably.)
>
>> Should the 'DEVICE' and the 'OVS_BRIDGE' configuration settings be
>> changed (to prevent the errors [above])? Or are they correct? e.g.,
>> Should one of those settings contain the name of the [virtual] port
>> 'bond3'?
>
> I've played with the /etc/sysconfig/network-scripts/ifcfg-bond3 file and I
> haven't quite figured out the significance of the DEVICE and the OVS_BRIDGE
> values yet except that:
>
>  * OVS_BRIDGE should be set to the name of the OVS bridge device.
>
>> Should the 'ifcfg-bond3' file be renamed? Or is 'ifcfg-bond3'
>> correct+appropriate?
>
> Seems to work just fine.
>
>> Should I modify the 'ifcfg-eth1' & 'ifcfg-eth5' settings (to prevent
>> the "TRNETLINK" errors [above])?
>
> I've set "ONBOOT=no" for both interfaces and that seems to work just fine.
>
> Eric Pretorious
> Portland, OR
>
>
> ________________________________
> From: Eric <epretorious at yahoo.com>
> To: "discuss at openvswitch.org" <discuss at openvswitch.org>
> Sent: Saturday, July 4, 2015 4:34 PM
>
> Subject: Re: Best Practices for Configuring a Bonded Interface with Open
> vSwitch?
>
> During a reboot, I see LOTS of errors about my sloppy typing:
>
>> BOND_IFACES="ext1 ext5"
>
> Otherwise, things are working pretty well:
>
>> [root at xen-2 network-scripts]# ovs-vsctl show
>> 0dfeba4c-bcae-4773-8bd9-582a9ebbd01a
>>     Bridge EXT
>>         Port "bond3"
>>             Interface "eth5"
>>             Interface "eth1"
>>         Port EXT
>>             Interface EXT
>>                 type: internal
>>     ovs_version: "2.3.2"
>>
>> [root at xen-2 network-scripts]# ovs-appctl bond/show bond3
>> ---- bond3 ----
>> bond_mode: active-backup
>> bond may use recirculation: no, Recirc-ID : -1
>> bond-hash-basis: 0
>> updelay: 0 ms
>> downdelay: 0 ms
>> lacp_status: off
>> active slave mac: 00:1f:29:69:6e:f0(eth1)
>>
>> slave eth1: enabled
>>         active slave
>>         may_enable: true
>>
>> slave eth5: enabled
>>         may_enable: true
>
> I'm still kind'a puzzled about the 'DEVICE' and the 'OVS_BRIDGE'
> configuration settings, though:
>
>> [root at xen-2 network-scripts]# cat ifcfg-bond3
>> DEVICE=EXT
>> DEVICETYPE=ovs
>> TYPE=OVSBond
>> OVS_BRIDGE=EXT
>> BOND_IFACES="eth1 eth5"
>> OVS_OPTIONS="bonding_mode=active-backup"
>> BOOTPROTO=static
>> IPADDR=XXX.YYY.237.65
>> NETMASK=255.255.255.0
>> ONBOOT=yes
>> USERCTL=no
>> NM_CONTROLLED=no
>
> e.g., When restarting the host's networking stack, the system complains:
>
>> Bringing up interface bond3:  ovs-vsctl: cannot create a port named EXT
>> because a bridge named EXT already exists
>> RTNETLINK answers: File exists
>> Bringing up interface eth1:  RTNETLINK answers: File exists
>> Bringing up interface eth5:  RTNETLINK answers: File exists
>
> ...and eth1 & eth5 are the only two [physical] interfaces to have IPv6
> addresses assigned to them. (The other four [physical] interfaces are in
> active bonds ['bond0' & 'bond1'].)
>
>> eth1      Link encap:Ethernet  HWaddr 00:1F:29:69:6E:F0
>>           inet6 addr: fe80::XXXX:29ff:fe69:6ef0/64 Scope:Link
>> eth5      Link encap:Ethernet  HWaddr 00:17:08:7D:66:1A
>>           inet6 addr: fe80::YYYY:8ff:fe7d:661a/64 Scope:Link
>
> Should the 'DEVICE' and the 'OVS_BRIDGE' configuration settings be changed
> (to prevent the errors [above])? Or are they correct? e.g., Should one of
> those settings contain the name of the [virtual] port 'bond3'?
>
> Should the 'ifcfg-bond3' file be renamed? Or is 'ifcfg-bond3'
> correct+appropriate?
>
> Should I modify the 'ifcfg-eth1' & 'ifcfg-eth5' settings (to prevent the
> "TRNETLINK" errors [above])?
>
> TIA,
> Eric Pretorious
> Portland, OR
>
>
>
>
>
> ________________________________
> From: Eric <epretorious at yahoo.com>
> To: "discuss at openvswitch.org" <discuss at openvswitch.org>
> Sent: Saturday, July 4, 2015 3:26 PM
> Subject: Re: Best Practices for Configuring a Bonded Interface with Open
> vSwitch?
>
> After reading "Red Hat network scripts integration"
> [https://github.com/openvswitch/ovs/blob/master/rhel/README.RHEL] I realized
> that I had a few typos in my configuration script so I decided to start all
> over.
>
> I disable the bonded [physical] interface 'bond2':
>
>> [root at xen-2 ~]# ifenslave -d bond2 eth1 eth5
>> [root at xen-2 ~]# grep ONBOOT /etc/sysconfig/network-scripts/ifcfg-bond2
>>  ONBOOT=no
>
> I removed the bonding configuration setting from eth1 & eth5:
>
>> [root at xen-2 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth{1,5}
>> DEVICE=eth1
>> HWADDR=00:1F:29:69:6E:F0
>> TYPE=Ethernet
>> UUID=c1e34dd4-4fc5-47e7-9f90-a628d455528c
>> ONBOOT=no
>> USERCTL=no
>> NM_CONTROLLED=no
>> BOOTPROTO=none
>> ...
>> DEVICE=eth5
>> HWADDR=00:17:08:7D:66:1A
>> TYPE=Ethernet
>> UUID=72ff5cc7-b0c6-4958-bc8c-11ed95bd18a4
>> ONBOOT=no
>> USERCTL=no
>> NM_CONTROLLED=no
>> BOOTPROTO=none
>
> I deleted all OVS bridges:
>
>> [root at xen-2 ~]# ovs-vsctl del-br SAN
>> [root at xen-2 ~]# ovs-vsctl del-br MGT
>> [root at xen-2 ~]# ovs-vsctl del-br EXT
>
> ...and then I started from scratch (*after* a reboot)...
>
> I created the virtual bridge 'EXT':
>
>> [root at xen-2 ~]# ovs-vsctl add-br EXT
>
> I added a [virtual] bonded port 'bond3' to the virtual bridge 'EXT':
>
>> [root at xen-2 ~]# ovs-vsctl add-bond EXT bond3 eth1 eth5
>
> ...and I created the configuration for the 'EXT' interface from scratch:
>
>> [root at xen-2 ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond3
>> DEVICE=EXT
>> DEVICETYPE=ovs
>> TYPE=OVSBond
>> OVS_BRIDGE=EXT
>> BOND_IFACES="ext1 ext5"
>> OVS_OPTIONS="bonding_mode=active-backup"
>> BOOTPROTO=static
>> IPADDR=XXX.YYY.237.65
>> NETMASK=255.255.255.0
>> ONBOOT=yes
>> USERCTL=no
>> NM_CONTROLLED=no
>
> This seems to work:
>
>> [root at xen-2 ~]# ifconfig
>> EXT       Link encap:Ethernet  HWaddr 00:1F:29:69:6E:F0
>>           inet addr:XXX.YYY.237.65  Bcast:XXX.YYY.237.255
>> Mask:255.255.255.0
>> <SNIP>
>>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>> <SNIP>
>>
>> [root at xen-2 ~]# ping -c3 XXX.YYY.237.1
>> PING XXX.YYY.237.1 (XXX.YYY.237.1) 56(84) bytes of data.
>> <SNIP>
>>
>> --- XXX.YYY.237.1 ping statistics ---
>> 3 packets transmitted, 3 received, 0% packet loss, time 2012ms
>> ...
>>
>> [root at xen-2 ~]# ovs-vsctl show
>> 0dfeba4c-bcae-4773-8bd9-582a9ebbd01a
>>     Bridge EXT
>>         Port "bond3"
>>             Interface "eth5"
>>             Interface "eth1"
>>         Port EXT
>>             Interface EXT
>>                 type: internal
>>     ovs_version: "2.3.2"
>
>> [root at xen-2 ~]# ovs-appctl bond/show bond3
>> ---- bond3 ----
>> bond_mode: active-backup
>> bond may use recirculation: no, Recirc-ID : -1
>> bond-hash-basis: 0
>> updelay: 0 ms
>> downdelay: 0 ms
>> lacp_status: off
>> active slave mac: 00:1f:29:69:6e:f0(eth1)
>>
>> slave eth1: enabled
>>         active slave
>>         may_enable: true
>>
>> slave eth5: enabled
>>         may_enable: true
>>
>
> I see that I have some adjustments to make in the 'OVS_BRIDGE' &
> 'DEVICETYPE' fields of the ifcfg-bond3 configuration file but, overall: Is
> this approach "more correct"?
>
> TIA,
> Eric Pretorious
> Portland, OR
>
>
>
> ________________________________
> From: Eric <epretorious at yahoo.com>
> To: "discuss at openvswitch.org" <discuss at openvswitch.org>
> Sent: Saturday, July 4, 2015 11:24 AM
> Subject: Best Practices for Configuring a Bonded Interface with Open
> vSwitch?
>
> Hello, All:
>
> I've just installed OVS on my Xen host and I'm wondering if there's a
> recommended configuration for bonding [pysical] interfaces with OVS? e.g.,
> My host system has six [physical] interfaces. I've combined those [physical]
> interfaces to create three bonded interfaces: 'bond0', 'bond1', and 'bond2'.
>
> For the initial OVS implementation, I created three virtual bridges: 'EXT',
> 'MGT', and 'SAN':
>
>> [root at xen-2 ~]# ovs-vsctl show
>> 0dfeba4c-bcae-4773-8bd9-582a9ebbd01a
>>     Bridge SAN
>>         Port SAN
>>             Interface SAN
>>                 type: internal
>>     Bridge EXT
>>         Port EXT
>>             Interface EXT
>>                 type: internal
>>     Bridge MGT
>>         Port MGT
>>             Interface MGT
>>                 type: internal
>>     ovs_version: "2.3.2"
>
> Then I removed the IP address from the bonded interface 'bond2' (manually
> *and* via configuration files) and connected it to the virtual bridge 'EXT':
>
>> [root at xen-2 ~]# ovs-vsctl show
>> 0dfeba4c-bcae-4773-8bd9-582a9ebbd01a
>>     Bridge SAN
>>         Port SAN
>>             Interface SAN
>>                 type: internal
>>     Bridge EXT
>>         Port "bond2"
>>             Interface "bond2"
>>         Port EXT
>>             Interface EXT
>>                 type: internal
>>     Bridge MGT
>>         Port MGT
>>             Interface MGT
>>                 type: internal
>>     ovs_version: "2.3.2"
>
> This works well enough but, after a reboot, the 'bond2' interface refuses to
> pass any traffic to the [physical] gateway, XXX.YYY.237.1 until I disconnect
> the bond (i.e., 'bond2') from the virtual switch and re-attach it:
>
>> [root at xen-2 ~]# ping -c3 XXX.YYY.237.1
>> PING XXX.YYY.237.1 (XXX.YYY.237.1) 56(84) bytes of data.
>> From XXX.YYY.237.65 icmp_seq=1 Destination Host Unreachable
>> From XXX.YYY.237.65 icmp_seq=2 Destination Host Unreachable
>> From XXX.YYY.237.65 icmp_seq=3 Destination Host Unreachable
>>
>> --- XXX.YYY.237.1 ping statistics ---
>> 3 packets transmitted, 0 received, +3 errors, 100% packet loss, time
>> 2937ms
>> pipe 3
>>
>> [root at xen-2 ~]# ovs-vsctl del-port EXT bond2 && ovs-vsctl --may-exist
>> add-port EXT bond2
>>
>> [root at xen-2 ~]# ping -c3 XXX.YYY.237.1
>> PING XXX.YYY.237.1 (XXX.YYY.237.1) 56(84) bytes of data.
>> 64 bytes from XXX.YYY.237.1: icmp_seq=1 ttl=64 time=56.7 ms
>> 64 bytes from XXX.YYY.237.1: icmp_seq=2 ttl=64 time=48.8 ms
>> 64 bytes from XXX.YYY.237.1: icmp_seq=3 ttl=64 time=47.6 ms
>>
>> --- XXX.YYY.237.1 ping statistics ---
>> 3 packets transmitted, 3 received, 0% packet loss, time 2049ms
>> rtt min/avg/max/mdev = 47.650/51.073/56.724/4.033 ms
>
> Should I destroy the 'bond2' interface and create the bond using OVS
> instead? Or is this configuration correct?
>
> FWIW: I've provided a few more details and demonstrations here:
> https://www.centos.org/forums/viewtopic.php?f=50&t=53219
>
> TIA,
> Eric Pretorious
> Portland, OR
>
> REFERENCE:
>
> [root at xen-2 ~]# cat /etc/sysconfig/network
> /etc/sysconfig/network-scripts/ifcfg-{bond2,EXT}
> NETWORKING=yes
> GATEWAY=XXX.YYY.237.1
> HOSTNAME=xen-2
> DNS1=208.67.220.220
> DNS2=208.67.222.222
> ...
> DEVICE=bond2
> BOOTPROTO=none
> ONBOOT=yes
> NM_CONTOLLED=no
> USERCTL=no
> BONDING_OPTS='mode=1 miimon=100'
> ...
> DEVICE=EXT
> OVSBRIDGE=EXT
> TYPE=ovs
> DEVICETYPE=OVSPort
> IPADDR=XXX.YYY.237.65
> NETMASK=255.255.255.0
> BOOTPROTO=static
> ONBOOT=yes
>
>
>
>
>
>
>
> _______________________________________________
> discuss mailing list
> discuss at openvswitch.org
> http://openvswitch.org/mailman/listinfo/discuss
>



More information about the discuss mailing list