[ovs-discuss] vlan and bridge problem on CentOS 6.3 with 1.6.1

Gianluca Cecchi gianluca.cecchi at gmail.com
Thu Aug 2 10:00:28 UTC 2012


On Thu, Jul 26, 2012 at 10:50 PM, Jesse Gross  wrote:
>
> You can look into "fake bridges", which emulate a bridge hanging off
> an access port of a switch (which in practice just makes all attached
> ports access ports on the original bridge).
>

Sorry for not coming back before... still fighting with CentOS 6.2 ->
6.3 update and clvmd mess...
In the mean time I was able to configure parent br0 bridge and two
fake switches as you suggested and it works like a charm, also in
terms of rhel init files integration

Just for giving clearer context, I'm testing passing from

CentOS 6.2 three nodes cluster with clvmd and virtual machines
configured as rhcs services with standard bridge config
to
CentOS 6.3 three nodes cluster with clvmd and virtual machines
configured as rhcs services with openvswitch and brcompat to be
transparent as far as vm config is related
physically:
2 eth interfaces in bond1 for intracluster
2 eth interfaces in bond0 for vlans (host and VMs). On this bond I
also have ip for my hosts, something like

                              ___ bond0.65 --> brvlan65 (lan for VMs here)
eth2, eth3 ---> bond0
                              ___ bond0.66 --> brvlan66 (both ip of
the server and lan for VMs here)

Now with openvswitch 1.6.1 all is ok
# ovs-vsctl show
...
    Bridge "br0"
        Port "br0"
            Interface "br0"
                type: internal
        Port "brvlan65"
            tag: 65
            Interface "brvlan65"
                type: internal
        Port "bond0"
            trunks: [65, 66]
            Interface "eth3"
            Interface "eth2"
        Port "brvlan66"
            tag: 66
            Interface "brvlan66"
                type: internal
..


# ovs-appctl bond/list
bond    type    slaves
bond0   balance-slb     eth3, eth2

# ovs-appctl bond/show bond0
---- bond0 ----
bond_mode: balance-slb
bond-hash-basis: 0
updelay: 200 ms
downdelay: 0 ms
next rebalance: 9946 ms
lacp_status: off

slave eth2: enabled
        may_enable: true

slave eth3: enabled
        active slave
        may_enable: true
        hash 174: 1 kB load

the server is a blade and Eth2 and eth3 connect to 2 different
switches inside the enclosure (not stacked together), so I think slb
is ok..
Not being able to complete the test loop with a vm on the SAN because
I have problems with clvmd hanging at startup (I think not related to
openvswitch), but this is another story.

If it could be useful for others, my working config:

--- ifcfg-bond0 ---
DEVICE=bond0
ONBOOT=yes
TYPE=Ethernet
BOOTPROTO=none
DEVICETYPE=ovs
TYPE=OVSBond
OVS_BRIDGE=br0
BOND_IFACES="eth2 eth3"
OVS_OPTIONS="bond_mode=balance-slb trunks=65,66 bond_updelay=200
other_config:bond-detect-mode=miimon
other_config:bond-miimon-interval=100"
HOTPLUG=no
NM_CONTROLLED=no

--- ifcfg-br0 ---
DEVICE=br0
DEVICETYPE=ovs
TYPE=OVSBridge
ONBOOT=yes
BOOTPROTO=static
STP=off
NM_CONTROLLED=no
HOTPLUG=no
OVS_EXTRA="br-set-external-id $DEVICE bridge-id $DEVICE"

--- ifcfg-brvlan65 ---
DEVICE=brvlan65
DEVICETYPE=ovs
TYPE=OVSBridge
ONBOOT=yes
BOOTPROTO=static
STP=off
NM_CONTROLLED=no
HOTPLUG=no
OVS_EXTRA="br-set-external-id $DEVICE bridge-id $DEVICE"
OVS_OPTIONS="br0 65"

--- ifcfg-brvlan66 ---
DEVICE=brvlan66
DEVICETYPE=ovs
TYPE=OVSBridge
ONBOOT=yes
BOOTPROTO=static
STP=off
NM_CONTROLLED=no
HOTPLUG=no
OVS_EXTRA="br-set-external-id $DEVICE bridge-id $DEVICE"
OVS_OPTIONS="br0 66"
IPADDR=IP_ADDR_ON_VLAN66
NETMASK=255.255.255.0

For the "brctl" output compatibility "problems" I noticed this,
comparing my previous config on the same node with standard bridges
and now with openvswitch:

# ll /sys/class/net/brvlan65/
diff between openvswitch compat and standard bridge
2a3
> flush
5a7,8
> hash_elasticity
> hash_max
8a12,21
> multicast_last_member_count
> multicast_last_member_interval
> multicast_membership_interval
> multicast_querier_interval
> multicast_query_interval
> multicast_query_response_interval
> multicast_router
> multicast_snooping
> multicast_startup_query_count
> multicast_startup_query_interval

previous with standard bridge (2 vm with vnet1 and vnet3 and bond0.65)
# ll /sys/class/net/brvlan65/brif/
total 0
lrwxrwxrwx 1 root root 0 Jul 25 16:44 bond0.65 -> ../../bond0.65/brport
lrwxrwxrwx 1 root root 0 Jul 25 16:44 vnet1 -> ../../vnet1/brport
lrwxrwxrwx 1 root root 0 Jul 25 16:44 vnet3 -> ../../vnet3/brport

# ll  /sys/class/net/bond0.65/brport/
total 0
lrwxrwxrwx 1 root root    0 Jul 27 23:41 bridge -> ../../brvlan65
-r--r--r-- 1 root root 4096 Jul 27 23:41 change_ack
-r--r--r-- 1 root root 4096 Jul 27 23:41 config_pending
...

ll  /sys/class/net
...
lrwxrwxrwx 1 root root 0 Jul 20 18:07 /sys/class/net/vnet1 ->
../../devices/virtual/net/vnet1

# ll  /sys/devices/virtual/net/
total 0
drwxr-xr-x 6 root root 0 Jul 20 16:15 bond0
drwxr-xr-x 6 root root 0 Jul 20 16:15 bond0.65
...
drwxr-xr-x 7 root root 0 Jul 20 16:15 brvlan65
...
drwxr-xr-x 6 root root 0 Jul 20 18:07 vnet1
...
drwxr-xr-x 6 root root 0 Jul 23 18:09 vnet3

# ll /sys/class/net/vnet1/
...
drwxr-xr-x 2 root root    0 Jul 25 16:44 brport

ll /sys/class/net/vnet1/brport/
total 0
lrwxrwxrwx 1 root root    0 Jul 27 23:55 bridge -> ../../brvlan65
-r--r--r-- 1 root root 4096 Jul 27 23:55 change_ack
-r--r--r-- 1 root root 4096 Jul 27 23:55 config_pending
...

# ll /sys/class/net/bond0.65/brport
total 0
lrwxrwxrwx 1 root root    0 Jul 27 23:41 bridge -> ../../brvlan65
-r--r--r-- 1 root root 4096 Jul 27 23:41 change_ack
-r--r--r-- 1 root root 4096 Jul 27 23:41 config_pending
...

Now with openvswitch (1 vm with vnet0 and eth2)
# ls -1 /sys/class/net/brvlan65/brif/

# ll  /sys/class/net/
total 0
...
lrwxrwxrwx 1 root root    0 Jul 25 17:19 vnet0 ->
../../devices/virtual/net/vnet0

# ll  /sys/devices/virtual/net/
total 0
..
drwxr-xr-x 7 root root 0 Jul 25 16:23 brvlan65
...
drwxr-xr-x 5 root root 0 Jul 25 17:19 vnet0

# ll /sys/class/net/vnet0/
--> no "brport" directory as it happens in real bridge...

# ll /sys/class/net/eth2/
--> no "brport" directory as it happens in real bridge

So I think the errors are related with these "brif" and "brport"
directories and links not created when bridge generated and searched
for by "brctl" command to elaborate its output...

BTW: I'm going to compile and test on CentOS 6.3 with 1.7.0



More information about the discuss mailing list