[ovs-discuss] Open vSwitch balance-slb bond mode with two upstream switches

Jason Burns jason.burns at nutanix.com
Thu Sep 10 14:02:55 UTC 2015

Andy and Ben, thanks for your response. The feedback is very helpful. The configuration I desire is with two upstream switches that are directly connected to each other.

Andy, based on the INTERNALS document, I assume that OVS accepts traffic on the inactive slaves, so the concern about the old FIB entry causing traffic to get dropped by OVS doesn’t seem like a problem. 
Bonding accepts unicast packets on any bond slave. [1]

Further, the FIB of both upstream switches would be updated with the new entry when the device connected to OVS sent broadcast traffic again (or if traffic from the OVS connected device went from the new switch to the old switch over the inter switch link)- so this may only be a very short period of time. It’s important to know that traffic may arrive on both incoming bond interfaces after a rebalance because the upstream switches have two separate FIB entries. Thanks for reminding me of that!

I’ve tested this in my lab with Arista 10GbE switches, running both iperf and multicast audio streaming (both client and server apps). I haven’t noticed any dropped packets or performance problems when the rebalance happens.

I did note that the rebalance algorithm does seem a bit aggressive. Assuming there are 5 active hashes (1 big flow on Slave A and 4 small flows on Slave B), and traffic between bond interfaces is almost even, but still outside the balance threshold - we see that in this condition all hashes get swapped during the rebalance - the end state being 4 on Slave A and 1 on Slave B. If traffic remains at this level, the swap back happens at the next rebalance interval.

[1] https://github.com/openvswitch/ovs/blob/master/vswitchd/INTERNALS

Jason Burns

> On Sep 10, 2015, at 3:58 AM, Andy Zhou <azhou at nicira.com> wrote:
> Assume two switches are connected to OVS SLB bond with two slave ports.
> In case the two non-OVS switch are not directly connected. The hosts
> behind one of the switches can not reach hosts behind OVS via
> multicast or broadcast, since only one of the bond slave port will be
> considered active slave. When rebalancing happens, the switched
> away traffic will update the FIB entry of the  new switch, but not
> that of the old switch.
> In case the two non-OVS switches are directly connected. Incoming
> multicast or broadcast won't be a problem any more. However, depends
> on
> traffic pattern, the new traffic may not actually pass through  the
> old switch via the directly connected link. When this happens, The
> stale FIB entries in the old switch now points to the bond slave that
> just got rebalanced away. Causing incoming traffic flow through the
> old switch to be dropped by OVS.  I am not sure this is actually a big
> problem in real life.
> SLB bond with single switch avoids both problems. It is possible there
> are better examples that I am not aware of. But those two examples
> are what I have in mind.
> On Wed, Sep 9, 2015 at 5:11 PM, Ben Pfaff <blp at nicira.com> wrote:
>> On Wed, Sep 09, 2015 at 04:28:07PM -0700, Andy Zhou wrote:
>>> On Wed, Sep 9, 2015 at 9:09 AM, Ben Pfaff <blp at nicira.com> wrote:
>>>> On Fri, Aug 21, 2015 at 07:14:33PM +0000, Jason Burns wrote:
>>>>> Can the balance-slb bond mode in Open vSwitch be used for a bond
>>>>> connected to two separate upstream switches?
>>>>> When I read the documentation [1] it states that only active-backup
>>>>> can be used to connect to two separate upstream switches, but no
>>>>> explanation is provided.
>>>> I don't recall the exact reason, but I remember that this caused
>>>> problems, and that's why we documented it.
>>>> I hope that someone else can speak up with the reason.
>>> Bond traffic rebalancing or link flapping may mass up the mac learning
>>> table of those separate upstream switches.
>> If you have a specific example in mind, then I think we'd all find it
>> enlightening.

More information about the discuss mailing list