[ovs-discuss] OVS - NIC Bonding - Bond Mode = Balance-TCP

bogdan.dancu at orange.com bogdan.dancu at orange.com
Wed Nov 25 16:06:37 UTC 2015

Hi guys,
I don't know if this is the best way to submit an issue, so please point me in the right direction if I'm mistaking.

Here's my configuration:

[root at KVM ~]# uname -a
Linux KVM.bogdan.local 3.10.0-229.14.1.el7.x86_64 #1 SMP Tue Sep 15 15:05:51 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

[root at KVM ~]# ovs-vsctl show
Bridge "ovsbr0"
        Port "vlan1"
            tag: 1
            Interface "vlan1"
                type: internal
        Port "bond0"
            trunks: [399, 1050]
            Interface "p4p1"
            Interface "em1"
        Port "vnic0"
            tag: 1
            Interface "vnic0"
                type: internal
        Port "vlan1050"
            tag: 1050
            Interface "vlan1050"
                type: internal
        Port "ovsbr0"
            Interface "ovsbr0"
                type: internal
        Port "vnic1"
            tag: 1
            Interface "vnic1"
                type: internal
        Port "vlan399"
            tag: 399
            Interface "vlan399"
                type: internal
    ovs_version: "2.4.0"

[root at KVM ~]# ovs-appctl bond/show
---- bond0 ----
bond_mode: balance-tcp
bond may use recirculation: yes, Recirc-ID : 1
bond-hash-basis: 0
updelay: 0 ms
downdelay: 0 ms
next rebalance: 5912 ms
lacp_status: negotiated
active slave mac: f8:b1:56:aa:32:a4(em1)

slave em1: enabled
            active slave
            may_enable: true

slave p4p1: enabled
            may_enable: true
            hash 185: 20 kB load

NOTE : I've attached the ifconfig print to this e-mail in case it's necessary.
So 2 x physical NICs configured in "balance-tcp".
Pretty simple.

Your documentation states :

               Balances flows among slaves based on L2, L3, and L4  protocol  information  such  as  destination MAC address, IP address, and TCP port.


However, when I initiate a single large file transfer (~1GB) between this machine and a Windows client via SCP, the TCP session seems to be spread literally 50/50 between the 2 physical NICs, ignoring the fact that this is a single TCP session.
Here's how it looks in tcpdump:

[root at KVM ~]# tcpdump -i p4p1 tcp port 62297
tcpdump: WARNING: p4p1: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on p4p1, link-type EN10MB (Ethernet), capture size 65535 bytes
17:59:47.083200 IP > KVM.bogdan.local.ssh: Flags [P.], seq 3938497263:3938497375, ack 2900220118, win 32759, length 112
17:59:47.085657 IP > KVM.bogdan.local.ssh: Flags [P.], seq 112:192, ack 81, win 32758, length 80
17:59:47.089082 IP > KVM.bogdan.local.ssh: Flags [P.], seq 192:272, ack 177, win 32758, length 80
17:59:47.091809 IP > KVM.bogdan.local.ssh: Flags [P.], seq 272:512, ack 177, win 32758, length 240
17:59:47.091824 IP > KVM.bogdan.local.ssh: Flags [.], ack 3097, win 32768, length 0
17:59:47.094497 IP > KVM.bogdan.local.ssh: Flags [P.], seq 512:592, ack 4337, win 32758, length 80
17:59:47.094508 IP > KVM.bogdan.local.ssh: Flags [.], ack 7257, win 32768, length 0
17:59:47.094662 IP > KVM.bogdan.local.ssh: Flags [.], ack 10177, win 32768, length 0
17:59:47.094930 IP > KVM.bogdan.local.ssh: Flags [.], ack 13097, win 32768, length 0
17:59:47.095460 IP > KVM.bogdan.local.ssh: Flags [.], ack 16017, win 32768, length 0
17:59:47.096738 IP > KVM.bogdan.local.ssh: Flags [P.], seq 592:752, ack 16721, win 32762, length 160
17:59:47.096874 IP > KVM.bogdan.local.ssh: Flags [.], ack 19641, win 32768, length 0
17:59:47.099184 IP > KVM.bogdan.local.ssh: Flags [P.], seq 752:912, ack 20881, win 32758, length 160

[root at KVM ~]# tcpdump -i em1 tcp port 62297
tcpdump: WARNING: em1: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on em1, link-type EN10MB (Ethernet), capture size 65535 bytes
17:59:47.085935 IP KVM.bogdan.local.ssh > Flags [P.], seq 2900220198:2900220294, ack 3938497455, win 157, length 96
17:59:47.089450 IP KVM.bogdan.local.ssh > Flags [.], seq 96:3016, ack 81, win 157, length 2920
17:59:47.089465 IP KVM.bogdan.local.ssh > Flags [P.], seq 3016:4256, ack 81, win 157, length 1240
17:59:47.092257 IP KVM.bogdan.local.ssh > Flags [.], seq 4256:7176, ack 321, win 167, length 2920
17:59:47.092266 IP KVM.bogdan.local.ssh > Flags [.], seq 7176:10096, ack 321, win 167, length 2920
17:59:47.092271 IP KVM.bogdan.local.ssh > Flags [.], seq 10096:13016, ack 321, win 167, length 2920
17:59:47.092275 IP KVM.bogdan.local.ssh > Flags [.], seq 13016:15936, ack 321, win 167, length 2920
17:59:47.092279 IP KVM.bogdan.local.ssh > Flags [P.], seq 15936:16640, ack 321, win 167, length 704
17:59:47.094757 IP KVM.bogdan.local.ssh > Flags [.], seq 16640:19560, ack 401, win 167, length 2920
17:59:47.094764 IP KVM.bogdan.local.ssh > Flags [P.], seq 19560:20800, ack 401, win 167, length 1240

Based on the definition in your official documentation, I was assuming that bond0 would keep a TCP session on one NIC and initiate the next TCP session through the 2nd NIC, and so on.
So a stickiness of TCP sessions if you will :)
However, what I'm seeing is that OVS is using both physical interfaces for the data transfer itself on 1 x NIC and control information (TCP ACKs) on the other NIC for the same TCP session.

Am I wrong in interpreting your documentation?
Is there any other LOG I could provide you with?

Thank you very much for your effort.
Best Regards,

[Description: cid:116511014 at 30092011-08A6]
Bogdan DANCU
Datacenter Technical Specialist
Orange/IMT/OINIS/ITE Romania
Mobile: (+40)744445044
Bogdan.Dancu at orange.com<mailto:Bogdan.Dancu at orange.com>


Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorization.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, France Telecom - Orange shall not be liable if this message was modified, changed or falsified.
Thank you.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20151125/c0ca3aae/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 1264 bytes
Desc: image001.gif
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20151125/c0ca3aae/attachment-0002.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ip_addr_add_txt_extension
Type: application/octet-stream
Size: 3137 bytes
Desc: ip_addr_add_txt_extension
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20151125/c0ca3aae/attachment-0002.obj>

More information about the discuss mailing list