[ovs-discuss] ovs-bonded igb nics flopping

Matias Vara matiasevara at gmail.com
Wed Jan 11 13:15:10 UTC 2017


Hi everyone and sorry for the multiple emails, we are facing a flopping of
igb (5.3.4.4) nics which are bonded by using openvswitch. I am in XenServer
which is based on CentOS. I am not sure if the problem is the driver or
openvswitch. We have six nics that are bonded into two groups of three. The
issue appers after some time that the nics are bonded. The *kern.log* shows
(part of):

Nov  1 15:39:56 AxenD1 kernel: [1017001.797897] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:39:56 AxenD1 kernel: [1017001.887470] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Down
Nov  1 15:40:00 AxenD1 kernel: [1017005.577876] igb 0000:15:00.1 eth1: igb:
eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:40:00 AxenD1 kernel: [1017005.781865] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:40:10 AxenD1 kernel: [1017015.861861] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:40:10 AxenD1 kernel: [1017015.959454] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Down
Nov  1 15:40:14 AxenD1 kernel: [1017019.689864] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:40:14 AxenD1 kernel: [1017020.157869] igb 0000:15:00.1 eth1: igb:
eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:40:25 AxenD1 kernel: [1017030.837873] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:40:25 AxenD1 kernel: [1017030.919331] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Down
Nov  1 15:40:28 AxenD1 kernel: [1017034.205864] igb 0000:15:00.1 eth1: igb:
eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:40:29 AxenD1 kernel: [1017034.829864] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:40:39 AxenD1 kernel: [1017044.837856] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:40:39 AxenD1 kernel: [1017044.931369] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Down
Nov  1 15:40:43 AxenD1 kernel: [1017048.649860] igb 0000:15:00.1 eth1: igb:
eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:40:43 AxenD1 kernel: [1017048.841867] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:40:53 AxenD1 kernel: [1017058.853860] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:40:53 AxenD1 kernel: [1017058.943421] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Down
Nov  1 15:40:56 AxenD1 kernel: [1017062.329849] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:40:57 AxenD1 kernel: [1017063.217873] igb 0000:15:00.1 eth1: igb:
eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:41:00 AxenD1 kernel: [1017066.032048] vif vif-537-0 vif537.0:
Guest Rx stalled
Nov  1 15:41:08 AxenD1 kernel: [1017073.845863] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:41:08 AxenD1 kernel: [1017073.935291] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Down
Nov  1 15:41:11 AxenD1 kernel: [1017077.133868] igb 0000:15:00.1 eth1: igb:
eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:41:12 AxenD1 kernel: [1017077.841902] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:41:22 AxenD1 kernel: [1017087.845881] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:41:22 AxenD1 kernel: [1017087.935415] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Down
Nov  1 15:41:26 AxenD1 kernel: [1017091.785868] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:41:26 AxenD1 kernel: [1017092.105875] igb 0000:15:00.1 eth1: igb:
eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:41:37 AxenD1 kernel: [1017102.773865] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:41:37 AxenD1 kernel: [1017102.863320] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Down
Nov  1 15:41:40 AxenD1 kernel: [1017106.101863] igb 0000:15:00.1 eth1: igb:
eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:41:41 AxenD1 kernel: [1017107.241867] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:41:51 AxenD1 kernel: [1017116.725860] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:41:51 AxenD1 kernel: [1017116.815206] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Down
Nov  1 15:41:55 AxenD1 kernel: [1017120.521882] igb 0000:15:00.1 eth1: igb:
eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:41:55 AxenD1 kernel: [1017120.725854] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:42:05 AxenD1 kernel: [1017130.780242] igb 0000:15:00.1 eth1: igb:
eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:42:05 AxenD1 kernel: [1017130.821934] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:42:05 AxenD1 kernel: [1017130.876058] igb 0000:15:00.1 eth1: igb:
eth1 NIC Link is Down
Nov  1 15:42:05 AxenD1 kernel: [1017130.911424] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Down
Nov  1 15:42:08 AxenD1 kernel: [1017134.322050] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:42:09 AxenD1 kernel: [1017134.629876] igb 0000:15:00.1 eth1: igb:
eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:42:19 AxenD1 kernel: [1017144.725857] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:42:19 AxenD1 kernel: [1017144.811352] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Down
Nov  1 15:42:23 AxenD1 kernel: [1017148.533871] igb 0000:15:00.1 eth1: igb:
eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Nov  1 15:42:23 AxenD1 kernel: [1017149.189892] igb 0000:15:00.0 eth0: igb:
eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None

*At the same time, I am getting this in daemon.log: *

Nov  1 15:39:58 AxenD1 ovsdb-server: ovs|277473|reconnect|INFO|ssl:
192.168.254.60:6632: connecting...
Nov  1 15:40:00 AxenD1 ovs-vswitchd: ovs|245685|bond|INFO|interface eth1:
link state up
Nov  1 15:40:00 AxenD1 ovs-vswitchd: ovs|245686|bond|INFO|interface eth1:
enabled
Nov  1 15:40:00 AxenD1 ovs-vswitchd: ovs|245687|bond|INFO|bond bond1:
active interface is now eth1
Nov  1 15:40:00 AxenD1 ovs-vswitchd: ovs|245688|bond|INFO|interface eth1:
link state down
Nov  1 15:40:00 AxenD1 ovs-vswitchd: ovs|245689|bond|INFO|interface eth1:
disabled
Nov  1 15:40:00 AxenD1 ovs-vswitchd: ovs|245690|bond|INFO|bond bond1: all
interfaces disabled
Nov  1 15:40:00 AxenD1 ovs-vswitchd: ovs|245691|bond|INFO|interface eth0:
link state up
Nov  1 15:40:00 AxenD1 ovs-vswitchd: ovs|245692|bond|INFO|interface eth0:
will be enabled if it stays up for 31000 ms

Only when the bond is destroyed, the nics stop to go down. Also, when the
bond contains only one nic, it works fine. In addition, we observed a lot
of overruns, however, I am not sure if they are related with the main
issue. Please don't hesitate to ask me for more infomation.


Thanks, Matias.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20170111/afaedf1e/attachment.html>


More information about the discuss mailing list