<div dir="ltr"><div class="gmail_quote"><div dir="ltr">Hi everyone and sorry for the multiple emails, we are facing a flopping of igb (5.3.4.4) nics which are bonded by using openvswitch. I am in XenServer which is based on CentOS. I am not sure if the problem is the driver or openvswitch. We have six nics that are bonded into two groups of three. The issue appers after some time that the nics are bonded. The <u>kern.log</u> shows (part of): <div><br></div><div><div>Nov 1 15:39:56 AxenD1 kernel: [1017001.797897] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:39:56 AxenD1 kernel: [1017001.887470] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Down</div><div>Nov 1 15:40:00 AxenD1 kernel: [1017005.577876] igb 0000:15:00.1 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:40:00 AxenD1 kernel: [1017005.781865] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:40:10 AxenD1 kernel: [1017015.861861] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:40:10 AxenD1 kernel: [1017015.959454] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Down</div><div>Nov 1 15:40:14 AxenD1 kernel: [1017019.689864] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:40:14 AxenD1 kernel: [1017020.157869] igb 0000:15:00.1 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:40:25 AxenD1 kernel: [1017030.837873] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:40:25 AxenD1 kernel: [1017030.919331] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Down</div><div>Nov 1 15:40:28 AxenD1 kernel: [1017034.205864] igb 0000:15:00.1 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:40:29 AxenD1 kernel: [1017034.829864] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:40:39 AxenD1 kernel: [1017044.837856] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:40:39 AxenD1 kernel: [1017044.931369] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Down</div><div>Nov 1 15:40:43 AxenD1 kernel: [1017048.649860] igb 0000:15:00.1 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:40:43 AxenD1 kernel: [1017048.841867] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:40:53 AxenD1 kernel: [1017058.853860] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:40:53 AxenD1 kernel: [1017058.943421] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Down</div><div>Nov 1 15:40:56 AxenD1 kernel: [1017062.329849] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:40:57 AxenD1 kernel: [1017063.217873] igb 0000:15:00.1 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:41:00 AxenD1 kernel: [1017066.032048] vif vif-537-0 vif537.0: Guest Rx stalled</div><div>Nov 1 15:41:08 AxenD1 kernel: [1017073.845863] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:41:08 AxenD1 kernel: [1017073.935291] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Down</div><div>Nov 1 15:41:11 AxenD1 kernel: [1017077.133868] igb 0000:15:00.1 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:41:12 AxenD1 kernel: [1017077.841902] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:41:22 AxenD1 kernel: [1017087.845881] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:41:22 AxenD1 kernel: [1017087.935415] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Down</div><div>Nov 1 15:41:26 AxenD1 kernel: [1017091.785868] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:41:26 AxenD1 kernel: [1017092.105875] igb 0000:15:00.1 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:41:37 AxenD1 kernel: [1017102.773865] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:41:37 AxenD1 kernel: [1017102.863320] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Down</div><div>Nov 1 15:41:40 AxenD1 kernel: [1017106.101863] igb 0000:15:00.1 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:41:41 AxenD1 kernel: [1017107.241867] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:41:51 AxenD1 kernel: [1017116.725860] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:41:51 AxenD1 kernel: [1017116.815206] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Down</div><div>Nov 1 15:41:55 AxenD1 kernel: [1017120.521882] igb 0000:15:00.1 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:41:55 AxenD1 kernel: [1017120.725854] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:42:05 AxenD1 kernel: [1017130.780242] igb 0000:15:00.1 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:42:05 AxenD1 kernel: [1017130.821934] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:42:05 AxenD1 kernel: [1017130.876058] igb 0000:15:00.1 eth1: igb: eth1 NIC Link is Down</div><div>Nov 1 15:42:05 AxenD1 kernel: [1017130.911424] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Down</div><div>Nov 1 15:42:08 AxenD1 kernel: [1017134.322050] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:42:09 AxenD1 kernel: [1017134.629876] igb 0000:15:00.1 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:42:19 AxenD1 kernel: [1017144.725857] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:42:19 AxenD1 kernel: [1017144.811352] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Down</div><div>Nov 1 15:42:23 AxenD1 kernel: [1017148.533871] igb 0000:15:00.1 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div><div>Nov 1 15:42:23 AxenD1 kernel: [1017149.189892] igb 0000:15:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None</div></div><div><br></div><div><u>At the same time, I am getting this in daemon.log: </u></div><div><br></div><div><div>Nov 1 15:39:58 AxenD1 ovsdb-server: ovs|277473|reconnect|INFO|ssl:<a href="http://192.168.254.60:6632" target="_blank"><wbr>192.168.254.60:6632</a>: connecting...</div><div>Nov 1 15:40:00 AxenD1 ovs-vswitchd: ovs|245685|bond|INFO|interface eth1: link state up</div><div>Nov 1 15:40:00 AxenD1 ovs-vswitchd: ovs|245686|bond|INFO|interface eth1: enabled</div><div>Nov 1 15:40:00 AxenD1 ovs-vswitchd: ovs|245687|bond|INFO|bond bond1: active interface is now eth1</div><div>Nov 1 15:40:00 AxenD1 ovs-vswitchd: ovs|245688|bond|INFO|interface eth1: link state down</div><div>Nov 1 15:40:00 AxenD1 ovs-vswitchd: ovs|245689|bond|INFO|interface eth1: disabled</div><div>Nov 1 15:40:00 AxenD1 ovs-vswitchd: ovs|245690|bond|INFO|bond bond1: all interfaces disabled</div><div>Nov 1 15:40:00 AxenD1 ovs-vswitchd: ovs|245691|bond|INFO|interface eth0: link state up</div><div>Nov 1 15:40:00 AxenD1 ovs-vswitchd: ovs|245692|bond|INFO|interface eth0: will be enabled if it stays up for 31000 ms</div></div><div><br></div><div>Only when the bond is destroyed, the nics stop to go down. Also, when the bond contains only one nic, it works fine. In addition, we observed a lot of overruns, however, I am not sure if they are related with the main issue. Please don't hesitate to ask me for more infomation.<br></div><div><br></div><div><br></div><div>Thanks, Matias. </div></div>
</div><br></div>