[ovs-discuss] Virtual network interface broken

Stefan Kooman stefan at bit.nl
Wed Nov 25 16:18:15 UTC 2015

Dear list,

We experience the following behaviour. After a while, this can be serveral
months, networking for a Virtual Machine (qemu / kvm on Ubuntu Trusty, openvswitch
2.0.2-0ubuntu0.14.04.2) stops working (correctly). At that point several TB of
traffic has passed through the virtual network interface. How do we know
networking stops working? The VM is unable to do any network related
work. If we reboot this VM it is unable to receive a DHCP lease. We
see the arp requests (ipxe) making it out of the virtual network. Four in
total but only one of them makes it onto the bridge / physical network
interface (confirmed with tcpdump). 

- ovs-ofctl dump-ports bridge output for this vnet gives a very high drop
counter  "drop=1795999" which is continuously increasing while the ipxe
bootrom is in a state "Booting from Hard Disk..." 

- ovs-appctl dpif/dump-flows bridge doesn't give an active datapath flow
for this interface

- ovs-appctl ofproto/trace bridge in_port=port-nr dl_src=mac-address,dl_dst=ff:ff:ff:ff:ff:ff gives
"Datapath actions: push_vlan(vid=139,pcp=0),10,12,pop_vlan,43"

So the openflow rules that are in affect allow the DHCP request.
OpenFlow rules active for this interface:

ovs-ofctl dump-flows uplink |grep in_port=17
 cookie=0x0, duration=5549854.234s, table=0, n_packets=8665537966,
 n_bytes=2785309429237, idle_age=1022, hard_age=65534,
 priority=40000,in_port=17,dl_src=mac-address actions=NORMAL
  cookie=0x0, duration=5549854.217s, table=0, n_packets=0, n_bytes=0,
  idle_age=65534, hard_age=65534, priority=39000,in_port=17 actions=drop

If we poweroff / poweron this VM, networking starts working again (reboot
of VM does not help).

The other ~ 30 VMs on the same hypervisor (configured in the same way)
just work.

We have seen this happening to a VM for the third time now.

So for some reason packets seem to dropped by OpenvSwitch. How should I
continue to debug this issue? Any help is appreciated.


Stefan Kooman

| BIT BV  http://www.bit.nl/        Kamer van Koophandel 09090351
| GPG: 0xD14839C6                   +31 318 648 688 / info at bit.nl

More information about the discuss mailing list