[ovs-discuss] Transitivity of Openvswitch connections.

Saul St. John sstjohn at cs.wisc.edu
Sun Dec 30 07:26:18 UTC 2012

On 12/29/2012 07:43 PM, Ben Pfaff wrote:
> On Sat, Dec 29, 2012 at 06:38:17PM -0600, Anand Krishnamurthy wrote:
>> 2012-12-30T00:22:49Z|49582|dpif|WARN|system at ovs-system: failed to flow_del (No such file or directory) ipv4_tunnel(tun_id=0x0,src=,dst=,tos=0x0,ttl=64,flags()),in_port(5),eth(src=76:e8:f3:9f:95:a6,dst=01:80:c2:00:00:0e),eth_type(0x88cc)
>> 2012-12-30T00:22:50Z|49583|ofproto_dpif|WARN|Dropped 3 log messages in last 45 seconds (most recently, 15 seconds ago) due to excessive rate
>> 2012-12-30T00:22:50Z|49584|ofproto_dpif|WARN|unexpected flow on br0: ipv4_tunnel(tun_id=0x0,src=,dst=,tos=0x0,ttl=64,flags()),in_port(5),eth(src=76:e8:f3:9f:95:a6,dst=01:80:c2:00:00:0e),eth_type(0x88cc)
> The above errors indicate a bug in Open vSwitch.  Their form indicates
> that you must be using a recent build from master.  What commit are you
> running?

We're using a build from commit a4454ac. I ran some other tests on the
strange behavior we've been seeing with this build that might be
related, deets below and in attachments.

(tl;dr: ovswitches are failing to forward traffic from one GRE interface
to another GRE interface correctly, but succeeds when a patch-pair is
interposed between the tunnels.)


Experimental setup consists of thee hosts, named legolas, sam, and saruman.

* Each host's LOM is assigned an IP in the subnet, and
they're all connected to a traditional flood-and-learn switch.

* Each host runs an openvswitch, the internal port of which is assigned
an IP address in the

* Each openvswitch has two GRE ports, each connecting to the other two
openvswitches through the LOMs.

(At this point, I have full connectivity across all the bridges, the
GRE tunnels work as expected, and each host in can ping each

* A slightly neutered Floodlight controller is connected to each
ovswitch. Specifically, the Forwarding component of the controller has
been removed.

(At this point, as expected, I've lost all connectivity across all
switches, as the controller doesn't install any flows by default.)

This concludes the experimental setup. The procedure follows:

1. A flow was installed in the legolas switch, directing traffic from
the local port out the GRE tunnel to sam, and vice-versa.

2. A flow was installed on the sam switch, directing traffic from
legolas's GRE tunnel to saruman's GRE tunnel, and vice-versa.

3. A flow was installed on the saruman switch, directing traffic from
sam's GRE tunnel to the local port, and vice-versa.

(This is setup by attachment xperiment_1.sh.)

My expectation would be that there should be connectivity between the
bridge interfaces on legolas and saruman, and I should be able to see
all that traffic entering and exiting the ovswitch at sam by
packet-sniffing it's LOM.

However, that is not the case. I see traffic destined for either machine
at that interface, but it never subsequently departs. (see pcap1.txt)

Weirdly, though, if I add a patch with both ends in the openvswitch on
sam, and construct the rules such that the GRE traffic is forced through
the patch cable prior to being directed to the other GRE tunnel,
everything works again. (This is setup by xperiment_2.sh, and
demonstrated by pcap2.txt.)

Last odd thing: in the "un-patch-cabled" (broken) configuration, when I
run ovs-appctl
dpif/dump-flows br0 on sam, it displays these two flows, which I can't
figure out:

root at sam:~# sudo ovs-appctl dpif/dump-flows br0
packets:1, bytes:98, used:1.412s,

packets:11, bytes:1078, used:1.068s,

If I'm reading that correctly, it's not correctly re-encapsulating
packets received on one GRE port and egressing via a different GRE port;
it's just restoring the original GRE header instead.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: xperiment_1.sh
Type: application/x-shellscript
Size: 2170 bytes
Desc: not available
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20121230/b4ab9303/attachment-0004.bin>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: legolas_config.txt
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20121230/b4ab9303/attachment-0010.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pcap1.txt
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20121230/b4ab9303/attachment-0011.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pcap2.txt
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20121230/b4ab9303/attachment-0012.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: sam_config.txt
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20121230/b4ab9303/attachment-0013.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: saruman_config.txt
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20121230/b4ab9303/attachment-0014.txt>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: xperiment_2.sh
Type: application/x-shellscript
Size: 2842 bytes
Desc: not available
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20121230/b4ab9303/attachment-0005.bin>

More information about the discuss mailing list