[ovs-discuss] combining Geneve and VXLAN tunnels

[Gerhard Stenzel]

On 11/26/2016 10:56 PM, Ben Pfaff wrote:
> On Thu, Nov 24, 2016 at 01:13:16PM +0100, Gerhard Stenzel wrote:
>> On 11/23/2016 06:57 PM, Ben Pfaff wrote:
>>> On Wed, Nov 23, 2016 at 10:35:41AM +0100, Gerhard Stenzel wrote:
>>>> On 11/22/2016 11:20 PM, Ben Pfaff wrote:
>>>>> On Thu, Nov 17, 2016 at 03:58:25PM +0100, Gerhard Stenzel wrote:
>>>>>> I have a setup with three machines with an OVS switch each. 1 and 2 are connected via VXLAN tunnel, 2 and 3 are connected via Geneve tunnel. The OVSs have each an IP address assigned. ping from 1 to 2 works and ping from 2 to 3 works.
>>>>>> But trying to ping from 1 to 3 or 3 to 1 does not work, because the packet coming via the Geneve tunnel from 3 is sent to Geneve port instead of to the VXLAN port on 1.
>>>>>>
>>>>>> tcpdump on 2 looks like this:
>>>>>>
>>>>>> IP 192.168.124.68.hp-device-disc > 192.168.124.132.6081: UDP, length 50
>>>>>> IP 192.168.124.132.37695 > 192.168.124.111.6081: UDP, length 50
>>>>>> IP 192.168.124.111 > 192.168.124.132: ICMP 192.168.124.111 udp port 6081 unreachable, length 86
>>>>>>
>>>>>> Is this working as expected or is this a bug?
>>>>>
>>>>> We don't know, because you didn't give us information like how the ports
>>>>> are set up or what you put in the flow table.
>>>>
>>>> Thanks for asking. Please let me know if you need any other details ...
>>>
>>> It seems reasonable at first glance.
>>>
>>> I'd try using "ovs-dpctl dump-flows" and "ovs-appctl ofproto/trace" to
>>> figure out what's happening to the packets that are not behaving as
>>> expected.
>>>
>>
>> Thanks for the suggestion.
>>
>> # ovs-dpctl dump-flows
>> recirc_id(0),tunnel(tun_id=0x0,src=192.168.124.68,dst=192.168.124.132,ttl=64,flags(-df-csum+key)),in_port(3),skb_mark(0),eth(src=fe:63:c1:68:b8:4f,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=10.0.0.68,tip=10.0.0.111,op=1/0xff), packets:2, bytes:84, used:2.370s, actions:2,set(tunnel(tun_id=0x0,dst=192.168.124.111,ttl=64,tp_src=3329,tp_dst=6081,flags(df|key))),1
>>
>> (Not sure, if ofproto/trace needs additional input)
> 
> I think I understand your report better now.  It seems bizarre, at the
> very least.
> 
> I guess that this is Open vSwitch 2.3.2, based on the output you've
> shown.
No, machine 2 (the one with the VXLAN and the Geneve tunnel) uses OVS 2.5.0. However, I use the ovs modules from the kernel and the kernel is 4.4 based.

> 
> Usually, you'd cut and paste the full flow information from dump-flows
> into ofproto/trace,
> e.g. "recirc_id(0),tunnel(tun_id=0x0,src=192.168.124.68,dst=192.168.124.132,ttl=64,flags(-df-csum+key)),in_port(3),skb_mark(0),eth(src=fe:63:c1:68:b8:4f,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=10.0.0.68,tip=10.0.0.111,op=1/0xff)"
> in this case.  Otherwise the ofproto/trace output will have limited
> usefulness.

(I had to restart from scratch, so ports seem to have changed)

# ovs-dpctl dump-flows      
recirc_id(0),tunnel(tun_id=0x0,src=192.168.124.68,dst=192.168.124.132,ttl=64,flags(-df-csum+key)),in_port(1),skb_mark(0),eth(src=fe:63:c1:68:b8:4f,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=10.0.0.68,tip=10.0.0.111,op=1/0xff), packets:2, bytes:84, used:0.550s, actions:3,set(tunnel(tun_id=0xa,dst=192.168.124.111,ttl=64,tp_src=3329,tp_dst=6081,flags(df|key))),2


# ovs-appctl ofproto/trace br0 in_port=1 "recirc_id(0),tunnel(tun_id=0x0,src=192.168.124.68,dst=192.168.124.132,ttl=64,flags(-df-csum+key)),in_port(1),skb_mark(0),eth(src=fe:63:c1:68:b8:4f,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=10.0.0.68,tip=10.0.0.111,op=1/0xff)"
Trailing garbage in packet data
ovs-appctl: ovs-vswitchd: server returned an error

> 
> I assume that port 1 is the datapath vxlan port?  (Run "ovs-dpctl
> show".)
> 

# ovs-dpctl show
system at ovs-system:
	lookups: hit:25 missed:4 lost:0
	flows: 0
	masks: hit:32 total:1 hit/pkt:1.10
	port 0: ovs-system (internal)
	port 1: genev_sys_6081 (geneve)
	port 2: vxlan_sys_4789 (vxlan)
	port 3: br0 (internal)