[ovs-discuss] OVS-DPDK - packets received at dpdk interface but not forwarded to VXLAN port

Avi Cohen (A) avi.cohen at huawei.com
Wed Jun 7 13:45:17 UTC 2017


Sugesh,

VNI  (now ) is set to 1000 on both sides.
The 2 machines are created on AWS
I had a configuration problem - fixed already 
Below you can see that I have a flow (marked **)  for vxlan encapsulation , but not for vxlan decap ..  why is that ?
the flow-dump on both sides are quite similar.


flow-dump from non-dpdk interfaces:
* recirc_id(0),in_port(4),eth(src=02:d7:d1:26:84:e5,dst=02:fb:9e:ce:f2:0d),eth_type(0x0800),ipv4(frag=no), packets:230, bytes:34040, used:0.402s, actions:5

** recirc_id(0),in_port(3),eth(src=fa:fc:41:d0:c5:2b,dst=c6:96:20:43:ee:ab),eth_type(0x0800),ipv4(tos=0/0x3,frag=no), packets:230, bytes:22540, used:0.402s, actions:1,tnl_push(tnl_port(2),header(size=50,type=4,eth(dst=02:fb:9e:ce:f2:0d,src=02:d7:d1:26:84:e5,dl_type=0x0800),ipv4(src=172.31.100.80,dst=172.31.100.44,proto=17,tos=0,ttl=64,frag=0x4000),udp(src=0,dst=4789,csum=0x0),vxlan(flags=0x8000000,vni=0x3e8)),out_port(4))
flow-dump from pmd on cpu core: 7

***recirc_id(0),in_port(5),eth(src=02:fb:9e:ce:f2:0d,dst=02:d7:d1:26:84:e5),eth_type(0x8100),vlan(vid=2047,pcp=0),encap(eth_type(0x0800),ipv4(frag=no)), packets:232, bytes:35264, used:0.017s, actions:4

root at ip-172-31-100-80:/home/ubuntu/openvswitch-2.6.1# utilities/ovs-appctl dpctl/show
netdev at ovs-netdev:
	lookups: hit:1056 missed:21 lost:0
	flows: 3
	port 0: ovs-netdev (tap)
	port 1: br-int (tap)
	port 2: vxlan_sys_4789 (vxlan)
	port 3: veth0
	port 4: br-phy (tap)
	port 5: dpdk0 (dpdk: configured_rx_queues=1, configured_tx_queues=4, mtu=9000, requested_rx_queues=1, requested_tx_queues=33)

> -----Original Message-----
> From: Chandran, Sugesh [mailto:sugesh.chandran at intel.com]
> Sent: Wednesday, 07 June, 2017 4:41 PM
> To: Avi Cohen (A); ovs-discuss at openvswitch.org
> Subject: RE: [ovs-discuss] OVS-DPDK - packets received at dpdk interface but
> not forwarded to VXLAN port
> 
> 
> 
> Regards
> _Sugesh
> 
> 
> > -----Original Message-----
> > From: Avi Cohen (A) [mailto:avi.cohen at huawei.com]
> > Sent: Wednesday, June 7, 2017 1:21 PM
> > To: Chandran, Sugesh <sugesh.chandran at intel.com>; ovs-
> > discuss at openvswitch.org
> > Subject: RE: [ovs-discuss] OVS-DPDK - packets received at dpdk
> > interface but not forwarded to VXLAN port
> >
> > Sugesh
> > This is the output from the remote device (the transmitter  - tries to
> > ping the other device)
> >
> > ovs-appctl dpctl/dump-flows netdev at ovs-netdev flow-dump from non- dpdk
> > interfaces:
> > recirc_id(0),in_port(4),eth(src=02:d7:d1:26:84:e5,dst=02:fb:9e:ce:f2:0
> > d),eth_ type(0x0800),ipv4(frag=no), packets:5904, bytes:543696,
> > used:0.725s,
> > actions:5
> >
> > recirc_id(0),in_port(3),eth(src=52:e9:07:36:77:ce,dst=ff:ff:ff:ff:ff:f
> > f),eth_typ e(0x0806),arp(sip=1.1.1.20,tip=1.1.1.10,op=1/0xff),
> > packets:5889, bytes:247338, used:0.725s,
> > actions:1,tnl_push(tnl_port(2),header(size=50,type=4,eth(dst=02:fb:9e:
> > ce:f2
> > :0d,src=02:d7:d1:26:84:e5,dl_type=0x0800),ipv4(src=172.31.100.80,dst=1
> > 72.3
> > 1.100.44,proto=17,tos=0,ttl=64,frag=0x4000),udp(src=0,dst=4789,csum=0x
> > 0),
> > vxlan(flags=0x8000000,vni=0x3e8)),out_port(4))
> >
> [Sugesh] It looks OK on the transmit side, However on receive side they are
> showing as QinQ.
> Are they connected back to back??
> Also  whats the vxlan key that you have configured for the tunnel interface on
> receive side?
> 
> >
> > ovs-appctl dpctl/show
> > netdev at ovs-netdev:
> >  	lookups: hit:1436 missed:33 lost:0
> >  	flows: 1
> >  	port 0: ovs-netdev (tap)
> >  	port 1: br-int (tap)
> >  	port 2: vxlan_sys_4789 (vxlan)
> >  	port 3: veth0
> >  	port 4: br-phy (tap)
> >  	port 5: dpdk0 (dpdk: configured_rx_queues=1,
> >
> >
> > > -----Original Message-----
> > > From: Avi Cohen (A)
> > > Sent: Wednesday, 07 June, 2017 2:13 PM
> > > To: 'Chandran, Sugesh'; ovs-discuss at openvswitch.org
> > > Subject: RE: [ovs-discuss] OVS-DPDK - packets received at dpdk
> > > interface but not forwarded to VXLAN port
> > >
> > > Thanks you Sugesh
> > > Correction
> > > See output below : - the last show  " ovs-appctl dpctl/dump-flows
> > netdev at ovs-
> > > netdev"  - the  action is  4   to the br-phy .
> > >
> > >
> > > ovs-appctl dpctl/dump-flows netdev at ovs-netdev flow-dump from pmd
> > on
> > > cpu
> > > core: 7
> > > recirc_id(0),in_port(5),eth(src=02:d7:d1:26:84:e5,dst=02:fb:9e:ce:f2
> > > :0
> > > d),eth_typ
> > > e(0x8100),vlan(vid=2047,pcp=0),encap(eth_type(0x0800),ipv4(frag=no))
> > > , packets:1334, bytes:128592, used:0.757s, actions:4
> > > recirc_id(0),in_port(5),eth(src=02:97:78:a7:a9:2d,dst=02:fb:9e:ce:f2
> > > :0
> > > d),eth_typ
> > > e(0x8100),vlan(vid=2047,pcp=0),encap(eth_type(0x0806),arp(op=1/0xff)
> > > ), packets:1, bytes:60, used:0.525s, actions:4 root at ip-172-31-100-
> > > 44:/home/ubuntu/openvswitch-2.6.1#
> > > root at ip-172-31-100-44:/home/ubuntu/openvswitch-2.6.1#
> > > root at ip-172-31-100-44:/home/ubuntu/openvswitch-2.6.1#
> > > root at ip-172-31-100-44:/home/ubuntu/openvswitch-2.6.1#
> > > root at ip-172-31-100-44:/home/ubuntu/openvswitch-2.6.1#
> > > root at ip-172-31-100-44:/home/ubuntu/openvswitch-2.6.1#
> > > root at ip-172-31-100-44:/home/ubuntu/openvswitch-2.6.1# ovs-appctl
> > > dpctl/dump-flows netdev at ovs-netdev flow-dump from pmd on cpu core:
> > 7
> > > recirc_id(0),in_port(5),eth(src=02:d7:d1:26:84:e5,dst=02:fb:9e:ce:f2
> > > :0
> > > d),eth_typ
> > > e(0x8100),vlan(vid=2047,pcp=0),encap(eth_type(0x0800),ipv4(frag=no))
> > > , packets:1341, bytes:129264, used:0.139s, actions:4
> > > recirc_id(0),in_port(5),eth(src=02:97:78:a7:a9:2d,dst=02:fb:9e:ce:f2
> > > :0
> > > d),eth_typ
> > > e(0x8100),vlan(vid=2047,pcp=0),encap(eth_type(0x0806),arp(op=1/0xff)
> > > ), packets:2, bytes:120, used:5.957s, actions:4 root at ip-172-31-100-
> > > 44:/home/ubuntu/openvswitch-2.6.1#
> > > root at ip-172-31-100-44:/home/ubuntu/openvswitch-2.6.1#
> > > root at ip-172-31-100-44:/home/ubuntu/openvswitch-2.6.1#
> > > root at ip-172-31-100-44:/home/ubuntu/openvswitch-2.6.1#
> > > root at ip-172-31-100-44:/home/ubuntu/openvswitch-2.6.1# ovs-appctl
> > > dpctl/show
> > > netdev at ovs-netdev:
> > > 	lookups: hit:1436 missed:33 lost:0
> > > 	flows: 1
> > > 	port 0: ovs-netdev (tap)
> > > 	port 1: br-int (tap)
> > > 	port 2: vxlan_sys_4789 (vxlan)
> > > 	port 3: veth0
> > > 	port 4: br-phy (tap)
> > > 	port 5: dpdk0 (dpdk: configured_rx_queues=1,
> > configured_tx_queues=4,
> > > mtu=9000, requested_rx_queues=1,
> > > requested_tx_queues=33)
> > >
> > > root at ip-172-31-100-44:/home/ubuntu/openvswitch-2.6.1# ovs-appctl
> > > dpctl/dump-flows netdev at ovs-netdev flow-dump from pmd on cpu core:
> > 7
> > > recirc_id(0),in_port(5),eth(src=02:d7:d1:26:84:e5,dst=02:fb:9e:ce:f2
> > > :0
> > > d),eth_typ
> > > e(0x8100),vlan(vid=2047,pcp=0),encap(eth_type(0x0800),ipv4(frag=no))
> > > , packets:1415, bytes:136368, used:0.317s, actions:4
> > > root at ip-172-31-100- 44:/home/ubuntu/openvswitch-2.6.1#
> > >
> > > > -----Original Message-----
> > > > From: Chandran, Sugesh [mailto:sugesh.chandran at intel.com]
> > > > Sent: Wednesday, 07 June, 2017 1:32 PM
> > > > To: Avi Cohen (A); ovs-discuss at openvswitch.org
> > > > Subject: RE: [ovs-discuss] OVS-DPDK - packets received at dpdk
> > > > interface but not forwarded to VXLAN port
> > > >
> > > >
> > > >
> > > > Regards
> > > > _Sugesh
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Avi Cohen (A) [mailto:avi.cohen at huawei.com]
> > > > > Sent: Wednesday, June 7, 2017 11:27 AM
> > > > > To: Chandran, Sugesh <sugesh.chandran at intel.com>; ovs-
> > > > > discuss at openvswitch.org
> > > > > Subject: RE: [ovs-discuss] OVS-DPDK - packets received at dpdk
> > > > > interface but not forwarded to VXLAN port
> > > > >
> > > > > Sugesh
> > > > > Dpctl dump-dps - show nothing - I think this make sense because
> > > > > we are in
> > > > [Sugesh] did U try ovs-appctl dpctl/dump-flows netdev at ovs-netdev?
> > > > Can you have a look at
> > > > Ovs-appctl dpctl/show
> > > > Ovs-appctl dpctl/dump-flows netdev at ovs-netdev
> > > >
> > > > > user-space only Ofctl dump-flows - show 1 NORMAL entry for both
> > > > > bridges Bset Regards avi
> > > > >
> > > > > -----Original Message-----
> > > > > From: Chandran, Sugesh [mailto:sugesh.chandran at intel.com]
> > > > > Sent: Wednesday, 07 June, 2017 11:59 AM
> > > > > To: Avi Cohen (A); ovs-discuss at openvswitch.org
> > > > > Subject: RE: [ovs-discuss] OVS-DPDK - packets received at dpdk
> > > > > interface but not forwarded to VXLAN port
> > > > >
> > > > >
> > > > >
> > > > > Regards
> > > > > _Sugesh
> > > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Avi Cohen (A) [mailto:avi.cohen at huawei.com]
> > > > > > Sent: Wednesday, June 7, 2017 9:26 AM
> > > > > > To: Chandran, Sugesh <sugesh.chandran at intel.com>; ovs-
> > > > > > discuss at openvswitch.org
> > > > > > Subject: RE: [ovs-discuss] OVS-DPDK - packets received at dpdk
> > > > > > interface but not forwarded to VXLAN port
> > > > > >
> > > > > > Thank you Sugesh
> > > > > > The problem is not an  arp/route  issue.
> > > > > > The problem is that Packets received in the peer dpdk  not
> > > > > > forwarded to the vxlan port for decap.
> > > > > > When I tried to investigate it - I saw that in ovs-dpdk setup
> > > > > > the outer header received at the peer dpdk interface is - for
> > > > > > some reason including vlan  2047, (on the other hand in native
> > > > > > ovs setup the outer eth header is untagged) I think that the
> > > > > > packet is dropped there - although I don't see any drops in
> > > > > > dpdk0 statistics.
> > > > > [Sugesh] Bit strange to me, I haven't seen anything as such in my setup.
> > > > > Just wondering what is your ofproto and datapath flows looks like?
> > > > > >
> > > > > > Output of vsctl show:
> > > > > >
> > > > > > utilities/ovs-vsctl show
> > > > > > 8cdae740-9eb3-4b79-ad6f-bca0652c5d54
> > > > > >     Bridge br-int
> > > > > >         fail_mode: standalone
> > > > > >         Port "vxlan0"
> > > > > >             Interface "vxlan0"
> > > > > >                 type: vxlan
> > > > > >                 options: {remote_ip="172.31.100.80"}
> > > > > >         Port "veth0"
> > > > > >             Interface "veth0"
> > > > > >         Port br-int
> > > > > >             Interface br-int
> > > > > >                 type: internal
> > > > > >     Bridge br-phy
> > > > > >         fail_mode: standalone
> > > > > >         Port "dpdk0"
> > > > > >             Interface "dpdk0"
> > > > > >                 type: dpdk
> > > > > >                 options: {dpdk-devargs="0000:00:03.0"}
> > > > > >         Port br-phy
> > > > > >             Interface br-phy
> > > > > >                 type: internal
> > > > > >
> > > > > >
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Chandran, Sugesh [mailto:sugesh.chandran at intel.com]
> > > > > > > Sent: Wednesday, 07 June, 2017 11:06 AM
> > > > > > > To: Avi Cohen (A); ovs-discuss at openvswitch.org
> > > > > > > Subject: RE: [ovs-discuss] OVS-DPDK - packets received at
> > > > > > > dpdk interface but not forwarded to VXLAN port
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Regards
> > > > > > > _Sugesh
> > > > > > >
> > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: ovs-discuss-bounces at openvswitch.org
> > > > > > > > [mailto:ovs-discuss- bounces at openvswitch.org] On Behalf Of
> > > > > > > > Avi Cohen (A)
> > > > > > > > Sent: Wednesday, June 7, 2017 7:42 AM
> > > > > > > > To: ovs-discuss at openvswitch.org
> > > > > > > > Subject: [ovs-discuss] OVS-DPDK - packets received at dpdk
> > > > > > > > interface but not forwarded to VXLAN port
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > When I  work with native OVS (no DPDK)  - the traffic is OK.
> > > > > > > > I see that with native OVS - I have NO VLAN in the outer
> > > > > > > > Ethernet header
> > > > > > .
> > > > > > > [Sugesh] I didn't understand the issue exactly..Is your
> > > > > > > traffic not forwarding at all Or Is it have wrong vlan tag.?
> > > > > > > If its issue with the traffic forwarding, it must be
> > > > > > > something to do with arp/route.
> > > > > > > Can you please have a look at datapath rules , arp and route
> > > > > > > entries in
> > > > > > ovs.??
> > > > > > > You can refer the following document for debug commands
> > > > > > >
> > > > > > > http://docs.openvswitch.org/en/latest/howto/userspace-
> > > > > > > tunneling/?highlight=tunneling
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > in OVS-DPDK - for some reason -   I do have VLAN 2047 - see
> > below
> > > ###
> > > > > > > >
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: Avi Cohen (A)
> > > > > > > > > Sent: Tuesday, 06 June, 2017 1:58 PM
> > > > > > > > > To: 'dpdk-ovs at lists.01.org'; 'users at dpdk.org'
> > > > > > > > > Subject: OVS-DPDK - packets received at dpdk interface
> > > > > > > > > but not forwarded to VXLAN port
> > > > > > > > >
> > > > > > > > > Hello All (I resend because the prev msg was sent as
> > > > > > > > > html)
> > > > > > > > >
> > > > > > > > > I have 2  OVS-DPDK installed  - with the following below
> > > > > > > > > symmetric
> > > > > > > > > configuration: see ***
> > > > > > > > >
> > > > > > > > > When I send ping from a namespace (connected to the
> > > > > > > > > ovs-dpdk)
> > > > > > > > > - I see that the packet is received at the peer  via the
> > > > > > > > > dpdk interface
> > > > > > > > > - BUT not sent to the vxlan port
> > > > > > > > >
> > > > > > > > > See  ###  the tcpdump of the packet (captured at the
> > > > > > > > > br-phy in the
> > > > > > > > > peer) - the packet  looks  OK but  not sent to vxlan
> > > > > > > > > port  for
> > decap.
> > > > > > > > >
> > > > > > > > > [note that in the TX direction - the behavior is OK -
> > > > > > > > > packet sent from namespace to veth0 then to vxlan encap,
> > > > > > > > > then to dpdk interface
> > > > > > > > > --> eventually the packet is received at the dpdk peer
> > > > > > > > > --> interface]
> > > > > > > > >
> > > > > > > > > Can someone see any problem with my configuration ?
> > > > > > > > >
> > > > > > > > > Best Regards
> > > > > > > > > avi
> > > > > > > > >
> > > > > > > > > ***
> > > > > > > > > utilities/ovs-vsctl show
> > > > > > > > > 8cdae740-9eb3-4b79-ad6f-bca0652c5d54
> > > > > > > > >     Bridge br-int
> > > > > > > > >         fail_mode: standalone
> > > > > > > > >         Port "vxlan0"
> > > > > > > > >             Interface "vxlan0"
> > > > > > > > >                 type: vxlan
> > > > > > > > >                 options: {remote_ip="172.31.100.80"}                // the
> > peer
> > > > has
> > > > > > > other
> > > > > > > > > ip
> > > > > > > > >         Port "veth0"
> > > > > > > > >             Interface "veth0"
> > > > > > > > >         Port br-int
> > > > > > > > >             Interface br-int
> > > > > > > > >                 type: internal
> > > > > > > > >     Bridge br-phy
> > > > > > > > >         fail_mode: standalone
> > > > > > > > >         Port "dpdk0"
> > > > > > > > >             Interface "dpdk0"
> > > > > > > > >                 type: dpdk
> > > > > > > > >                 options: {dpdk-devargs="0000:00:03.0"}
> > > > > > > > >         Port br-phy
> > > > > > > > >             Interface br-phy
> > > > > > > > >                 type: internal
> > > > > > > > >
> > > > > > > > > ###
> > > > > > > > > 10:27:15.201711 02:d7:d1:26:84:e5 (oui Unknown) >
> > > > > > > > > 02:fb:9e:ce:f2:0d (oui Unknown), ethertype 802.1Q
> > > > > > > > > (0x8100), length
> > > > > > > > > 152: vlan 2047, p 0, ethertype IPv4,
> > > > > > > > > ip-172-31-100-80.eu-central-1.compute.internal.58740 >
> > > > > > > > > ip-172-31-100-
> > > > > > > > > 44.eu-central-1.compute.internal.4789: VXLAN, flags [I]
> > > > > > > > > (0x08), vni
> > > > > > > > > 0
> > > > > > > > > 7a:fd:3c:4c:ee:d3 (oui Unknown) > d2:ea:96:e1:ef:d2 (oui
> > > > > > > > > Unknown), ethertype
> > > > > > > > > IPv4 (0x0800), length 98: 1.1.1.20 > 1.1.1.10: ICMP echo
> > > > > > > > > request, id 2235, seq 1176, length 64
> > > > > > > >
> > > > > > > > _______________________________________________
> > > > > > > > discuss mailing list
> > > > > > > > discuss at openvswitch.org
> > > > > > > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss


More information about the discuss mailing list