[ovs-discuss] [ovs-dev] packet drop
Tony Liu
tonyliu0592 at hotmail.com
Thu Aug 6 18:39:34 UTC 2020
Inline...
Thanks!
Tony
> -----Original Message-----
> From: Numan Siddique <numans at ovn.org>
> Sent: Thursday, August 6, 2020 10:03 AM
> To: Tony Liu <tonyliu0592 at hotmail.com>
> Cc: ovs-discuss at openvswitch.org; ovs-dev at openvswitch.org
> Subject: Re: [ovs-dev] packet drop
>
>
>
> On Thu, Aug 6, 2020 at 4:05 AM Tony Liu <tonyliu0592 at hotmail.com
> <mailto:tonyliu0592 at hotmail.com> > wrote:
>
>
>
> The drop is caused by flow change.
>
> When packet is dropped.
> ========
> recirc_id(0),tunnel(tun_id=0x19aca,src=10.6.30.92,dst=10.6.30.22,ge
> neve({class=0x102,type=0x80,len=4,0x20003/0x7fffffff}),flags(-
> df+csum+key)),in_port(3),eth(src=fa:16:3e:df:1e:85,dst=00:00:00:00:00:00
> /01:00:00:00:00:00),eth_type(0x0800),ipv4(proto=1,frag=no),icmp(type=8/0
> xf8), packets:14, bytes:1372, used:0.846s, actions:drop
> recirc_id(0),in_port(12),eth(src=fa:16:3e:7d:bb:85,dst=fa:16:3e:df:
> 1e:85),eth_type(0x0800),ipv4(src=192.168.236.152/255.255.255.252,dst=10.
> 6.40.9,proto=1,tos=0/0x3,ttl=64,frag=no
> <http://192.168.236.152/255.255.255.252,dst=10.6.40.9,proto=1,tos=0/0x3,
> ttl=64,frag=no> ),icmp(type=0), packets:6, bytes:588, used:8.983s,
> actions:drop
> ========
>
> When packet goes through.
> ========
> recirc_id(0),tunnel(tun_id=0x19aca,src=10.6.30.92,dst=10.6.30.22,ge
> neve({class=0x102,type=0x80,len=4,0x20003/0x7fffffff}),flags(-
> df+csum+key)),in_port(3),eth(src=fa:16:3e:df:1e:85,dst=00:00:00:00:00:00
> /01:00:00:00:00:00),eth_type(0x0800),ipv4(proto=1,frag=no),icmp(type=8/0
> xf8), packets:3, bytes:294, used:0.104s, actions:12
> recirc_id(0),in_port(12),eth(src=fa:16:3e:7d:bb:85,dst=fa:16:3e:df:
> 1e:85),eth_type(0x0800),ipv4(src=192.168.236.152/255.255.255.252,dst=10.
> 6.40.9,proto=1,tos=0/0x3,ttl=64,frag=no
> <http://192.168.236.152/255.255.255.252,dst=10.6.40.9,proto=1,tos=0/0x3,
> ttl=64,frag=no> ),icmp(type=0), packets:3, bytes:294, used:0.103s,
> actions:ct_clear,set(tunnel(tun_id=0x1a8ee,dst=10.6.30.92,ttl=64,tp_dst=
> 6081,geneve({class=0x102,type=0x80,len=4,0x1000b}),flags(df|csum|key))),
> set(eth(src=fa:16:3e:75:b7:e5,dst=52:54:00:0c:ef:b9)),set(ipv4(ttl=63)),
> 3
> ========
>
> Is that flow programmed by ovn-controller via ovs-vswitchd?
>
> What version of OVN and OVS are you using ?
ovn-20.03.0-2.el8.x86_64
openvswitch-2.12.0-1.el8.x86_64
> Can you share your OVN NB DB ?
Yes, I can. Let me know how.
> If I understand correctly the packet is received from the patch port to
> br-int on the gateway node and then tunnelled to the compute node right ?
> And the packet is dropped on the compute node ?
Yes and yes.
> If you could share your NB DB if it's fine with you and tell the
> destination logical port, I can try it out locally.
Here is what I had.
On compute node, ovn-controller is very busy. It keeps saying
"commit failed".
====================
2020-08-05T02:44:23.927Z|04125|reconnect|INFO|tcp:10.6.20.84:6642: connected
2020-08-05T02:44:23.936Z|04126|main|INFO|OVNSB commit failed, force recompute next time.
2020-08-05T02:44:23.938Z|04127|ovsdb_idl|INFO|tcp:10.6.20.84:6642: clustered database server is disconnected from cluster; trying another server
2020-08-05T02:44:23.939Z|04128|reconnect|INFO|tcp:10.6.20.84:6642: connection attempt timed out
2020-08-05T02:44:23.939Z|04129|reconnect|INFO|tcp:10.6.20.84:6642: waiting 2 seconds before reconnect
====================
The connection to local OVSDB keeps being dropped, because no probe
response.
========
2020-08-05T02:47:15.437Z|04351|poll_loop|INFO|wakeup due to [POLLIN] on fd 20 (10.6.20.22:42362<->10.6.20.86:6642) at lib/stream-fd.c:157 (100% CPU usage)
2020-08-05T02:47:15.438Z|04352|reconnect|WARN|tcp:127.0.0.1:6640: connection dropped (Broken pipe)
2020-08-05T02:47:15.438Z|04353|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connecting...
2020-08-05T02:47:15.449Z|04354|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connected
========
After set probe interval to 30s, the problem seems gone. I mentioned
this in another thread "[OVN] no response to inactivity probe".
I can restore probe interval back to default 5s, see if the problem
can be reproduced. For me, it's important to understand what happens
behind it.
>
> Thanks
> Numan
>
>
>
>
>
> Thanks!
>
> Tony
>
> > -----Original Message-----
> > From: discuss <ovs-discuss-bounces at openvswitch.org <mailto:ovs-
> discuss-bounces at openvswitch.org> > On Behalf Of Tony
> > Liu
> > Sent: Wednesday, August 5, 2020 2:48 PM
> > To: ovs-discuss at openvswitch.org <mailto:ovs-
> discuss at openvswitch.org> ; ovs-dev at openvswitch.org <mailto:ovs-
> dev at openvswitch.org>
> > Subject: [ovs-discuss] packet drop
> >
> > Hi,
> >
> > I am running ping from external to VM via OVN gateway.
> > On the compute node, ICMP request packet is consistently coming
> into
> > interface "ovn-gatewa-1". But there is about 10 out of 25 packet
> loss on
> > tap interface. It's like the switch pauses 10s after every 15s.
> >
> > Has anyone experiences such issue?
> > Any advice how to look into it?
> >
> > ========
> > 21fed09f-909e-4efc-b117-f5d5fcb636c9
> > Bridge br-int
> > fail_mode: secure
> > datapath_type: system
> > Port "ovn-gatewa-0"
> > Interface "ovn-gatewa-0"
> > type: geneve
> > options: {csum="true", key=flow,
> remote_ip="10.6.30.91"}
> > bfd_status: {diagnostic="No Diagnostic",
> flap_count="1",
> > forwarding="true", remote_diagnostic="No Diagnostic",
> remote_state=up,
> > state=up}
> > Port "tap2588bb4e-35"
> > Interface "tap2588bb4e-35"
> > Port "ovn-gatewa-1"
> > Interface "ovn-gatewa-1"
> > type: geneve
> > options: {csum="true", key=flow,
> remote_ip="10.6.30.92"}
> > bfd_status: {diagnostic="No Diagnostic",
> flap_count="1",
> > forwarding="true", remote_diagnostic="No Diagnostic",
> remote_state=up,
> > state=up}
> > Port "tap37f6b2d7-cc"
> > Interface "tap37f6b2d7-cc"
> > Port "tap2c4b3b0f-8b"
> > Interface "tap2c4b3b0f-8b"
> > Port "tap23245491-a4"
> > Interface "tap23245491-a4"
> > Port "tap51660269-2c"
> > Interface "tap51660269-2c"
> > Port "tap276cd1ef-e1"
> > Interface "tap276cd1ef-e1"
> > Port "tap138526d3-b3"
> > Interface "tap138526d3-b3"
> > Port "tapd1ae48a1-2d"
> > Interface "tapd1ae48a1-2d"
> > Port br-int
> > Interface br-int
> > type: internal
> > Port "tapdd08f476-94"
> > Interface "tapdd08f476-94"
> > ========
> >
> >
> > Thanks!
> >
> > Tony
> >
> > _______________________________________________
> > discuss mailing list
> > discuss at openvswitch.org <mailto:discuss at openvswitch.org>
> > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
> _______________________________________________
> dev mailing list
> dev at openvswitch.org <mailto:dev at openvswitch.org>
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
>
More information about the discuss
mailing list