[ovs-discuss] one patch was omitted to be pushed to branch-2.3---datapath: Fix recirc bug where skb is double freed

ZHANG Zhiming zhangzhiming at yunshan.net.cn
Wed May 27 03:34:41 UTC 2015


Hi Andy,

Could you please tell me why the bond needs recirculation?

Thanks!


Switch configurations:

interface eth-0-7
 jumboframe enable
 ip arp inspection trust
 switchport mode trunk
 switchport trunk allowed vlan add 10-2046
 channel-group 1 mode active

interface eth-0-19
 jumboframe enable
 ip arp inspection trust
 switchport mode trunk
 switchport trunk allowed vlan add 10-2046
 channel-group 1 mode active

interface agg1
 jumboframe enable
 switchport mode trunk
 switchport trunk allowed vlan add 10-2046

Before setting lacp=active:

[root at centos125 ~]# ovs-vsctl -- find port name=bond-eth24-eth25
_uuid               : 3c2ef220-5655-4b35-b0b0-d7de3e9250de
bond_active_slave   : "00:1b:21:bb:22:4c"
bond_downdelay      : 0
bond_fake_iface     : false
bond_mode           : balance-tcp
bond_updelay        : 0
external_ids        : {}
fake_bridge         : false
interfaces          : [49f7116a-79af-42dc-a803-cb71af2aa8a3, a8949d8e-666c-44a1-8634-55f72fe9478d]
lacp                : []
mac                 : []
name                : "bond-eth24-eth25"
other_config        : {}
qos                 : []
statistics          : {}
status              : {}
tag                 : []
trunks              : []
vlan_mode           : []
[root at centos125 ~]# ovs-appctl bond/show                        
---- bond-eth24-eth25 ----
bond_mode: balance-tcp
bond may use recirculation: yes, Recirc-ID : 301
bond-hash-basis: 0
updelay: 0 ms
downdelay: 0 ms
next rebalance: 64 ms
lacp_status: off
active slave mac: 00:1b:21:bb:22:4c(eth24)

slave eth24: enabled
        active slave
        may_enable: true

slave eth25: enabled
        may_enable: true
[root at centos125 ~]# ovs-dpctl dump-flows
recirc_id(0),skb_priority(0),in_port(1),eth(src=86:47:ee:35:8a:e6,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=172.16.37.1/255.255.255.255,tip=172.16.1.120/255.255.255.255,op=1/0xff,sha=86:47:ee:35:8a:e6/00:00:00:00:00:00,tha=00:00:00:00:00:00/00:00:00:00:00:00), packets:0, bytes:0, used:never, actions:2
recirc_id(0),skb_priority(0),in_port(1),eth(src=7e:51:5f:37:e0:58,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=172.16.39.2/255.255.255.255,tip=172.16.79.128/255.255.255.255,op=1/0xff,sha=7e:51:5f:37:e0:58/00:00:00:00:00:00,tha=00:00:00:00:00:00/00:00:00:00:00:00), packets:48123, bytes:2887380, used:0.249s, actions:2
recirc_id(0),skb_priority(0),in_port(1),eth(src=02:43:5a:97:75:a6,dst=f8:b1:56:de:3c:03),eth_type(0x0800),ipv4(src=172.16.37.32/0.0.0.0,dst=172.16.1.125/0.0.0.0,proto=6/0,tos=0x10/0,ttl=64/0,frag=no/0xff), packets:11, bytes:966, used:0.001s, flags:P., actions:2
recirc_id(0),skb_priority(0),in_port(1),eth(src=e4:68:a3:97:9b:f0,dst=01:80:c2:00:00:00),eth_type(0/0xffff), packets:0, bytes:0, used:never, actions:drop
recirc_id(0),skb_priority(0),in_port(1),eth(src=78:2b:cb:47:e4:4c,dst=78:2b:cb:79:8a:e2),eth_type(0x0806), packets:1, bytes:60, used:0.978s, actions:2
recirc_id(0),skb_priority(0),in_port(1),eth(src=b8:ca:3a:b0:62:a1,dst=01:00:5e:7f:01:d2),eth_type(0x0800),ipv4(src=172.16.1.119/0.0.0.0,dst=239.255.1.210/0.0.0.0,proto=17/0,tos=0/0,ttl=1/0,frag=no/0xff), packets:35193, bytes:5666073, used:0.086s, actions:2
recirc_id(0),skb_priority(0),in_port(1),eth(src=78:2b:cb:76:e5:9f,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=10.33.23.66/0.0.0.0,tip=10.33.23.66/0.0.0.0,op=2/0xff,sha=78:2b:cb:76:e5:9f/00:00:00:00:00:00,tha=00:00:00:00:00:00/00:00:00:00:00:00), packets:0, bytes:0, used:never, actions:2
recirc_id(0),skb_priority(0x6),in_port(2),eth(src=f8:b1:56:de:3c:03,dst=02:43:5a:97:75:a6),eth_type(0x0800),ipv4(src=172.16.1.125/0.0.0.0,dst=172.16.37.32/0.0.0.0,proto=6/0,tos=0x10/0,ttl=64/0,frag=no/0xff), packets:5, bytes:634, used:0.002s, flags:P., actions:1


After setting lacp=active and configure an IP address to the bridge of bond-eth24-eth25, the server rebooted immediately.
[root at centos125 ~]# ifconfig nspbr1 192.168.1.125/16 

The outputs of another console:
[root at centos125 ~]# watch -n 1 "ovs-dpctl dump-flows" 
recirc_id(0),skb_priority(0),in_port(5),eth(src=52:54:00:81:7f:38,dst=ff:ff:ff:ff:ff:ff),eth_type(0x8100),vlan(vid=41/0xfff,pcp=0/0x0,cfi=1/1),encap(eth_type(0x0806),ar
p(sip=10.10.3.10/255.255.255.255,tip=10.10.3.1/255.255.255.255,op=1/0xff,sha=52:54:00:81:7f:38/00:00:00:00:00:00,tha=00:00:00:00:00:00/00:00:00:00:00:00)), packets:0, b
ytes:0, used:never, actions:drop
recirc_id(0),skb_priority(0),in_port(1),eth(src=78:2b:cb:48:41:55,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=172.16.1.107/255.255.255.255,tip=172.16.148.0/255.255.
255.255,op=1/0xff,sha=78:2b:cb:48:41:55/00:00:00:00:00:00,tha=00:00:00:00:00:00/00:00:00:00:00:00), packets:0, bytes:0, used:never, actions:2
recirc_id(0),skb_priority(0),in_port(6),eth(src=78:2b:cb:79:8a:e2,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=192.168.0.1/255.255.255.255,tip=192.168.137.172/255.25
5.255.255,op=1/0xff,sha=78:2b:cb:79:8a:e2/00:00:00:00:00:00,tha=00:00:00:00:00:00/00:00:00:00:00:00), packets:0, bytes:0, used:never, actions:7
recirc_id(0),skb_priority(0),in_port(1),eth(src=7e:51:5f:37:e0:58,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=172.16.39.2/255.255.255.255,tip=172.16.79.128/255.255.
255.255,op=1/0xff,sha=7e:51:5f:37:e0:58/00:00:00:00:00:00,tha=00:00:00:00:00:00/00:00:00:00:00:00), packets:48325, bytes:2899500, used:0.370s, actions:2
recirc_id(0),skb_priority(0),in_port(1),eth(src=02:43:5a:97:75:a6,dst=f8:b1:56:de:3c:03),eth_type(0x0800),ipv4(src=172.16.37.32/0.0.0.0,dst=172.16.1.125/0.0.0.0,proto=6
/0,tos=0x10/0,ttl=64/0,frag=no/0xff), packets:99, bytes:9723, used:1.002s, flags:SFP., actions:2
recirc_id(0),skb_priority(0),in_port(1),eth(src=be:6e:e7:c1:4a:03,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=172.16.41.2/255.255.255.255,tip=172.16.44.253/255.255.
255.255,op=1/0xff,sha=be:6e:e7:c1:4a:03/00:00:00:00:00:00,tha=00:00:00:00:00:00/00:00:00:00:00:00), packets:2, bytes:120, used:0.379s, actions:2
recirc_id(0),skb_priority(0),in_port(1),eth(src=d4:ae:52:a2:56:e8,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=172.17.1.104/0.0.0.0,tip=172.17.1.104/0.0.0.0,op=2/0xf
f,sha=d4:ae:52:a2:56:e8/00:00:00:00:00:00,tha=00:00:00:00:00:00/00:00:00:00:00:00), packets:0, bytes:0, used:never, actions:2
recirc_id(0),skb_priority(0),in_port(6),eth(src=00:1c:73:69:7a:0f,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=192.168.22.111/255.255.255.255,tip=192.168.1.125/255.2
55.255.255,op=1/0xff,sha=00:1c:73:69:7a:0f/00:00:00:00:00:00,tha=00:00:00:00:00:00/00:00:00:00:00:00), packets:2, bytes:120, used:0.918s, actions:7
recirc_id(0),skb_priority(0),in_port(1),eth(src=e4:68:a3:97:9b:f0,dst=01:80:c2:00:00:00),eth_type(0/0xffff), packets:0, bytes:0, used:never, actions:drop
recirc_id(0),skb_priority(0),in_port(1),eth(src=78:2b:cb:48:41:55,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=172.16.1.107/255.255.255.255,tip=172.16.148.9/255.255.
255.255,op=1/0xff,sha=78:2b:cb:48:41:55/00:00:00:00:00:00,tha=00:00:00:00:00:00/00:00:00:00:00:00), packets:0, bytes:0, used:never, actions:2    
recirc_id(0),skb_priority(0),in_port(1),eth(src=b8:ca:3a:b0:62:a1,dst=01:00:5e:7f:01:d2),eth_type(0x0800),ipv4(src=172.16.1.119/0.0.0.0,dst=239.255.1.210/0.0.0.0,proto=
17/0,tos=0/0,ttl=1/0,frag=no/0xff), packets:35340, bytes:5689740, used:1.089s, actions:2
recirc_id(0),skb_priority(0x6),in_port(2),eth(src=f8:b1:56:de:3c:03,dst=02:43:5a:97:75:a6),eth_type(0x0800),ipv4(src=172.16.1.125/0.0.0.0,dst=172.16.37.32/0.0.0.0,proto
=6/0,tos=0x10/0,ttl=64/0,frag=no/0xff), packets:50, bytes:24916, used:1.002s, flags:P., actions:1







ZHANG Zhiming
Yunshan Networks

From: Andy Zhou
Date: 2015-05-27 10:13
To: zhangzhiming
CC: Alex Wang; discuss
Subject: Re: Re: [ovs-discuss] one patch was omitted to be pushed to branch-2.3---datapath: Fix recirc bug where skb is double freed
On Sun, May 24, 2015 at 1:46 AM, ZHANG Zhiming
<zhangzhiming at yunshan.net.cn> wrote:
> Hi,
>
>
> Here you are.
> This bug is triggered by configuring bond_mode=banlace-tcp and lacp=active
> simultaneously.
> The version of OVS is 2.3.1.
>
>

Thanks, this helps. The backtrace suggested there is a stack
corruption from 'execute_recirc()',  But it not
obvious what is the problem.  I will need to think about it more. In
the mean time, would you please post
a snap shot of the datapath flows,  using the 'ovs-dpctl dump-flows' command.

Thanks,

Andy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20150527/d0c79a2d/attachment-0002.html>


More information about the discuss mailing list