[ovs-discuss] Openvswitch first packets losts ?
Sébastien Riccio
sr at swisscenter.com
Wed Jul 6 03:40:04 UTC 2011
On 05.07.2011 21:04, Sébastien Riccio wrote:
>
> On 05.07.2011 20:46, Justin Pettit wrote:
>> Most of the packet processing intelligence happens in user-space, and
>> the kernel module maintains a cache of the active flows. Flows are
>> expired from the cache after five seconds of inactivity. My guess is
>> that there is some sort of delay in the user-space processing. I
>> have not heard of such a long delay. Are you running an off-box
>> controller through OpenFlow? Is there a lot of load that is
>> preventing ovs-vswitchd from running? What is the output of
>> "ovs-vsctl list bridge" and "ovs-ofctl dump-flows<br>" for the
>> various bridges?
>>
>> --Justin
>>
>>
>> On Jul 5, 2011, at 12:18 AM, Sébastien Riccio wrote:
>>
>>> Hi,
>>>
>>> Being new to openvswitch I'm still trying to get my setup working
>>> like a charm.
>>> I'm close to reach my goal but I've run into an annoying problem.
>>>
>>> When there is no activity on a bridge openvswitch miss the first
>>> packets
>>> that are sent from or to the host, and I have no idea why :(
>>>
>>> Distrib: Debian 6
>>> Kernel: 2.6.32-35
>>>
>>> hardware:
>>> eth0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709S
>>> Gigabit Ethernet (rev 20)
>>> eth1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709S
>>> Gigabit Ethernet (rev 20)
>>> eth2 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709S
>>> Gigabit Ethernet (rev 20)
>>> eth3 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709S
>>> Gigabit Ethernet (rev 20)
>>> eth4 Ethernet controller: Broadcom Corporation NetXtreme II BCM57711
>>> 10-Gigabit PCIe
>>> eth5 Ethernet controller: Broadcom Corporation NetXtreme II BCM57711
>>> 10-Gigabit PCIe
>>>
>>>
>>> PIFs:
>>>
>>> eth0 trunk port
>>> eth1 trunk port
>>> eth2 iscsi path2
>>> eth3 iscsi path1
>>> eth4 unused yet yet
>>> eth5 unused yet
>>>
>>>
>>> What I did to setup networking:
>>>
>>> ifconfig eth0 up
>>> ifconfig eth1 up
>>> ifconfig eth2 up
>>> ifconfig eth3 up
>>>
>>> ovs-vsctl add-br trunk0
>>> ovs-vsctl add-bond trunk0 bond0 eth0 eth1
>>> ovs-vsctl add-port trunk0 mgmt0 tag=85
>>> ovs-vsctl set internface mgmt0 type=internal
>>>
>>> ovs-vsctl add-br stor0
>>> ovs-vsctl add-port stor0 eth3
>>>
>>> ovs-vsctl add-br stor1
>>> ovs-vsctl add-port stor1 eth2
>>>
>>> ifconfig mgmt0 10.111.10.116 netmask 255.255.0.0
>>> ifconfig stor0 10.50.50.16 netmask 255.255.255.0
>>> ifconfig stor1 10.50.60.16 netmask 255.255.255.0
>>>
>>>
>>>
>>> Testing bonded trunk:
>>>
>>> root at xen-blade16:~# ping 10.111.0.1
>>> PING 10.111.0.1 (10.111.0.1) 56(84) bytes of data.
>>> 64 bytes from 10.111.0.1: icmp_req=3 ttl=255 time=0.718 ms
>>> 64 bytes from 10.111.0.1: icmp_req=4 ttl=255 time=0.426 ms
>>> 64 bytes from 10.111.0.1: icmp_req=5 ttl=255 time=0.300 ms
>>> 64 bytes from 10.111.0.1: icmp_req=6 ttl=255 time=0.356 ms
>>> 64 bytes from 10.111.0.1: icmp_req=7 ttl=255 time=0.390 ms
>>> 64 bytes from 10.111.0.1: icmp_req=8 ttl=255 time=0.346 ms
>>> ^C
>>> --- 10.111.0.1 ping statistics ---
>>> 8 packets transmitted, 6 received, 25% packet loss, time 7014ms
>>> rtt min/avg/max/mdev = 0.300/0.422/0.718/0.139 ms
>>>
>>>
>>> Testing iscsi path1
>>>
>>> root at xen-blade16:~# ping 10.50.50.15
>>> PING 10.50.50.15 (10.50.50.15) 56(84) bytes of data.
>>> 64 bytes from 10.50.50.15: icmp_req=4 ttl=64 time=0.179 ms
>>> 64 bytes from 10.50.50.15: icmp_req=5 ttl=64 time=0.181 ms
>>> 64 bytes from 10.50.50.15: icmp_req=6 ttl=64 time=0.150 ms
>>> 64 bytes from 10.50.50.15: icmp_req=7 ttl=64 time=0.164 ms
>>> 64 bytes from 10.50.50.15: icmp_req=8 ttl=64 time=0.177 ms
>>> ^C
>>> --- 10.50.50.15 ping statistics ---
>>> 8 packets transmitted, 5 received, 37% packet loss, time 7022ms
>>> rtt min/avg/max/mdev = 0.150/0.170/0.181/0.014 ms
>>>
>>>
>>> Testing iscsi path2
>>>
>>> root at xen-blade16:~# ping 10.50.60.15
>>> PING 10.50.60.15 (10.50.60.15) 56(84) bytes of data.
>>> 64 bytes from 10.50.60.15: icmp_req=4 ttl=64 time=0.163 ms
>>> 64 bytes from 10.50.60.15: icmp_req=5 ttl=64 time=0.161 ms
>>> 64 bytes from 10.50.60.15: icmp_req=6 ttl=64 time=0.165 ms
>>> 64 bytes from 10.50.60.15: icmp_req=7 ttl=64 time=0.168 ms
>>> 64 bytes from 10.50.60.15: icmp_req=8 ttl=64 time=0.154 ms
>>> ^C
>>> --- 10.50.60.15 ping statistics ---
>>> 8 packets transmitted, 5 received, 37% packet loss, time 7026ms
>>>
>>>
>>>
>>> As you can see the first packets are dropped, either on the bonded
>>> interface and on the "normal" interfaces. So doesn't seems to be
>>> an issue with the bonding.
>>>
>>> Also if I ping, ctrl-c and then directly ping again, it doesn't miss
>>> the first packets.
>>>
>>> Seems that after a few seconds of inactivity it "forgets" the paths.
>>>
>>>
>>>
>>> more infos:
>>>
>>> root at xen-blade16:~# ovs-dpctl show
>>> system at trunk0:
>>> lookups: frags:0, hit:1141399, missed:377043, lost:0
>>> port 0: trunk0 (internal)
>>> port 1: eth1
>>> port 2: mgmt0 (internal)
>>> port 3: eth0
>>> system at stor0:
>>> lookups: frags:0, hit:515, missed:68, lost:0
>>> port 0: stor0 (internal)
>>> port 1: eth3
>>> system at stor1:
>>> lookups: frags:0, hit:501, missed:62, lost:0
>>> port 0: stor1 (internal)
>>> port 1: eth2
>>>
>>>
>>> root at xen-blade16:~# ovs-appctl bond/show bond0
>>> bond_mode: balance-slb
>>> lacp: off
>>> bond-hash-algorithm: balance-slb
>>> bond-detect-mode: carrier
>>> updelay: 0 ms
>>> downdelay: 0 ms
>>> next rebalance: 6578 ms
>>>
>>> slave eth0: enabled
>>> active slave
>>> hash 95: 0 kB load
>>> 00:23:20:d4:c7:7b
>>>
>>> slave eth1: enabled
>>>
>>>
>>> root at xen-blade16:~# ovs-vsctl list port
>>> _uuid : 7c18f4b0-e644-45f2-81ba-cb588e8bdaf8
>>> bond_downdelay : 0
>>> bond_fake_iface : false
>>> bond_mode : []
>>> bond_updelay : 0
>>> external_ids : {}
>>> fake_bridge : false
>>> interfaces : [c956aabf-da4c-4d2b-970e-65338c842006]
>>> lacp : []
>>> mac : []
>>> name : "trunk0"
>>> other_config : {}
>>> qos : []
>>> tag : []
>>> trunks : []
>>>
>>> _uuid : df40b36c-c55b-406b-8613-cff1e59b35d6
>>> bond_downdelay : 0
>>> bond_fake_iface : false
>>> bond_mode : []
>>> bond_updelay : 0
>>> external_ids : {}
>>> fake_bridge : false
>>> interfaces : [e7f2fee3-93a3-4703-a818-500d91bff8b2]
>>> lacp : []
>>> mac : []
>>> name : "mgmt0"
>>> other_config : {}
>>> qos : []
>>> tag : 85
>>> trunks : []
>>>
>>> _uuid : 8ecce71c-7ff6-436f-a968-e6974d6ee100
>>> bond_downdelay : 0
>>> bond_fake_iface : false
>>> bond_mode : []
>>> bond_updelay : 0
>>> external_ids : {}
>>> fake_bridge : false
>>> interfaces : [34b58c6b-b97b-47c8-a57d-e8cef8ad918b]
>>> lacp : []
>>> mac : []
>>> name : "stor0"
>>> other_config : {}
>>> qos : []
>>> tag : []
>>> trunks : []
>>>
>>> _uuid : d435be0d-f371-4adf-ba50-3ff3989f7c18
>>> bond_downdelay : 0
>>> bond_fake_iface : false
>>> bond_mode : []
>>> bond_updelay : 0
>>> external_ids : {}
>>> fake_bridge : false
>>> interfaces : [485b7425-8ae5-4658-a65a-705b3f565934,
>>> 8f589dae-4d42-4cf0-b9e4-be7ab6693c4d]
>>> lacp : []
>>> mac : []
>>> name : "bond0"
>>> other_config : {}
>>> qos : []
>>> tag : []
>>> trunks : []
>>>
>>> _uuid : c3c55f92-04f9-4bdd-a119-9d851273e158
>>> bond_downdelay : 0
>>> bond_fake_iface : false
>>> bond_mode : []
>>> bond_updelay : 0
>>> external_ids : {}
>>> fake_bridge : false
>>> interfaces : [d55b71e6-e470-4921-8c44-f3154683ea7b]
>>> lacp : []
>>> mac : []
>>> name : "eth3"
>>> other_config : {}
>>> qos : []
>>> tag : []
>>> trunks : []
>>>
>>> _uuid : ec958047-ddbf-425f-97cc-dad30c5914c0
>>> bond_downdelay : 0
>>> bond_fake_iface : false
>>> bond_mode : []
>>> bond_updelay : 0
>>> external_ids : {}
>>> fake_bridge : false
>>> interfaces : [3cc9f429-5306-45ed-bfff-cd4cf9205748]
>>> lacp : []
>>> mac : []
>>> name : "stor1"
>>> other_config : {}
>>> qos : []
>>> tag : []
>>> trunks : []
>>>
>>> _uuid : 2d7b24b7-7049-47df-9d5f-60e6ff96ce10
>>> bond_downdelay : 0
>>> bond_fake_iface : false
>>> bond_mode : []
>>> bond_updelay : 0
>>> external_ids : {}
>>> fake_bridge : false
>>> interfaces : [fee9ee9c-d427-4c09-8182-1ea8492851c5]
>>> lacp : []
>>> mac : []
>>> name : "eth2"
>>> other_config : {}
>>> qos : []
>>> tag : []
>>> trunks : []
>>>
>>>
>>> Any ideas ? :)
>>>
>>> Thanks a lot for your help.
>>>
>>> Sébastien
>>> _______________________________________________
>>> discuss mailing list
>>> discuss at openvswitch.org
>>> http://openvswitch.org/mailman/listinfo/discuss
>>
>
> Hi Justin,
>
> Thanks for your reply.
>
> I'm not (yet) running an off-box controller. There is 0 load on the
> machine. The only process
> that appears sometime in the top of "top" is the switchd daemon.
> Is it a normal TIME+ count for the daemon, on a box up for one day
> without activity?
>
> root at xen-blade16:~# uptime
> 21:01:42 up 1 day, 5:12, 20 users, load average: 0.01, 0.00, 0.00
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 2070 root 20 0 22608 2492 1144 S 0 0.0 3:13.89 ovs-vswitchd
>
> root at xen-blade16:~# ovs-vsctl list bridge
> _uuid : 125a159c-b20f-4dfa-a7b4-d607db94f480
> controller : []
> datapath_id : "0000bc305b3883aa"
> datapath_type : ""
> external_ids : {}
> fail_mode : []
> flood_vlans : []
> mirrors : []
> name : "stor1"
> netflow : []
> other_config : {}
> ports : [2d7b24b7-7049-47df-9d5f-60e6ff96ce10,
> ec958047-ddbf-425f-97cc-dad30c5914c0]
> sflow : []
>
> _uuid : 9d01feb1-695c-413d-b333-e4532173b9f4
> controller : []
> datapath_id : "0000bc305b3883ac"
> datapath_type : ""
> external_ids : {}
> fail_mode : []
> flood_vlans : []
> mirrors : []
> name : "stor0"
> netflow : []
> other_config : {}
> ports : [8ecce71c-7ff6-436f-a968-e6974d6ee100,
> c3c55f92-04f9-4bdd-a119-9d851273e158]
> sflow : []
>
> _uuid : 353c2bb1-4ce0-4164-a1cb-3edb0553c809
> controller : []
> datapath_id : "0000bc305b3883a6"
> datapath_type : ""
> external_ids : {}
> fail_mode : []
> flood_vlans : []
> mirrors : []
> name : "trunk0"
> netflow : []
> other_config : {}
> ports : [7c18f4b0-e644-45f2-81ba-cb588e8bdaf8,
> d435be0d-f371-4adf-ba50-3ff3989f7c18,
> df40b36c-c55b-406b-8613-cff1e59b35d6]
> sflow : []
>
> root at xen-blade16:~# ovs-ofctl dump-flows trunk0
> NXST_FLOW reply (xid=0x4):
> cookie=0x0, duration=43370.807s, table_id=0, n_packets=1883934,
> n_bytes=143638556, priority=0 actions=NORMAL
>
> root at xen-blade16:~# ovs-ofctl dump-flows stor0
> NXST_FLOW reply (xid=0x4):
> cookie=0x0, duration=43389.711s, table_id=0, n_packets=91742,
> n_bytes=5651456, priority=0 actions=NORMAL
>
> root at xen-blade16:~# ovs-ofctl dump-flows stor1
> NXST_FLOW reply (xid=0x4):
> cookie=0x0, duration=43393.091s, table_id=0, n_packets=91796,
> n_bytes=5653888, priority=0 actions=NORMAL
>
> Best regards,
> Sébastien
>
> _______________________________________________
> discuss mailing list
> discuss at openvswitch.org
> http://openvswitch.org/mailman/listinfo/discuss
>
Hi Justin,
I've rm'ed and recreated the conf.db file and restarted from scratch
with the exact same setup as described in
my previous mails, and now it's wokring good. I don't get it... :)
Sorry about that false problem.
Sébastien
More information about the discuss
mailing list