[ovs-discuss] vxlan udp csum and openstack ovs performance

kldeng kldeng05 at gmail.com
Thu Feb 25 13:14:15 UTC 2016


Hi, ALL.

Recently, we're trying to fix the poor VXLAN throughput issue by 
applying the series of patches @Ramu posted in ovs-dev mailing-list.
http://openvswitch.org/pipermail/dev/2015-August/059337.html 
<http://openvswitch.org/pipermail/dev/2015-August/059337.html>

Our test environment are as belows.
------------------------------------------
OVS: 2.4.0
Kernel: 3.18.22

Topology:
------------------------------------------
vm1 -- bridge -- vxlan -- 10Gnic -- 10Gnic -- vxlan -- bridge -- vm2

vxlan port settings:
------------------------------------------
Port "vxlan-0acd614d"
Interface "vxlan-0acd614d"
type: vxlan
options: {csum="true", df_default="true", in_key=flow, 
local_ip="10.205.97.135", out_key=flow, remote_ip="10.205.97.77"}


Patch applied:
------------------------------------------
--- a/src/net/openvswitch/vport-vxlan.c
+++ b/src/net/openvswitch/vport-vxlan.c
@@ -100,6 +100,7 @@ static struct vport *vxlan_tnl_create(const struct 
vport_parms *parms)
struct nlattr *a;
u16 dst_port;
int err;
+ u32 vxlan_sock_flags = 0;

if (!options) {
err = -EINVAL;
@@ -122,7 +123,8 @@ static struct vport *vxlan_tnl_create(const struct 
vport_parms *parms)
vxlan_port = vxlan_vport(vport);
strncpy(vxlan_port->name, parms->name, IFNAMSIZ);

- vs = vxlan_sock_add(net, htons(dst_port), vxlan_rcv, vport, true, 0);
+ vxlan_sock_flags |= VXLAN_F_UDP_CSUM;
+ vs = vxlan_sock_add(net, htons(dst_port), vxlan_rcv, vport, true, 
vxlan_sock_flags);
if (IS_ERR(vs)) {


But, it seems doesn't work, the VXLAN throughput is still very low(about 
1.7Gbps).

After inspecting, we found GRO is seems effective for VXLAN traffic for 
we can see aggregated packets on receiver nic and the function call 
graph also proves that.

- 0.60% 0.12% vhost-31236 [kernel.kallsyms] [k] tcp_gro_receive ?
- tcp_gro_receive ?
- 99.74% tcp4_gro_receive ?
inet_gro_receive ?
vxlan_gro_receive ?
udp_gro_receive ?
udp4_gro_receive ?
inet_gro_receive ?
dev_gro_receive ?
napi_gro_receive ?
ixgbe_clean_rx_irq ?
ixgbe_poll ?
net_rx_action ?
+ __do_softirq

But, on the qvo, qvb interface, the packet is fragemented again.

@jingting seems also running into the same problem.
http://openvswitch.org/pipermail/discuss/2016-January/019965.html 
<http://openvswitch.org/pipermail/discuss/2016-January/019965.html>

Any help is really appreciated. Thank you!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20160225/473d9982/attachment-0002.html>


More information about the discuss mailing list