[ovs-discuss] vxlan udp csum and openstack ovs performance
kldeng
kldeng05 at gmail.com
Thu Feb 25 13:14:15 UTC 2016
Hi, ALL.
Recently, we're trying to fix the poor VXLAN throughput issue by
applying the series of patches @Ramu posted in ovs-dev mailing-list.
http://openvswitch.org/pipermail/dev/2015-August/059337.html
<http://openvswitch.org/pipermail/dev/2015-August/059337.html>
Our test environment are as belows.
------------------------------------------
OVS: 2.4.0
Kernel: 3.18.22
Topology:
------------------------------------------
vm1 -- bridge -- vxlan -- 10Gnic -- 10Gnic -- vxlan -- bridge -- vm2
vxlan port settings:
------------------------------------------
Port "vxlan-0acd614d"
Interface "vxlan-0acd614d"
type: vxlan
options: {csum="true", df_default="true", in_key=flow,
local_ip="10.205.97.135", out_key=flow, remote_ip="10.205.97.77"}
Patch applied:
------------------------------------------
--- a/src/net/openvswitch/vport-vxlan.c
+++ b/src/net/openvswitch/vport-vxlan.c
@@ -100,6 +100,7 @@ static struct vport *vxlan_tnl_create(const struct
vport_parms *parms)
struct nlattr *a;
u16 dst_port;
int err;
+ u32 vxlan_sock_flags = 0;
if (!options) {
err = -EINVAL;
@@ -122,7 +123,8 @@ static struct vport *vxlan_tnl_create(const struct
vport_parms *parms)
vxlan_port = vxlan_vport(vport);
strncpy(vxlan_port->name, parms->name, IFNAMSIZ);
- vs = vxlan_sock_add(net, htons(dst_port), vxlan_rcv, vport, true, 0);
+ vxlan_sock_flags |= VXLAN_F_UDP_CSUM;
+ vs = vxlan_sock_add(net, htons(dst_port), vxlan_rcv, vport, true,
vxlan_sock_flags);
if (IS_ERR(vs)) {
But, it seems doesn't work, the VXLAN throughput is still very low(about
1.7Gbps).
After inspecting, we found GRO is seems effective for VXLAN traffic for
we can see aggregated packets on receiver nic and the function call
graph also proves that.
- 0.60% 0.12% vhost-31236 [kernel.kallsyms] [k] tcp_gro_receive ?
- tcp_gro_receive ?
- 99.74% tcp4_gro_receive ?
inet_gro_receive ?
vxlan_gro_receive ?
udp_gro_receive ?
udp4_gro_receive ?
inet_gro_receive ?
dev_gro_receive ?
napi_gro_receive ?
ixgbe_clean_rx_irq ?
ixgbe_poll ?
net_rx_action ?
+ __do_softirq
But, on the qvo, qvb interface, the packet is fragemented again.
@jingting seems also running into the same problem.
http://openvswitch.org/pipermail/discuss/2016-January/019965.html
<http://openvswitch.org/pipermail/discuss/2016-January/019965.html>
Any help is really appreciated. Thank you!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20160225/473d9982/attachment-0002.html>
More information about the discuss
mailing list