[ovs-dev] iperf tcp issue on veth using afxdp

[William Tu]

Hi Ilya and Eelco,

Yiyang reports very poor TCP performance on his setup and I can
also reproduce it on my machine. Somehow I think this might be a
kernel issue, but I don't know where to debug this. Need your suggestion
about how to debug.

So the setup is like the system-traffic, creating 2 namespaces and
veth devices and attach to OVS. I do remember to turn off tx offload
and ping, UDP, nc (tcp-mode) works fine.

TCP using iperf drops to 0Mbps after 4 seconds.
At server side:
root at osboxes:~/ovs# ip netns exec at_ns0 iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size:  128 KByte (default)
------------------------------------------------------------
[  4] local 10.1.1.1 port 5001 connected with 10.1.1.2 port 40384
Waiting for server threads to complete. Interrupt again to force quit.

At client side
root at osboxes:~/bpf-next# ip netns exec at_ns1 iperf -c 10.1.1.1 -i 1 -t 10
------------------------------------------------------------
Client connecting to 10.1.1.1, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 10.1.1.2 port 40384 connected with 10.1.1.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 1.0 sec  17.0 MBytes   143 Mbits/sec
[  3]  1.0- 2.0 sec  9.62 MBytes  80.7 Mbits/sec
[  3]  2.0- 3.0 sec  6.75 MBytes  56.6 Mbits/sec
[  3]  3.0- 4.0 sec  11.0 MBytes  92.3 Mbits/sec
[  3]  5.0- 6.0 sec  0.00 Bytes  0.00 bits/sec
[  3]  6.0- 7.0 sec  0.00 Bytes  0.00 bits/sec
[  3]  7.0- 8.0 sec  0.00 Bytes  0.00 bits/sec
[  3]  8.0- 9.0 sec  0.00 Bytes  0.00 bits/sec
[  3]  9.0-10.0 sec  0.00 Bytes  0.00 bits/sec
[  3] 10.0-11.0 sec  0.00 Bytes  0.00 bits/sec

(after this, even ping stops working)

Script to reproduce
-------------------------
ovs-vsctl -- add-br br0 -- set Bridge br0 datapath_type=netdev

ip netns add at_ns0
ip link add p0 type veth peer name afxdp-p0
ip link set p0 netns at_ns0
ip link set dev afxdp-p0 up
ovs-vsctl add-port br0 afxdp-p0

ovs-vsctl -- set interface afxdp-p0 options:n_rxq=1 type="afxdp"
options:xdp-mode=native
ip netns exec at_ns0 sh << NS_EXEC_HEREDOC
ip addr add "10.1.1.1/24" dev p0
ip link set dev p0 up
NS_EXEC_HEREDOC

ip netns add at_ns1
ip link add p1 type veth peer name afxdp-p1
ip link set p1 netns at_ns1
ip link set dev afxdp-p1 up
ovs-vsctl add-port br0 afxdp-p1 -- \
               set interface afxdp-p1 options:n_rxq=1 type="afxdp"
options:xdp-mode=native

ip netns exec at_ns1 sh << NS_EXEC_HEREDOC
ip addr add "10.1.1.2/24" dev p1
ip link set dev p1 up
NS_EXEC_HEREDOC

ethtool -K afxdp-p0 tx off
ethtool -K afxdp-p1 tx off
ip netns exec at_ns0 ethtool -K p0 tx off
ip netns exec at_ns1 ethtool -K p1 tx off

ip netns exec at_ns0 ping  -c 10 -i .2 10.1.1.2
echo "ip netns exec at_ns1 iperf -c 10.1.1.1 -i 1 -t 10"
ip netns exec at_ns0 iperf -s

Thank you
William