[ovs-dev] [PATCH v5 6/6] system-dpdk: Connect network namespaces via dpdkvhostuser ports
Lam, Tiago
tiago.lam at intel.com
Tue Sep 18 21:35:54 UTC 2018
Hi Bala,
On 14/09/2018 15:12, Bala Sankaran wrote:
> This adds a few unit tests to the 'check-dpdk' subsystem that will
> exercise allocations of two network namespaces, PMDs, and the
> vhost-user and vhost-user-client code path(separate tests).
> As part of the tests, userspace bridge is added and attached to OVS.
> Also, the tap devices created are added to the network namespaces.
>
> The ultimatum is to connect the two network namespaces by pinging them.
>
> Signed-off-by: Bala Sankaran <bsankara at redhat.com>
> Co-authored-by: Aaron Conole <aconole at redhat.com>
> Signed-off-by: Aaron Conole <aconole at redhat.com>
> ---
Both these tests pass for me on my system (a two NUMA node system) with
18.05+ (including 18.11-rc0).
As you mentioned before, losing the "--socket-mem" argument would be the
preferred solution, but that seems to bring further issues and, in my
system, testpmd doesn't seem to forward the traffic between ports. I
think that will require further investigation to get to the bottom of
it, but until then this should run successfully on multi NUMA node
systems by allocating memory for each node:
Reviewed-by Tiago Lam <tiago.lam at intel.com>
Tested-by: Tiago Lam <tiago.lam at intel.com>
I can have a look at the rest of the series later this week if no one
beats me to it.
Tiago.
> tests/system-dpdk.at | 160 ++++++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 159 insertions(+), 1 deletion(-)
>
> diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at
> index 58dc8aaae..1da020aba 100644
> --- a/tests/system-dpdk.at
> +++ b/tests/system-dpdk.at
> @@ -1,7 +1,10 @@
> +m4_define([CONFIGURE_VETH_OFFLOADS],
> + [AT_CHECK([ethtool -K $1 tx off], [0], [ignore], [ignore])])
> +
> AT_BANNER([OVS-DPDK unit tests])
>
> dnl --------------------------------------------------------------------------
> -dnl Check if EAL init is successfull
> +dnl Check if EAL init is successful
> AT_SETUP([OVS-DPDK datapath - EAL init])
> AT_KEYWORDS([dpdk])
> OVS_DPDK_PRE_CHECK()
> @@ -74,3 +77,158 @@ OVS_VSWITCHD_STOP(["\@does not exist. The Open vSwitch kernel module is probably
> \@EAL: No free hugepages reported in hugepages-1048576kB at d"])
> AT_CLEANUP
> dnl --------------------------------------------------------------------------
> +
> +
> +
> +dnl --------------------------------------------------------------------------
> +dnl Ping vhost-user port
> +AT_SETUP([OVS-DPDK datapath - ping vhost-user ports])
> +AT_KEYWORDS([dpdk])
> +OVS_DPDK_PRE_CHECK()
> +AT_SKIP_IF([! which testpmd >/dev/null 2>/dev/null])
> +OVS_DPDK_START()
> +
> +dnl Find number of sockets
> +AT_CHECK([lscpu], [], [stdout])
> +AT_CHECK([cat stdout | grep "NUMA node(s)" | awk '{c=1; while (c++<$(3)) {printf "512,"}; print "512"}' > NUMA_NODE])
> +
> +dnl Add userspace bridge and attach it to OVS
> +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev])
> +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuser0 -- set Interface dpdkvhostuser0 \
> + type=dpdkvhostuser], [],
> + [stdout], [stderr])
> +AT_CHECK([ovs-vsctl show], [], [stdout])
> +
> +dnl Parse log file
> +AT_CHECK([grep "VHOST_CONFIG: vhost-user server: socket created" \
> + ovs-vswitchd.log], [], [stdout])
> +AT_CHECK([grep "Socket $OVS_RUNDIR/dpdkvhostuser0 created for vhost-user port dpdkvhostuser0" \
> + ovs-vswitchd.log], [], [stdout])
> +AT_CHECK([grep "VHOST_CONFIG: bind to $OVS_RUNDIR/dpdkvhostuser0" ovs-vswitchd.log], [],
> + [stdout])
> +
> +dnl Set up namespaces
> +ADD_NAMESPACES(ns1, ns2)
> +
> +dnl Add veth device
> +ADD_VETH(tap1, ns2, br10, "172.31.110.12/24")
> +
> +dnl Execute testpmd in background
> +on_exit "pkill -f -x -9 'tail -f /dev/null'"
> +tail -f /dev/null | testpmd --socket-mem="$(cat NUMA_NODE)" --no-pci\
> + --vdev="net_virtio_user,path=$OVS_RUNDIR/dpdkvhostuser0" \
> + --vdev="net_tap0,iface=tap0" --file-prefix page0 \
> + --single-file-segments -- -a >$OVS_RUNDIR/testpmd-dpdkvhostuser0.log 2>&1 &
> +
> +dnl Give settling time to the testpmd processes - NOTE: this is bad form.
> +sleep 10
> +
> +dnl Move the tap devices to the namespaces
> +AT_CHECK([ps aux | grep testpmd], [], [stdout], [stderr])
> +AT_CHECK([ip link show], [], [stdout], [stderr])
> +AT_CHECK([ip link set tap0 netns ns1], [], [stdout], [stderr])
> +
> +AT_CHECK([ip netns exec ns1 ip link show], [], [stdout], [stderr])
> +AT_CHECK([ip netns exec ns1 ip link show | grep tap0], [], [stdout], [stderr])
> +AT_CHECK([ip netns exec ns1 ip link set tap0 up], [], [stdout], [stderr])
> +AT_CHECK([ip netns exec ns1 ip addr add 172.31.110.11/24 dev tap0], [],
> + [stdout], [stderr])
> +
> +AT_CHECK([ip netns exec ns1 ip link show], [], [stdout], [stderr])
> +AT_CHECK([ip netns exec ns2 ip link show], [], [stdout], [stderr])
> +AT_CHECK([ip netns exec ns1 ping -c 4 -I tap0 172.31.110.12], [], [stdout],
> + [stderr])
> +
> +dnl Clean up the testpmd now
> +pkill -f -x -9 'tail -f /dev/null'
> +
> +dnl Clean up
> +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuser0], [], [stdout], [stderr])
> +OVS_VSWITCHD_STOP(["\@does not exist. The Open vSwitch kernel module is probably not loaded. at d
> +\@Failed to enable flow control at d
> +\@VHOST_CONFIG: recvmsg failed at d
> +\@VHOST_CONFIG: failed to connect to $OVS_RUNDIR/dpdkvhostuser0: No such file or directory at d
> +\@Global register is changed during at d
> +\@dpdkvhostuser ports are considered deprecated; please migrate to dpdkvhostuserclient ports. at d
> +\@failed to enumerate system datapaths: No such file or directory at d
> +\@EAL: Invalid NUMA socket, default to 0 at d
> +\@EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !@d
> +\@EAL: No free hugepages reported in hugepages-1048576kB at d"])
> +AT_CLEANUP
> +dnl --------------------------------------------------------------------------
> +
> +dnl --------------------------------------------------------------------------
> +dnl Ping vhost-user-client port
> +AT_SETUP([OVS-DPDK datapath - ping vhost-user-client ports])
> +AT_KEYWORDS([dpdk])
> +OVS_DPDK_PRE_CHECK()
> +AT_SKIP_IF([! which testpmd >/dev/null 2>/dev/null])
> +OVS_DPDK_START()
> +
> +dnl Find number of sockets
> +AT_CHECK([lscpu], [], [stdout])
> +AT_CHECK([cat stdout | grep "NUMA node(s)" | awk '{c=1; while (c++<$(3)) {printf "512,"}; print "512"}' > NUMA_NODE])
> +
> +dnl Add userspace bridge and attach it to OVS
> +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev])
> +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface \
> + dpdkvhostuserclient0 \
> + type=dpdkvhostuserclient \
> + options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [],
> + [stdout], [stderr])
> +AT_CHECK([ovs-vsctl show], [], [stdout])
> +
> +dnl Parse log file
> +AT_CHECK([grep "VHOST_CONFIG: vhost-user client: socket created" ovs-vswitchd.log], [], [stdout])
> +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout])
> +AT_CHECK([grep "VHOST_CONFIG: $OVS_RUNDIR/dpdkvhostclient0: reconnecting..." ovs-vswitchd.log], [], [stdout])
> +
> +dnl Set up namespaces
> +ADD_NAMESPACES(ns1, ns2)
> +
> +dnl Add veth device
> +ADD_VETH(tap1, ns2, br10, "172.31.110.12/24")
> +
> +dnl Execute testpmd in background
> +on_exit "pkill -f -x -9 'tail -f /dev/null'"
> +tail -f /dev/null | testpmd --socket-mem="$(cat NUMA_NODE)" --no-pci\
> + --vdev="net_virtio_user,path=$OVS_RUNDIR/dpdkvhostclient0,server=1" \
> + --vdev="net_tap0,iface=tap0" --file-prefix page0 \
> + --single-file-segments -- -a >$OVS_RUNDIR/testpmd-dpdkvhostuserclient0.log 2>&1 &
> +
> +dnl Give settling time to the testpmd processes - NOTE: this is bad form.
> +sleep 10
> +
> +dnl Move the tap devices to the namespaces
> +AT_CHECK([ps aux | grep testpmd], [], [stdout], [stderr])
> +AT_CHECK([ip link show], [], [stdout], [stderr])
> +AT_CHECK([ip link set tap0 netns ns1], [], [stdout], [stderr])
> +
> +AT_CHECK([ip netns exec ns1 ip link show], [], [stdout], [stderr])
> +AT_CHECK([ip netns exec ns1 ip link show | grep tap0], [], [stdout], [stderr])
> +AT_CHECK([ip netns exec ns1 ip link set tap0 up], [], [stdout], [stderr])
> +AT_CHECK([ip netns exec ns1 ip addr add 172.31.110.11/24 dev tap0], [],
> + [stdout], [stderr])
> +
> +AT_CHECK([ip netns exec ns1 ip link show], [], [stdout], [stderr])
> +AT_CHECK([ip netns exec ns2 ip link show], [], [stdout], [stderr])
> +AT_CHECK([ip netns exec ns1 ping -c 4 -I tap0 172.31.110.12], [], [stdout],
> + [stderr])
> +
> +dnl Clean up the testpmd now
> +pkill -f -x -9 'tail -f /dev/null'
> +
> +dnl Clean up
> +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr])
> +OVS_VSWITCHD_STOP(["\@does not exist. The Open vSwitch kernel module is probably not loaded. at d
> +\@Failed to enable flow control at d
> +\@VHOST_CONFIG: recvmsg failed at d
> +\@VHOST_CONFIG: failed to connect to $OVS_RUNDIR/dpdkvhostclient0: No such file or directory at d
> +\@Global register is changed during at d
> +\@dpdkvhostuser ports are considered deprecated; please migrate to dpdkvhostuserclient ports. at d
> +\@failed to enumerate system datapaths: No such file or directory at d
> +\@EAL: Invalid NUMA socket, default to 0 at d
> +\@EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !@d
> +\@EAL: No free hugepages reported in hugepages-1048576kB at d"])
> +AT_CLEANUP
> +dnl --------------------------------------------------------------------------
>
More information about the dev
mailing list