[ovs-dev] [PATCH v2] tests: Add system-dpdk-testsuite

Stokes, Ian ian.stokes at intel.com
Wed Jan 31 10:39:44 UTC 2018


> -----Original Message-----
> From: ovs-dev-bounces at openvswitch.org [mailto:ovs-dev-
> bounces at openvswitch.org] On Behalf Of Marcin Rybka
> Sent: Tuesday, January 2, 2018 2:36 PM
> To: dev at openvswitch.org
> Cc: Rybka, MarcinX <marcinx.rybka at intel.com>
> Subject: [ovs-dev] [PATCH v2] tests: Add system-dpdk-testsuite
> 
> New OVS-DPDK testsuite, which can be launched via `make check-dpdk`, tests
> OVS using a DPDK datapath. The testsuite contains already initial tests:
>  1. EAL init
>  2. Add standard DPDK PHY port
>  3. Add vhost-user-client port
> 
> Signed-off-by: Marcin Rybka <marcinx.rybka at intel.com>
> ---
>  Documentation/topics/testing.rst | 19 ++++++++++++
>  tests/automake.mk                | 17 +++++++++++
>  tests/system-dpdk-macros.at      | 54 +++++++++++++++++++++++++++++++++
>  tests/system-dpdk-testsuite.at   | 25 ++++++++++++++++
>  tests/system-dpdk.at             | 65
> ++++++++++++++++++++++++++++++++++++++++
>  5 files changed, 180 insertions(+)
>  create mode 100644 tests/system-dpdk-macros.at  create mode 100644
> tests/system-dpdk-testsuite.at  create mode 100644 tests/system-dpdk.at
> 
> diff --git a/Documentation/topics/testing.rst
> b/Documentation/topics/testing.rst
> index a49336b..74e0d3f 100644
> --- a/Documentation/topics/testing.rst
> +++ b/Documentation/topics/testing.rst
> @@ -297,6 +297,25 @@ To invoke the datapath testsuite with the userspace
> datapath, run::
> 
>  The results of the testsuite are in ``tests/system-userspace-
> testsuite.dir``.
> 
> +DPDK datapath
> +'''''''''''''
> +
> +To test :doc:`/intro/install/dpdk` (i.e., the build was configured with
> +``--with-dpdk``,the ``DPDK`` is installed), run the testsuite and
> +generate a report by using the ``check-dpdk`` target::
> +
> +    $ make check-dpdk
> +
> +To see a list of all the available tests, run::
> +
> +    $ make check-dpdk TESTSUITEFLAGS=--list
> +
> +These tests require a `DPDK supported NIC`_ and proper DPDK variables
> +(``DPDK_DIR`` and ``DPDK_BUILD``). Moreover you need to load the
> +required modules and bind the NIC to the DPDK-compatible driver.
> +
> +.. _DPDK supported NIC: http://dpdk.org/doc/nics
> +

Hi Marcin,

Thanks for working on this, a few comments inline below.

These tests will require elevated privileges (root or sudo) on systems to be run, otherwise they will fail as memory will not be allocated etc. by ovs-dpdk.

I'd like to see this called out explicitly in the documentation along with a short explanation (it's obvious to those familiar with DPDK but we must assume a user may not be, also existing ovs unit test can be run without these privileges so someone might work from that assumption).

In testing I also see the following error although it does not stop the test from running.

set /bin/sh './tests/system-dpdk-testsuite' -C tests  AUTOTEST_PATH='utilities:vswitchd:ovsdb:vtep:tests::ovn/controller-vtep:ovn/northd:ovn/utilities:ovn/controller'  -j1; \
"$@" || (test X'' = Xyes && "$@" --recheck)
Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "/usr/lib64/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 99] Cannot assign requested address

>  Kernel datapath
>  '''''''''''''''
> 
> diff --git a/tests/automake.mk b/tests/automake.mk index 8157641..7be5712
> 100644
> --- a/tests/automake.mk
> +++ b/tests/automake.mk
> @@ -5,10 +5,12 @@ EXTRA_DIST += \
>  	$(SYSTEM_KMOD_TESTSUITE_AT) \
>  	$(SYSTEM_USERSPACE_TESTSUITE_AT) \
>  	$(SYSTEM_OFFLOADS_TESTSUITE_AT) \
> +	$(SYSTEM_DPDK_TESTSUITE_AT) \
>  	$(TESTSUITE) \
>  	$(SYSTEM_KMOD_TESTSUITE) \
>  	$(SYSTEM_USERSPACE_TESTSUITE) \
>  	$(SYSTEM_OFFLOADS_TESTSUITE) \
> +	$(SYSTEM_DPDK_TESTSUITE) \
>  	tests/atlocal.in \
>  	$(srcdir)/package.m4 \
>  	$(srcdir)/tests/testsuite \
> @@ -126,6 +128,12 @@ SYSTEM_OFFLOADS_TESTSUITE_AT = \
>  	tests/system-offloads-traffic.at \
>  	tests/system-offloads-testsuite.at
> 
> +SYSTEM_DPDK_TESTSUITE_AT = \
> +	tests/system-common-macros.at \
> +	tests/system-dpdk-macros.at \
> +	tests/system-dpdk-testsuite.at \
> +	tests/system-dpdk.at
> +
>  check_SCRIPTS += tests/atlocal
> 
>  TESTSUITE = $(srcdir)/tests/testsuite
> @@ -133,6 +141,7 @@ TESTSUITE_PATCH = $(srcdir)/tests/testsuite.patch
> SYSTEM_KMOD_TESTSUITE = $(srcdir)/tests/system-kmod-testsuite
>  SYSTEM_USERSPACE_TESTSUITE = $(srcdir)/tests/system-userspace-testsuite
>  SYSTEM_OFFLOADS_TESTSUITE = $(srcdir)/tests/system-offloads-testsuite
> +SYSTEM_DPDK_TESTSUITE = $(srcdir)/tests/system-dpdk-testsuite
>  DISTCLEANFILES += tests/atconfig tests/atlocal
> 
>  AUTOTEST_PATH =
> utilities:vswitchd:ovsdb:vtep:tests:$(PTHREAD_WIN32_DIR_DLL):ovn/controlle
> r-vtep:ovn/northd:ovn/utilities:ovn/controller
> @@ -256,6 +265,10 @@ check-offloads: all
>  	set $(SHELL) '$(SYSTEM_OFFLOADS_TESTSUITE)' -C tests
> AUTOTEST_PATH='$(AUTOTEST_PATH)' $(TESTSUITEFLAGS) -j1; \
>  	"$$@" || (test X'$(RECHECK)' = Xyes && "$$@" --recheck)
> 
> +check-dpdk: all
> +	set $(SHELL) '$(SYSTEM_DPDK_TESTSUITE)' -C tests
> AUTOTEST_PATH='$(AUTOTEST_PATH)' $(TESTSUITEFLAGS) -j1; \
> +	"$$@" || (test X'$(RECHECK)' = Xyes && "$$@" --recheck)
> +
>  clean-local:
>  	test ! -f '$(TESTSUITE)' || $(SHELL) '$(TESTSUITE)' -C tests --clean
> 
> @@ -284,6 +297,10 @@ $(SYSTEM_OFFLOADS_TESTSUITE): package.m4
> $(SYSTEM_TESTSUITE_AT) $(SYSTEM_OFFLOAD
>  	$(AM_V_GEN)$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at
>  	$(AM_V_at)mv $@.tmp $@
> 
> +$(SYSTEM_DPDK_TESTSUITE): package.m4 $(SYSTEM_TESTSUITE_AT)
> $(SYSTEM_DPDK_TESTSUITE_AT) $(COMMON_MACROS_AT)
> +	$(AM_V_GEN)$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at
> +	$(AM_V_at)mv $@.tmp $@
> +
>  # The `:;' works around a Bash 3.2 bug when the output is not writeable.
>  $(srcdir)/package.m4: $(top_srcdir)/configure.ac
>  	$(AM_V_GEN):;{ \
> diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at new
> file mode 100644 index 0000000..cbd0f69
> --- /dev/null
> +++ b/tests/system-dpdk-macros.at
> @@ -0,0 +1,54 @@
> +# OVS_DPDK_PRE_CHECK()
> +#
> +# Check prerequisites for DPDK tests. Following settings are checked:
> +#  - Hugepages
> +#  - UIO driver
> +#
> +m4_define([OVS_DPDK_PRE_CHECK],
> +  [dnl Check Hugepages
> +   AT_CHECK([cat /proc/meminfo], [], [stdout])
> +   AT_CHECK([grep HugePages_ stdout], [], [stdout])
> +   AT_CHECK([mount], [], [stdout])
> +   AT_CHECK([grep 'hugetlbfs' stdout], [], [stdout], [])
> +
> +   dnl Check if VFIO or UIO driver is loaded
> +   AT_CHECK([lsmod | grep -E "igb_uio|vfio"], [], [stdout])
> +
> +   dnl Find PCI address candidate, skip if there is no DPDK-compatible
> NIC
> +   AT_CHECK([$DPDK_DIR/usertools/dpdk-devbind.py -s | head -n +4 | tail -
> 1], [], [stdout])
> +   AT_CHECK([cat stdout | cut -d" " -s -f1 > PCI_ADDR])
> +   AT_CHECK([test -s PCI_ADDR || exit 77])
> +])
> +
> +
> +# OVS_DPDK_START()
> +#
> +# Create an empty database and start ovsdb-server. Add special
> +configuration # dpdk-init to enable DPDK functionality. Start
> +ovs-vswitchd connected to that # database using system devices (no
> dummies).
> +#
> +m4_define([OVS_DPDK_START],
> +  [dnl Create database.
> +   AT_CHECK([touch .conf.db.~lock~])
> +   AT_CHECK([ovsdb-tool create conf.db
> +$abs_top_srcdir/vswitchd/vswitch.ovsschema])
> +
> +   dnl Start ovsdb-server.
> +   AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --log-file --
> remote=punix:$OVS_RUNDIR/db.sock], [0], [stdout], [stderr])
> +   on_exit "kill `cat ovsdb-server.pid`"
> +   AT_CHECK([[sed < stderr '
> +/vlog|INFO|opened log file/d
> +/ovsdb_server|INFO|ovsdb-server (Open vSwitch)/d']])
> +   AT_CAPTURE_FILE([ovsdb-server.log])
> +
> +   dnl Initialize database.
> +   AT_CHECK([ovs-vsctl --no-wait init])
> +
> +   dnl Enable DPDK functionality
> +   AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-
> init=true])
> +   AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch .
> + other_config:dpdk-socket-mem="1024,1024"])

Have you tested above on a single socket system? I guess the requirements assumes a dual socket server because a pci device may be attached to either socket but would this fail if there was only 1 socket?

Thanks
Ian

> +
> +   dnl Start ovs-vswitchd.
> +   AT_CHECK([ovs-vswitchd --detach --no-chdir --pidfile --log-file -
> vvconn -vofproto_dpif -vunixctl], [0], [stdout], [stderr])
> +   AT_CAPTURE_FILE([ovs-vswitchd.log])
> +   on_exit "kill_ovs_vswitchd `cat ovs-vswitchd.pid`"
> +])
> diff --git a/tests/system-dpdk-testsuite.at b/tests/system-dpdk-
> testsuite.at new file mode 100644 index 0000000..382f09e
> --- /dev/null
> +++ b/tests/system-dpdk-testsuite.at
> @@ -0,0 +1,25 @@
> +AT_INIT
> +
> +AT_COPYRIGHT([Copyright (c) 2017 Intel Corporation
> +
> +Licensed under the Apache License, Version 2.0 (the "License"); you may
> +not use this file except in compliance with the License.
> +You may obtain a copy of the License at:
> +
> +    http://www.apache.org/licenses/LICENSE-2.0
> +
> +Unless required by applicable law or agreed to in writing, software
> +distributed under the License is distributed on an "AS IS" BASIS,
> +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> +See the License for the specific language governing permissions and
> +limitations under the License.])
> +
> +m4_ifdef([AT_COLOR_TESTS], [AT_COLOR_TESTS])
> +
> +m4_include([tests/ovs-macros.at])
> +m4_include([tests/ovsdb-macros.at])
> +m4_include([tests/ofproto-macros.at])
> +m4_include([tests/system-common-macros.at])
> +m4_include([tests/system-dpdk-macros.at])
> +
> +m4_include([tests/system-dpdk.at])
> diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at new file mode
> 100644 index 0000000..f6041e4
> --- /dev/null
> +++ b/tests/system-dpdk.at
> @@ -0,0 +1,65 @@
> +AT_BANNER([OVS-DPDK unit tests])
> +
> +dnl
> +-----------------------------------------------------------------------
> +---
> +dnl Check if EAL init is successfull
> +AT_SETUP([OVS-DPDK datapath - EAL init])
> +AT_KEYWORDS([dpdk])
> +dnl OVS_DPDK_PRE_CHECK()
> +OVS_DPDK_START()
> +AT_CHECK([grep "DPDK Enabled - initializing..." ovs-vswitchd.log], [],
> +[stdout]) AT_CHECK([grep "EAL" ovs-vswitchd.log], [], [stdout])
> +AT_CHECK([grep "DPDK Enabled - initialized" ovs-vswitchd.log], [],
> +[stdout])
> +OVS_VSWITCHD_STOP()
> +AT_CLEANUP
> +dnl
> +-----------------------------------------------------------------------
> +---
> +
> +
> +
> +dnl
> +-----------------------------------------------------------------------
> +---
> +dnl Add standard DPDK PHY port
> +AT_SETUP([OVS-DPDK datapath - add standard DPDK port])
> +AT_KEYWORDS([dpdk])
> +
> +OVS_DPDK_PRE_CHECK()
> +OVS_DPDK_START()
> +
> +dnl Add userspace bridge and attach it to OVS AT_CHECK([ovs-vsctl
> +add-br br10 -- set bridge br10 datapath_type=netdev])
> +AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk
> +options:dpdk-devargs=$(cat PCI_ADDR)], [], [stdout], [stderr])
> +AT_CHECK([ovs-vsctl show], [], [stdout]) sleep 2
> +
> +dnl Clean up
> +AT_CHECK([ovs-vsctl del-port br10 phy0], [], [stdout], [stderr])
> +OVS_VSWITCHD_STOP("/does not exist. The Open vSwitch kernel module is
> +probably not loaded./d /Failed to enable flow control/d") AT_CLEANUP
> +dnl
> +-----------------------------------------------------------------------
> +---
> +
> +
> +
> +dnl
> +-----------------------------------------------------------------------
> +---
> +dnl Add vhost-user-client port
> +AT_SETUP([OVS-DPDK datapath - add vhost-user-client port])
> +AT_KEYWORDS([dpdk])
> +
> +OVS_DPDK_START()
> +
> +dnl Add userspace bridge and attach it to OVS AT_CHECK([ovs-vsctl
> +add-br br10 -- set bridge br10 datapath_type=netdev])
> +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface
> +dpdkvhostuserclient0 type=dpdkvhostuserclient
> +options:vhost-server-path=/tmp/dpdkvhostclient0], [], [stdout],
> +[stderr]) AT_CHECK([ovs-vsctl show], [], [stdout]) sleep 2
> +
> +dnl Parse log file
> +AT_CHECK([grep "VHOST_CONFIG: vhost-user client: socket created"
> +ovs-vswitchd.log], [], [stdout]) AT_CHECK([grep "vHost User device
> +'dpdkvhostuserclient0' created in 'client' mode, using client socket"
> +ovs-vswitchd.log], [], [stdout]) AT_CHECK([grep "VHOST_CONFIG:
> +/tmp/dpdkvhostclient0: reconnecting..." ovs-vswitchd.log], [],
> +[stdout])
> +
> +dnl Clean up
> +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout],
> +[stderr]) OVS_VSWITCHD_STOP("/does not exist. The Open vSwitch kernel
> +module is probably not loaded./d /Failed to enable flow control/d
> +/failed to connect to \/tmp\/dpdkvhostclient0: No such file or
> +directory/d") AT_CLEANUP dnl
> +-----------------------------------------------------------------------
> +---
> --
> 1.9.3
> 
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev


More information about the dev mailing list