[ovs-dev] [PATCH v2 3/3] system-dpdk: Test with mlx5 devices.

Eelco Chaudron echaudro at redhat.com
Wed Nov 24 16:14:05 UTC 2021



On 23 Nov 2021, at 15:15, David Marchand wrote:

> The DPDK unit test only runs if vfio or igb_uio kernel module is loaded:
> on systems with only mlx5, this test is always skipped.
>
> Besides, the test tries to grab the first device listed by dpdk-devbind.py,
> regardless of the PCI device status regarding kmod binding.
>
> Remove dependency on this DPDK script and use a minimal script that
> reads PCI sysfs.
>
> This script is not perfect, as one can imagine PCI devices bound to
> vfio-pci for virtual machines.
> Add a new environment variable DPDK_PCI_ADDR for testers to select the
> PCI device of their liking.
> For consistency and grep, the temporary file PCI_ADDR is renamed
> to DPDK_PCI_ADDR.
>
> Note: with mlx5 devices, there is now more OVS/DPDK warnings to waive.

Uncovered some build issues with my scripts as now my MLX5 card in the system is used :)

Changes look good to me, one small nit.

Acked-by: Eelco Chaudron <echaudro at redhat.com>

> Signed-off-by: David Marchand <david.marchand at redhat.com>
> ---
>  Documentation/topics/testing.rst |  1 -
>  tests/automake.mk                |  1 +
>  tests/system-dpdk-find-device.py | 44 ++++++++++++++++++++++++++++++++
>  tests/system-dpdk-macros.at      | 10 ++------
>  tests/system-dpdk.at             |  4 ++-
>  5 files changed, 50 insertions(+), 10 deletions(-)
>  create mode 100755 tests/system-dpdk-find-device.py
>
> diff --git a/Documentation/topics/testing.rst b/Documentation/topics/testing.rst
> index 0ddcbd58ab..00f734a77a 100644
> --- a/Documentation/topics/testing.rst
> +++ b/Documentation/topics/testing.rst
> @@ -343,7 +343,6 @@ To see a list of all the available tests, run::
>
>  These tests support a `DPDK supported NIC`_. The tests operate on a wider set of
>  environments, for instance, when a virtual port is used.
> -They do require proper DPDK variables (``DPDK_DIR`` and ``DPDK_BUILD``).
>  Moreover you need to have root privileges to load the required modules and to bind
>  the NIC to the DPDK-compatible driver.
>
> diff --git a/tests/automake.mk b/tests/automake.mk
> index c519a5cb36..ead92ace0f 100644
> --- a/tests/automake.mk
> +++ b/tests/automake.mk
> @@ -188,6 +188,7 @@ SYSTEM_OFFLOADS_TESTSUITE_AT = \
>
>  SYSTEM_DPDK_TESTSUITE_AT = \
>  	tests/system-common-macros.at \
> +	tests/system-dpdk-find-device.py \
>  	tests/system-dpdk-macros.at \
>  	tests/system-dpdk-testsuite.at \
>  	tests/system-dpdk.at
> diff --git a/tests/system-dpdk-find-device.py b/tests/system-dpdk-find-device.py
> new file mode 100755
> index 0000000000..4253326e75
> --- /dev/null
> +++ b/tests/system-dpdk-find-device.py
> @@ -0,0 +1,44 @@
> +#!/usr/bin/env python3
> +# Copyright (c) 2021 Red Hat, Inc.
> +#
> +# Licensed under the Apache License, Version 2.0 (the "License");
> +# you may not use this file except in compliance with the License.
> +# You may obtain a copy of the License at:
> +#
> +#     http://www.apache.org/licenses/LICENSE-2.0
> +#
> +# Unless required by applicable law or agreed to in writing, software
> +# distributed under the License is distributed on an "AS IS" BASIS,
> +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> +# See the License for the specific language governing permissions and
> +# limitations under the License.
> +
> +
> +from pathlib import Path
> +import os
> +import sys
> +
> +# The tester might want to select a PCI device, if so, trust it.
> +if 'DPDK_PCI_ADDR' in os.environ:
> +    print(os.environ['DPDK_PCI_ADDR'])
> +    sys.exit(0)
> +
> +mlx5_ib_available = Path('/sys/module/mlx5_ib').exists()
> +
> +for device in sorted(Path('/sys/bus/pci/devices').iterdir()):
> +    class_path = device / 'class'
> +    # Only consider Network class devices
> +    if class_path.read_text().strip() != '0x020000':
> +        continue
> +    kmod_path = device / 'driver' / 'module'
> +    kmod_name = kmod_path.resolve().name
> +    # Devices of interest:
> +    # - device is bound to vfio_pci or igb_uio,
> +    # - device is bound to mlx5_core and mlx5 is loaded,
> +    if (kmod_name not in ['vfio_pci', 'igb_uio'] and
> +        (kmod_name not in ['mlx5_core'] or not mlx5_ib_available)):

Flake8 is complaining about this. But for some reason, I like this better ;)
Quick online search reviews unclear interpretations, so I would leave it as is…

system-dpdk-find-device.py:39:9: E129 visually indented line with same indent as next logical line

> +        continue
> +    print(device.resolve().name)
> +    sys.exit(0)
> +
> +sys.exit(1)
> diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at
> index b3614f0565..066fae0ba5 100644
> --- a/tests/system-dpdk-macros.at
> +++ b/tests/system-dpdk-macros.at
> @@ -22,14 +22,8 @@ m4_define([OVS_DPDK_PRE_PHY_SKIP],
>    [dnl Perform the precheck
>     OVS_DPDK_PRE_CHECK()
>
> -   dnl Check if VFIO or UIO driver is loaded
> -   AT_SKIP_IF([ ! (lsmod | grep -E "igb_uio|vfio") ], [], [stdout])
> -
> -   dnl Find PCI address candidate, skip if there is no DPDK-compatible NIC
> -   AT_CHECK([$DPDK_DIR/usertools/dpdk-devbind.py -s | head -n +4 | tail -1], [], [stdout])
> -   AT_CHECK([cat stdout | cut -d" " -s -f1 > PCI_ADDR])
> -   AT_SKIP_IF([ ! test -s PCI_ADDR ])
> -
> +   dnl Check if a device is available for DPDK
> +   AT_SKIP_IF([ ! $abs_top_srcdir/tests/system-dpdk-find-device.py > DPDK_PCI_ADDR])
>  ])
>
>
> diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at
> index 7b5d063f62..894e01316f 100644
> --- a/tests/system-dpdk.at
> +++ b/tests/system-dpdk.at
> @@ -7,6 +7,8 @@ m4_define([SYSTEM_DPDK_ALLOWED_LOGS],[
>  \@does not exist. The Open vSwitch kernel module is probably not loaded. at d
>  \@EAL:   Invalid NUMA socket, default to 0 at d
>  \@EAL: No \(available\|free\) hugepages reported in hugepages- at d
> +\@Rx checksum offload is not supported on at d
> +\@does not support MTU configuration, at d
>  ])
>
>  dnl --------------------------------------------------------------------------
> @@ -34,7 +36,7 @@ OVS_DPDK_START()
>
>  dnl Add userspace bridge and attach it to OVS
>  AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev])
> -AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat PCI_ADDR)], [], [stdout], [stderr])
> +AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat DPDK_PCI_ADDR)], [], [stdout], [stderr])
>  AT_CHECK([ovs-vsctl show], [], [stdout])
>  sleep 2
>
> -- 
> 2.23.0



More information about the dev mailing list