[ovs-dev] [v6 07/11] test/sytem-dpdk: Add unit test for mfex autovalidator
Eelco Chaudron
echaudro at redhat.com
Wed Jul 7 14:52:43 UTC 2021
On 6 Jul 2021, at 15:11, Cian Ferriter wrote:
> From: Kumar Amber <kumar.amber at intel.com>
>
> Tests:
> 6: OVS-DPDK - MFEX Autovalidator
> 7: OVS-DPDK - MFEX Autovalidator Fuzzy
>
> Added a new directory to store the PCAP file used
> in the tests and a script to generate the fuzzy traffic
> type pcap to be used in fuzzy unit test.
>
> Signed-off-by: Kumar Amber <kumar.amber at intel.com>
>
> ---
>
> v5:
> - fix review comments(Ian, Flavio, Eelco)
> - remove sleep from first test and added minor 5 sec sleep to fuzzy
> ---
> ---
> Documentation/topics/dpdk/bridge.rst | 55
> +++++++++++++++++++++++++++
> tests/automake.mk | 5 +++
> tests/mfex_fuzzy.py | 32 ++++++++++++++++
> tests/pcap/mfex_test | Bin 0 -> 416 bytes
> tests/system-dpdk.at | 46 ++++++++++++++++++++++
> 5 files changed, 138 insertions(+)
> create mode 100755 tests/mfex_fuzzy.py
> create mode 100644 tests/pcap/mfex_test
>
> diff --git a/Documentation/topics/dpdk/bridge.rst
> b/Documentation/topics/dpdk/bridge.rst
> index 8495687e8..8a8ef3782 100644
> --- a/Documentation/topics/dpdk/bridge.rst
> +++ b/Documentation/topics/dpdk/bridge.rst
> @@ -341,3 +341,58 @@ A compile time option is available in order to
> test it with the OVS unit
> test suite. Use the following configure option ::
>
> $ ./configure --enable-mfex-default-autovalidator
> +
> +Unit Test Miniflow Extract
> +++++++++++++++++++++++++++
> +
> +Unit test can also be used to test the workflow mentioned above by
> running
> +the following test-case in tests/system-dpdk.at ::
> +
> + make check-dpdk TESTSUITEFLAGS='-k MFEX'
> + OVS-DPDK - MFEX Autovalidator
> +
> +The unit test uses mulitple traffic type to test the correctness of
> the
> +implementaions.
> +
> +Running Fuzzy test with Autovalidator
> ++++++++++++++++++++++++++++++++++++++
> +
> +Fuzzy tests can also be done on miniflow extract with the help of
> +auto-validator and Scapy. The steps below describes the steps to
> +reproduce the setup with IP being fuzzed to generate packets.
> +
> +Scapy is used to create fuzzy IP packets and save them into a PCAP ::
> +
> + pkt = fuzz(Ether()/IP()/TCP())
> +
> +Set the miniflow extract to autovalidator using ::
> +
> + $ ovs-appctl dpif-netdev/miniflow-parser-set autovalidator
> +
> +OVS is configured to receive the generated packets ::
> +
> + $ ovs-vsctl add-port br0 pcap0 -- \
> + set Interface pcap0 type=dpdk options:dpdk-devargs=net_pcap0
> + "rx_pcap=fuzzy.pcap"
> +
> +With this workflow, the autovalidator will ensure that all MFEX
> +implementations are classifying each packet in exactly the same way.
> +If an optimized MFEX implementation causes a different miniflow to be
> +generated, the autovalidator has ovs_assert and logging statements
> that
> +will inform about the issue.
> +
> +Unit Fuzzy test with Autovalidator
> ++++++++++++++++++++++++++++++++++++++
> +
> +The prerquiste before running the unit test is to run the script
> provided ::
> +
> + tests/mfex_fuzzy.py
> +
> +This script generates a pcap with mulitple type of fuzzed packets to
> be used
> +in the below unit test-case.
> +
> +Unit test can also be used to test the workflow mentioned above by
> running
> +the following test-case in tests/system-dpdk.at ::
> +
> + make check-dpdk TESTSUITEFLAGS='-k MFEX'
> + OVS-DPDK - MFEX Autovalidator Fuzzy
> diff --git a/tests/automake.mk b/tests/automake.mk
> index f45f8d76c..e94ccd27c 100644
> --- a/tests/automake.mk
> +++ b/tests/automake.mk
> @@ -143,6 +143,11 @@ $(srcdir)/tests/fuzz-regression-list.at:
> tests/automake.mk
> echo "TEST_FUZZ_REGRESSION([$$basename])"; \
> done > $@.tmp && mv $@.tmp $@
>
> +EXTRA_DIST += $(MFEX_AUTOVALIDATOR_TESTS)
> +MFEX_AUTOVALIDATOR_TESTS = \
> + tests/pcap/mfex_test \
> + tests/mfex_fuzzy.py
> +
> OVSDB_CLUSTER_TESTSUITE_AT = \
> tests/ovsdb-cluster-testsuite.at \
> tests/ovsdb-execution.at \
> diff --git a/tests/mfex_fuzzy.py b/tests/mfex_fuzzy.py
> new file mode 100755
> index 000000000..a8051ba2b
> --- /dev/null
> +++ b/tests/mfex_fuzzy.py
> @@ -0,0 +1,32 @@
> +#!/usr/bin/python3
> +try:
> + from scapy.all import *
> +except ModuleNotFoundError as err:
> + print(err + ": Scapy")
> +import sys
> +import os
> +
> +path = os.environ['OVS_DIR'] + "/tests/pcap/fuzzy"
This is failing in my setup, as OVS_DIR is not defined:
Is it something you set manually? As from make check-dpdk it fails:
7. system-dpdk.at:260: testing OVS-DPDK - MFEX Autovalidator Fuzzy ...
DEPRECATION: The default format will switch to columns in the future.
You can use --format=(legacy|columns) (or define a
format=(legacy|columns) in your pip.conf
under the [list] section) to disable this warning.
scapy (2.4.4)
./system-dpdk.at:263: $PYTHON3 $srcdir/mfex_fuzzy.py
--- /dev/null 2021-07-02 08:15:52.158758028 -0400
+++
/root/Documents/Scratch/ovs_review_mfex/OVS_master_DPDK_v20.11.1/ovs_github/tests/system-dpdk-testsuite.dir/at-groups/7/stderr 2021-07-07
10:34:09.364877
754 -0400
@@ -0,0 +1,6 @@
+Traceback (most recent call last):
+ File "../.././mfex_fuzzy.py", line 10, in <module>
+ path = os.environ['OVS_DIR'] + "/tests/pcap/fuzzy"
+ File "/usr/lib64/python3.6/os.py", line 669, in __getitem__
+ raise KeyError(key) from None
+KeyError: 'OVS_DIR'
stdout:
./system-dpdk.at:263: exit code was 1, expected 0
7. system-dpdk.at:260: 7. OVS-DPDK - MFEX Autovalidator Fuzzy
(system-dpdk.at:260): FAILED (system-dpdk.at:263)
If I set the environment variable it works fine, but should not be
needed.
> +pktdump = PcapWriter(path, append=False, sync=True)
> +
> +for i in range(0, 2000):
> +
> + # Generate random protocol bases, use a fuzz() over the combined
> packet for full fuzzing.
> + eth = Ether(src=RandMAC(), dst=RandMAC())
> + vlan = Dot1Q()
> + ipv4 = IP(src=RandIP(), dst=RandIP())
> + ipv6 = IPv6(src=RandIP6(), dst=RandIP6())
> + udp = UDP()
> + tcp = TCP()
> +
I think we should also randomize the UDP/TCP ports as they get
extracted.
> + # IPv4 packets with fuzzing
> + pktdump.write(fuzz(eth/ipv4/udp))
> + pktdump.write(fuzz(eth/ipv4/tcp))
> + pktdump.write(fuzz(eth/vlan/ipv4/udp))
> + pktdump.write(fuzz(eth/vlan/ipv4/tcp))
> +
> + # IPv6 packets with fuzzing
> + pktdump.write(fuzz(eth/ipv6/udp))
> + pktdump.write(fuzz(eth/ipv6/tcp))
> + pktdump.write(fuzz(eth/vlan/ipv6/udp))
> + pktdump.write(fuzz(eth/vlan/ipv6/tcp))
The generated pcap file does not have an extension, might be nice to at
it?
Also add the generated pcap file to the .git_ignore file.
> \ No newline at end of file
> diff --git a/tests/pcap/mfex_test b/tests/pcap/mfex_test
I would give the pcap file the .pcap extension just to be clear!
> new file mode 100644
> index
> 0000000000000000000000000000000000000000..1aac67b8d643ecb016c758cba4cc32212a80f52a
> GIT binary patch
> literal 416
> zcmca|c+)~A1{MYw`2U}Qff2}Q<eHVR>K`M68ITRa|G at yFii5$Gfk6YL%z>@uY&}o|
> z2s4N<1VH2&7y^V87$)XGOtD~MV$cFgfG~zBGGJ2#YtF$<F=a4i;9x8Q*<ZrSM6Ufz
> xK>KST_NTIwYriok6N4Vm)gX-Q@<yO<!C`>c^{cp<7_5LgK^UuU{2>VS0RZ!RQ+EIW
>
> literal 0
> HcmV?d00001
>
> diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at
> index 802895488..fcab92729 100644
> --- a/tests/system-dpdk.at
> +++ b/tests/system-dpdk.at
> @@ -232,3 +232,49 @@ OVS_VSWITCHD_STOP(["\@does not exist. The Open
> vSwitch kernel module is probably
> \@EAL: No free hugepages reported in hugepages-1048576kB at d"])
> AT_CLEANUP
> dnl
> --------------------------------------------------------------------------
> +
> +dnl
> --------------------------------------------------------------------------
> +dnl Add standard DPDK PHY port
I think we should also skip these tests if we do not have a machine that
has AVX512. Just to make sure we do not generate an OK where we are not
even testing the AVX512 functions.
> +AT_SETUP([OVS-DPDK - MFEX Autovalidator])
> +AT_KEYWORDS([dpdk])
> +
> +OVS_DPDK_START()
> +
> +dnl Add userspace bridge and attach it to OVS
> +AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0
> datapath_type=netdev])
> +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk
> options:dpdk-devargs=net_pcap1,rx_pcap=$srcdir/pcap/mfex_test,infinite_rx=1],
> [], [stdout], [stderr])
> +AT_CHECK([ovs-vsctl show], [], [stdout])
> +
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator],
> [0], [dnl
> +Miniflow implementation set to autovalidator
> +])
>
Here we set the autovalidator, and we stop it right after how do we know
enough packets were running through it? I thought we discussed looking
at frame count to make sure at least the pcap file was run through it
once?
> +dnl Clean up
> +AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr])
> +AT_CLEANUP
> +dnl
> --------------------------------------------------------------------------
> +
> +dnl
> --------------------------------------------------------------------------
> +dnl Add standard DPDK PHY port
> +AT_SETUP([OVS-DPDK - MFEX Autovalidator Fuzzy])
> +AT_KEYWORDS([dpdk])
> +AT_SKIP_IF([! pip3 list | grep scapy], [], [])
> +AT_CHECK([$PYTHON3 $srcdir/mfex_fuzzy.py], [], [stdout])
> +OVS_DPDK_START()
> +
> +dnl Add userspace bridge and attach it to OVS
> +AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0
> datapath_type=netdev])
> +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk
> options:dpdk-devargs=net_pcap1,rx_pcap=$srcdir/pcap/fuzzy,infinite_rx=1],
> [], [stdout], [stderr])
> +AT_CHECK([ovs-vsctl show], [], [stdout])
> +
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator],
> [0], [dnl
> +Miniflow implementation set to autovalidator
> +])
> +sleep 5
See above, I do not feel like a sleep will work overtime, we might miss
packets on a slower machine.
> +
> +dnl Clean up
> +AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr])
> +AT_CLEANUP
> +dnl
> --------------------------------------------------------------------------
> --
> 2.32.0
More information about the dev
mailing list