[ovs-dev] [v12 07/11] test/sytem-dpdk: Add unit test for mfex autovalidator

Amber, Kumar kumar.amber at intel.com
Thu Jul 15 02:19:28 UTC 2021 

Hi Eelco,




From: Eelco Chaudron <echaudro at redhat.com>
Sent: Wednesday, July 14, 2021 8:26 PM
To: Amber, Kumar <kumar.amber at intel.com>
Cc: ovs-dev at openvswitch.org; fbl at sysclose.org; i.maximets at ovn.org; Van Haaren, Harry <harry.van.haaren at intel.com>; Ferriter, Cian <cian.ferriter at intel.com>; Stokes, Ian <ian.stokes at intel.com>
Subject: Re: [v12 07/11] test/sytem-dpdk: Add unit test for mfex autovalidator


On 14 Jul 2021, at 16:14, kumar Amber wrote:

From: Kumar Amber <kumar.amber at intel.com<mailto:kumar.amber at intel.com>>

Tests:
6: OVS-DPDK - MFEX Autovalidator
7: OVS-DPDK - MFEX Autovalidator Fuzzy

Added a new directory to store the PCAP file used
in the tests and a script to generate the fuzzy traffic
type pcap to be used in fuzzy unit test.

Signed-off-by: Kumar Amber <kumar.amber at intel.com<mailto:kumar.amber at intel.com>>
Acked-by: Flavio Leitner <fbl at sysclose.org<mailto:fbl at sysclose.org>>
---
v12:
- change skip paramter for unit test
v11:
- fix comments from Eelco
v7:
- fix review comments(Eelco)
v5:
- fix review comments(Ian, Flavio, Eelco)
- remove sleep from first test and added minor 5 sec sleep to fuzzy
---
---
Documentation/topics/dpdk/bridge.rst | 48 ++++++++++++++++++++++++
tests/.gitignore | 1 +
tests/automake.mk | 6 +++
tests/mfex_fuzzy.py | 33 +++++++++++++++++
tests/pcap/mfex_test.pcap | Bin 0 -> 416 bytes
tests/system-dpdk.at | 53 +++++++++++++++++++++++++++
6 files changed, 141 insertions(+)
create mode 100755 tests/mfex_fuzzy.py
create mode 100644 tests/pcap/mfex_test.pcap

diff --git a/Documentation/topics/dpdk/bridge.rst b/Documentation/topics/dpdk/bridge.rst
index 8c500c504..2f065422b 100644
--- a/Documentation/topics/dpdk/bridge.rst
+++ b/Documentation/topics/dpdk/bridge.rst
@@ -346,3 +346,51 @@ A compile time option is available in order to test it with the OVS unit
test suite. Use the following configure option ::

$ ./configure --enable-mfex-default-autovalidator
+
+Unit Test Miniflow Extract
+++++++++++++++++++++++++++
+
+Unit test can also be used to test the workflow mentioned above by running
+the following test-case in tests/system-dpdk.at ::
+
+ make check-dpdk TESTSUITEFLAGS='-k MFEX'
+ OVS-DPDK - MFEX Autovalidator
+
+The unit test uses mulitple traffic type to test the correctness of the
+implementaions.
+
+Running Fuzzy test with Autovalidator
++++++++++++++++++++++++++++++++++++++
+
+Fuzzy tests can also be done on miniflow extract with the help of
+auto-validator and Scapy. The steps below describes the steps to
+reproduce the setup with IP being fuzzed to generate packets.
+
+Scapy is used to create fuzzy IP packets and save them into a PCAP ::
+
+ pkt = fuzz(Ether()/IP()/TCP())
+
+Set the miniflow extract to autovalidator using ::
+
+ $ ovs-appctl dpif-netdev/miniflow-parser-set autovalidator
+
+OVS is configured to receive the generated packets ::
+
+ $ ovs-vsctl add-port br0 pcap0 -- \
+ set Interface pcap0 type=dpdk options:dpdk-devargs=net_pcap0
+ "rx_pcap=fuzzy.pcap"
+
+With this workflow, the autovalidator will ensure that all MFEX
+implementations are classifying each packet in exactly the same way.
+If an optimized MFEX implementation causes a different miniflow to be
+generated, the autovalidator has ovs_assert and logging statements that
+will inform about the issue.
+
+Unit Fuzzy test with Autovalidator
++++++++++++++++++++++++++++++++++++++
+
+Unit test can also be used to test the workflow mentioned above by running
+the following test-case in tests/system-dpdk.at ::
+
+ make check-dpdk TESTSUITEFLAGS='-k MFEX'
+ OVS-DPDK - MFEX Autovalidator Fuzzy
diff --git a/tests/.gitignore b/tests/.gitignore
index 45b4f67b2..a3d927e5d 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -11,6 +11,7 @@
/ovsdb-cluster-testsuite
/ovsdb-cluster-testsuite.dir/
/ovsdb-cluster-testsuite.log
+/pcap/
/pki/
/system-afxdp-testsuite
/system-afxdp-testsuite.dir/
diff --git a/tests/automake.mk b/tests/automake.mk
index f45f8d76c..a6c15ba55 100644
--- a/tests/automake.mk
+++ b/tests/automake.mk
@@ -143,6 +143,11 @@ $(srcdir)/tests/fuzz-regression-list.at: tests/automake.mk
echo "TEST_FUZZ_REGRESSION([$$basename])"; \
done > $@.tmp<mailto:$@.tmp> && mv $@.tmp<mailto:$@.tmp> $@

+EXTRA_DIST += $(MFEX_AUTOVALIDATOR_TESTS)
+MFEX_AUTOVALIDATOR_TESTS = \
+ tests/pcap/mfex_test.pcap \
+ tests/mfex_fuzzy.py
+
OVSDB_CLUSTER_TESTSUITE_AT = \
tests/ovsdb-cluster-testsuite.at \
tests/ovsdb-execution.at \
@@ -512,6 +517,7 @@ tests_test_type_props_SOURCES = tests/test-type-props.c
CHECK_PYFILES = \
tests/appctl.py \
tests/flowgen.py \
+ tests/mfex_fuzzy.py \
tests/ovsdb-monitor-sort.py \
tests/test-daemon.py \
tests/test-json.py \
diff --git a/tests/mfex_fuzzy.py b/tests/mfex_fuzzy.py
new file mode 100755
index 000000000..5b056bb48
--- /dev/null
+++ b/tests/mfex_fuzzy.py
@@ -0,0 +1,33 @@
+#!/usr/bin/python3
+try:
+ from scapy.all import RandMAC, RandIP, PcapWriter, RandIP6, RandShort, fuzz
+ from scapy.all import IPv6, Dot1Q, IP, Ether, UDP, TCP
+except ModuleNotFoundError as err:
+ print(err + ": Scapy")
+import sys
+
+path = str(sys.argv[1]) + "/pcap/fuzzy.pcap"
+pktdump = PcapWriter(path, append=False, sync=True)
+
+for i in range(0, 2000):
+
+ # Generate random protocol bases, use a fuzz() over the combined packet
+ # for full fuzzing.
+ eth = Ether(src=RandMAC(), dst=RandMAC())
+ vlan = Dot1Q()
+ ipv4 = IP(src=RandIP(), dst=RandIP())
+ ipv6 = IPv6(src=RandIP6(), dst=RandIP6())
+ udp = UDP(dport=RandShort(), sport=RandShort())
+ tcp = TCP(dport=RandShort(), sport=RandShort())
+
+ # IPv4 packets with fuzzing
+ pktdump.write(fuzz(eth / ipv4 / udp))
+ pktdump.write(fuzz(eth / ipv4 / tcp))
+ pktdump.write(fuzz(eth / vlan / ipv4 / udp))
+ pktdump.write(fuzz(eth / vlan / ipv4 / tcp))
+
+ # IPv6 packets with fuzzing
+ pktdump.write(fuzz(eth / ipv6 / udp))
+ pktdump.write(fuzz(eth / ipv6 / tcp))
+ pktdump.write(fuzz(eth / vlan / ipv6 / udp))
+ pktdump.write(fuzz(eth / vlan / ipv6 / tcp))
diff --git a/tests/pcap/mfex_test.pcap b/tests/pcap/mfex_test.pcap
new file mode 100644
index 0000000000000000000000000000000000000000..1aac67b8d643ecb016c758cba4cc32212a80f52a
GIT binary patch
literal 416
zcmca|c+)~A1{MYw`2U}Qff2}Q<eHVR>K`M68ITRa|G at yFii5$Gfk6YL%z>@uY&}o|
z2s4N<1VH2&7y^V87$)XGOtD~MV$cFgfG~zBGGJ2#YtF$<F=a4i;9x8Q*<ZrSM6Ufz
xK>KST_NTIwYriok6N4Vm)gX-Q@<yO<!C`>c^{cp<7_5LgK^UuU{2>VS0RZ!RQ+EIW

literal 0
HcmV?d00001

diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at
index 802895488..af257aab2 100644
--- a/tests/system-dpdk.at
+++ b/tests/system-dpdk.at
@@ -232,3 +232,56 @@ OVS_VSWITCHD_STOP(["\@does not exist. The Open vSwitch kernel module is probably
\@EAL: No free hugepages reported in hugepages-1048576kB at d"])
AT_CLEANUP
dnl --------------------------------------------------------------------------
+
+dnl --------------------------------------------------------------------------
+dnl Add standard DPDK PHY port
+AT_SETUP([OVS-DPDK - MFEX Autovalidator])
+AT_KEYWORDS([dpdk])
+
+OVS_DPDK_START()
+
+dnl Add userspace bridge and attach it to OVS
+AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev])
+AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk options:dpdk-devargs=net_pcap1,rx_pcap=$srcdir/pcap/mfex_test.pcap,infinite_rx=1], [], [stdout], [stderr])
+AT_CHECK([ovs-vsctl show], [], [stdout])
+
+AT_SKIP_IF([! ovs-appctl dpif-netdev/miniflow-parser-get | sed 1,4d | grep -v "True"], [], [dnl
+])
+

I bet you did not test this for all cases as you still have the -v? Same below…
Tests should be:

  *   At this patch on a system with and without AVX512 capabilities
  *   At the last patch of the series with and without AVX512 capabilities

[Amb] -> There was a setup problem that was causing diff between your tests and mine btw removing “-v”
             Is the correct solution.

+AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator], [0], [dnl
+Miniflow extract implementation set to autovalidator.
+])
+
+OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1 statistics | grep -oP 'rx_packets=\s*\K\d+'` -ge 1000])
+
+dnl Clean up
+AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr])
+AT_CLEANUP
+dnl --------------------------------------------------------------------------
+
+dnl --------------------------------------------------------------------------
+dnl Add standard DPDK PHY port
+AT_SETUP([OVS-DPDK - MFEX Autovalidator Fuzzy])
+AT_KEYWORDS([dpdk])
+AT_SKIP_IF([! pip3 list | grep scapy], [], [])
+AT_CHECK([$PYTHON3 $srcdir/mfex_fuzzy.py $srcdir], [], [stdout])
+OVS_DPDK_START()
+
+dnl Add userspace bridge and attach it to OVS
+AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev])
+AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk options:dpdk-devargs=net_pcap1,rx_pcap=$srcdir/pcap/fuzzy.pcap,infinite_rx=1], [], [stdout], [stderr])
+AT_CHECK([ovs-vsctl show], [], [stdout])
+
+AT_SKIP_IF([! ovs-appctl dpif-netdev/miniflow-parser-get | sed 1,4d | grep -v "True"], [], [dnl
+])
+
+AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator], [0], [dnl
+Miniflow extract implementation set to autovalidator.
+])
+
+OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1 statistics | grep -oP 'rx_packets=\s*\K\d+'` -ge 100000])
+
+dnl Clean up
+AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr])
+AT_CLEANUP
+dnl --------------------------------------------------------------------------
--

Missing the command line test cases, see discussion with Harry for the proper solution to get them working.

[Amb] -> The test-case [dnl have problems with new lines but a new test-case is added with all the commands both positive and negative ones .



Br

Amber

2.25.1

More information about the dev mailing list