[ovs-dev] [PATCH v13 07/11] test/sytem-dpdk: Add unit test for mfex autovalidator
Flavio Leitner
fbl at sysclose.org
Thu Jul 15 15:25:54 UTC 2021
On Thu, Jul 15, 2021 at 06:12:13PM +0530, kumar Amber wrote:
> From: Kumar Amber <kumar.amber at intel.com>
>
> Tests:
> 6: OVS-DPDK - MFEX Autovalidator
> 7: OVS-DPDK - MFEX Autovalidator Fuzzy
> 8: OVS-DPDK - MFEX Configuration
>
> Added a new directory to store the PCAP file used
> in the tests and a script to generate the fuzzy traffic
> type pcap to be used in fuzzy unit test.
>
> Signed-off-by: Kumar Amber <kumar.amber at intel.com>
> Acked-by: Flavio Leitner <fbl at sysclose.org>
>
> ---
> v13:
> - fix -v in the command
> - added the configuration test case and supporting doc update
> v12:
> - change skip paramter for unit test
> v11:
> - fix comments from Eelco
> v7:
> - fix review comments(Eelco)
> v5:
> - fix review comments(Ian, Flavio, Eelco)
> - remove sleep from first test and added minor 5 sec sleep to fuzzy
> ---
> ---
> Documentation/topics/dpdk/bridge.rst | 56 ++++++++++++
> tests/.gitignore | 1 +
> tests/automake.mk | 6 ++
> tests/mfex_fuzzy.py | 33 +++++++
> tests/pcap/mfex_test.pcap | Bin 0 -> 416 bytes
> tests/system-dpdk.at | 129 +++++++++++++++++++++++++++
> 6 files changed, 225 insertions(+)
> create mode 100755 tests/mfex_fuzzy.py
> create mode 100644 tests/pcap/mfex_test.pcap
>
> diff --git a/Documentation/topics/dpdk/bridge.rst b/Documentation/topics/dpdk/bridge.rst
> index 8c500c504..913b3e6f6 100644
> --- a/Documentation/topics/dpdk/bridge.rst
> +++ b/Documentation/topics/dpdk/bridge.rst
> @@ -346,3 +346,59 @@ A compile time option is available in order to test it with the OVS unit
> test suite. Use the following configure option ::
>
> $ ./configure --enable-mfex-default-autovalidator
> +
> +Unit Test Miniflow Extract
> +++++++++++++++++++++++++++
> +
> +Unit test can also be used to test the workflow mentioned above by running
> +the following test-case in tests/system-dpdk.at ::
> +
> + make check-dpdk TESTSUITEFLAGS='-k MFEX'
> + OVS-DPDK - MFEX Autovalidator
> +
> +The unit test uses mulitple traffic type to test the correctness of the
> +implementaions.
> +
> +The MFEX commands can also be tested for negative and positive cases to
> +verify that the MFEX set command does not allow for incorrect parameters.
> +A user can directly run the following configuration test case in
> +tests/system-dpdk.at ::
> +
> + make check-dpdk TESTSUITEFLAGS='-k MFEX'
> + OVS-DPDK - MFEX Configuration
> +
> +Running Fuzzy test with Autovalidator
> ++++++++++++++++++++++++++++++++++++++
> +
> +Fuzzy tests can also be done on miniflow extract with the help of
> +auto-validator and Scapy. The steps below describes the steps to
> +reproduce the setup with IP being fuzzed to generate packets.
> +
> +Scapy is used to create fuzzy IP packets and save them into a PCAP ::
> +
> + pkt = fuzz(Ether()/IP()/TCP())
> +
> +Set the miniflow extract to autovalidator using ::
> +
> + $ ovs-appctl dpif-netdev/miniflow-parser-set autovalidator
> +
> +OVS is configured to receive the generated packets ::
> +
> + $ ovs-vsctl add-port br0 pcap0 -- \
> + set Interface pcap0 type=dpdk options:dpdk-devargs=net_pcap0
> + "rx_pcap=fuzzy.pcap"
> +
> +With this workflow, the autovalidator will ensure that all MFEX
> +implementations are classifying each packet in exactly the same way.
> +If an optimized MFEX implementation causes a different miniflow to be
> +generated, the autovalidator has ovs_assert and logging statements that
> +will inform about the issue.
> +
> +Unit Fuzzy test with Autovalidator
> ++++++++++++++++++++++++++++++++++++++
> +
> +Unit test can also be used to test the workflow mentioned above by running
> +the following test-case in tests/system-dpdk.at ::
> +
> + make check-dpdk TESTSUITEFLAGS='-k MFEX'
> + OVS-DPDK - MFEX Autovalidator Fuzzy
> diff --git a/tests/.gitignore b/tests/.gitignore
> index 45b4f67b2..a3d927e5d 100644
> --- a/tests/.gitignore
> +++ b/tests/.gitignore
> @@ -11,6 +11,7 @@
> /ovsdb-cluster-testsuite
> /ovsdb-cluster-testsuite.dir/
> /ovsdb-cluster-testsuite.log
> +/pcap/
> /pki/
> /system-afxdp-testsuite
> /system-afxdp-testsuite.dir/
> diff --git a/tests/automake.mk b/tests/automake.mk
> index f45f8d76c..a6c15ba55 100644
> --- a/tests/automake.mk
> +++ b/tests/automake.mk
> @@ -143,6 +143,11 @@ $(srcdir)/tests/fuzz-regression-list.at: tests/automake.mk
> echo "TEST_FUZZ_REGRESSION([$$basename])"; \
> done > $@.tmp && mv $@.tmp $@
>
> +EXTRA_DIST += $(MFEX_AUTOVALIDATOR_TESTS)
> +MFEX_AUTOVALIDATOR_TESTS = \
> + tests/pcap/mfex_test.pcap \
> + tests/mfex_fuzzy.py
> +
> OVSDB_CLUSTER_TESTSUITE_AT = \
> tests/ovsdb-cluster-testsuite.at \
> tests/ovsdb-execution.at \
> @@ -512,6 +517,7 @@ tests_test_type_props_SOURCES = tests/test-type-props.c
> CHECK_PYFILES = \
> tests/appctl.py \
> tests/flowgen.py \
> + tests/mfex_fuzzy.py \
> tests/ovsdb-monitor-sort.py \
> tests/test-daemon.py \
> tests/test-json.py \
> diff --git a/tests/mfex_fuzzy.py b/tests/mfex_fuzzy.py
> new file mode 100755
> index 000000000..5b056bb48
> --- /dev/null
> +++ b/tests/mfex_fuzzy.py
> @@ -0,0 +1,33 @@
> +#!/usr/bin/python3
> +try:
> + from scapy.all import RandMAC, RandIP, PcapWriter, RandIP6, RandShort, fuzz
> + from scapy.all import IPv6, Dot1Q, IP, Ether, UDP, TCP
> +except ModuleNotFoundError as err:
> + print(err + ": Scapy")
> +import sys
> +
> +path = str(sys.argv[1]) + "/pcap/fuzzy.pcap"
> +pktdump = PcapWriter(path, append=False, sync=True)
> +
> +for i in range(0, 2000):
> +
> + # Generate random protocol bases, use a fuzz() over the combined packet
> + # for full fuzzing.
> + eth = Ether(src=RandMAC(), dst=RandMAC())
> + vlan = Dot1Q()
> + ipv4 = IP(src=RandIP(), dst=RandIP())
> + ipv6 = IPv6(src=RandIP6(), dst=RandIP6())
> + udp = UDP(dport=RandShort(), sport=RandShort())
> + tcp = TCP(dport=RandShort(), sport=RandShort())
> +
> + # IPv4 packets with fuzzing
> + pktdump.write(fuzz(eth / ipv4 / udp))
> + pktdump.write(fuzz(eth / ipv4 / tcp))
> + pktdump.write(fuzz(eth / vlan / ipv4 / udp))
> + pktdump.write(fuzz(eth / vlan / ipv4 / tcp))
> +
> + # IPv6 packets with fuzzing
> + pktdump.write(fuzz(eth / ipv6 / udp))
> + pktdump.write(fuzz(eth / ipv6 / tcp))
> + pktdump.write(fuzz(eth / vlan / ipv6 / udp))
> + pktdump.write(fuzz(eth / vlan / ipv6 / tcp))
> diff --git a/tests/pcap/mfex_test.pcap b/tests/pcap/mfex_test.pcap
> new file mode 100644
> index 0000000000000000000000000000000000000000..1aac67b8d643ecb016c758cba4cc32212a80f52a
> GIT binary patch
> literal 416
> zcmca|c+)~A1{MYw`2U}Qff2}Q<eHVR>K`M68ITRa|G at yFii5$Gfk6YL%z>@uY&}o|
> z2s4N<1VH2&7y^V87$)XGOtD~MV$cFgfG~zBGGJ2#YtF$<F=a4i;9x8Q*<ZrSM6Ufz
> xK>KST_NTIwYriok6N4Vm)gX-Q@<yO<!C`>c^{cp<7_5LgK^UuU{2>VS0RZ!RQ+EIW
>
> literal 0
> HcmV?d00001
>
> diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at
> index 802895488..a764c69ca 100644
> --- a/tests/system-dpdk.at
> +++ b/tests/system-dpdk.at
> @@ -232,3 +232,132 @@ OVS_VSWITCHD_STOP(["\@does not exist. The Open vSwitch kernel module is probably
> \@EAL: No free hugepages reported in hugepages-1048576kB at d"])
> AT_CLEANUP
> dnl --------------------------------------------------------------------------
> +
> +dnl --------------------------------------------------------------------------
> +dnl Add standard DPDK PHY port
> +AT_SETUP([OVS-DPDK - MFEX Autovalidator])
> +AT_KEYWORDS([dpdk])
> +
> +OVS_DPDK_START()
> +
> +dnl Add userspace bridge and attach it to OVS
> +AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev])
> +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk options:dpdk-devargs=net_pcap1,rx_pcap=$srcdir/pcap/mfex_test.pcap,infinite_rx=1], [], [stdout], [stderr])
> +AT_CHECK([ovs-vsctl show], [], [stdout])
> +
> +AT_SKIP_IF([! ovs-appctl dpif-netdev/miniflow-parser-get | sed 1,4d | grep "True"], [], [dnl
> +])
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator], [0], [dnl
> +Miniflow extract implementation set to autovalidator.
> +])
> +
> +OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1 statistics | grep -oP 'rx_packets=\s*\K\d+'` -ge 1000])
> +
> +dnl Clean up
> +AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr])
> +AT_CLEANUP
> +dnl --------------------------------------------------------------------------
> +
> +dnl --------------------------------------------------------------------------
> +dnl Add standard DPDK PHY port
> +AT_SETUP([OVS-DPDK - MFEX Autovalidator Fuzzy])
> +AT_KEYWORDS([dpdk])
> +AT_SKIP_IF([! pip3 list | grep scapy], [], [])
> +AT_CHECK([$PYTHON3 $srcdir/mfex_fuzzy.py $srcdir], [], [stdout])
> +OVS_DPDK_START()
> +
> +dnl Add userspace bridge and attach it to OVS
> +AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev])
> +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk options:dpdk-devargs=net_pcap1,rx_pcap=$srcdir/pcap/fuzzy.pcap,infinite_rx=1], [], [stdout], [stderr])
> +AT_CHECK([ovs-vsctl show], [], [stdout])
> +
> +AT_SKIP_IF([! ovs-appctl dpif-netdev/miniflow-parser-get | sed 1,4d | grep "True"], [], [dnl
> +])
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator], [0], [dnl
> +Miniflow extract implementation set to autovalidator.
> +])
> +
> +OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1 statistics | grep -oP 'rx_packets=\s*\K\d+'` -ge 100000])
> +
> +dnl Clean up
> +AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr])
> +AT_CLEANUP
> +dnl --------------------------------------------------------------------------
> +
> +dnl --------------------------------------------------------------------------
> +AT_SETUP([OVS-DPDK - MFEX Configuration])
> +AT_KEYWORDS([dpdk])
> +OVS_DPDK_START()
> +dnl Add userspace bridge and attach it to OVS
> +AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev])
> +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk options:dpdk-devargs=net_pcap1,rx_pcap=$srcdir/pcap/mfex_test.pcap,infinite_rx=1], [], [stdout], [stderr])
> +AT_CHECK([ovs-vsctl show], [], [stdout])
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set scalar 1], [2],
> +[], [dnl
> +Error: unknown argument 1.
> +ovs-appctl: ovs-vswitchd: server returned an error
> +])
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 3 study 300 xyz], [2],
> +[], [dnl
> +Error: Invalid study_pkt_cnt value: xyz.
> +ovs-appctl: ovs-vswitchd: server returned an error
> +])
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set scalar abcd], [2],
> +[], [dnl
> +Error: unknown argument abcd.
> +ovs-appctl: ovs-vswitchd: server returned an error
> +])
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 0 scalar abcd], [2],
> +[], [dnl
> +Error: unknown argument abcd.
> +ovs-appctl: ovs-vswitchd: server returned an error
> +])
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd], [2],
> +[], [dnl
> +Error: -pmd option requires a thread id argument.
> +ovs-appctl: ovs-vswitchd: server returned an error
> +])
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set tudy abcd], [2],
> +[], [dnl
> +Error: unknown argument abcd.
> +ovs-appctl: ovs-vswitchd: server returned an error
> +])
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 3 study abcd], [2],
> +[], [dnl
> +Error: Invalid study_pkt_cnt value: abcd.
> +ovs-appctl: ovs-vswitchd: server returned an error
> +])
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 21 study], [0], [dnl
> +Miniflow extract implementation set to study, on pmd thread 21, studying 128 packets.
> +])
This one actually fails to me because the pmd id is not valid:
--- /dev/null 2021-07-14 14:56:03.508411934 -0400
+++
/ovs/tests/system-dpdk-testsuite.dir/at-groups/8/stderr
2021-07-15 10:59:47.441060921 -0400
@@ -0,0 +1,2 @@
+Error: Miniflow parser not changed, PMD thread 21 not in use, pass a valid pmd thread ID.
+ovs-appctl: ovs-vswitchd: server returned an error
Most probably the other ones relying on -pmd 21 will fail as well.
The valid PMD id here was 11. I think the test needs to find out the
PMD id first, then issue the command.
fbl
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 21 study 512], [0], [dnl
> +Miniflow extract implementation set to study, on pmd thread 21, studying 512 packets.
> +])
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set study 512], [0], [dnl
> +Miniflow extract implementation set to study, studying 512 packets.
> +])
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set study], [0], [dnl
> +Miniflow extract implementation set to study, studying 128 packets.
> +])
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 21 autovalidator], [0], [dnl
> +Miniflow extract implementation set to autovalidator, on pmd thread 21.
> +])
> +
> +dnl Clean up
> +AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr])
> +AT_CLEANUP dnl
> +dnl --------------------------------------------------------------------------
> --
> 2.25.1
>
--
fbl
More information about the dev
mailing list