[ovs-dev] [PATCH v13 07/11] test/sytem-dpdk: Add unit test for mfex autovalidator
Stokes, Ian
ian.stokes at intel.com
Thu Jul 15 15:06:10 UTC 2021
> Added some missing negative test cases, see below.
>
Thanks for these Eelco, would you like me to add you as a co-author for this patch as you have code that is part of the next revison?
Regards
Ian
> On 15 Jul 2021, at 14:42, kumar Amber wrote:
>
> > From: Kumar Amber <kumar.amber at intel.com>
> >
> > Tests:
> > 6: OVS-DPDK - MFEX Autovalidator
> > 7: OVS-DPDK - MFEX Autovalidator Fuzzy
> > 8: OVS-DPDK - MFEX Configuration
> >
> > Added a new directory to store the PCAP file used
> > in the tests and a script to generate the fuzzy traffic
> > type pcap to be used in fuzzy unit test.
> >
> > Signed-off-by: Kumar Amber <kumar.amber at intel.com>
> > Acked-by: Flavio Leitner <fbl at sysclose.org>
> >
> > ---
> > v13:
> > - fix -v in the command
> > - added the configuration test case and supporting doc update
> > v12:
> > - change skip paramter for unit test
> > v11:
> > - fix comments from Eelco
> > v7:
> > - fix review comments(Eelco)
> > v5:
> > - fix review comments(Ian, Flavio, Eelco)
> > - remove sleep from first test and added minor 5 sec sleep to fuzzy
> > ---
> > ---
> > Documentation/topics/dpdk/bridge.rst | 56 ++++++++++++
> > tests/.gitignore | 1 +
> > tests/automake.mk | 6 ++
> > tests/mfex_fuzzy.py | 33 +++++++
> > tests/pcap/mfex_test.pcap | Bin 0 -> 416 bytes
> > tests/system-dpdk.at | 129 +++++++++++++++++++++++++++
> > 6 files changed, 225 insertions(+)
> > create mode 100755 tests/mfex_fuzzy.py
> > create mode 100644 tests/pcap/mfex_test.pcap
> >
> > diff --git a/Documentation/topics/dpdk/bridge.rst
> b/Documentation/topics/dpdk/bridge.rst
> > index 8c500c504..913b3e6f6 100644
> > --- a/Documentation/topics/dpdk/bridge.rst
> > +++ b/Documentation/topics/dpdk/bridge.rst
> > @@ -346,3 +346,59 @@ A compile time option is available in order to test it
> with the OVS unit
> > test suite. Use the following configure option ::
> >
> > $ ./configure --enable-mfex-default-autovalidator
> > +
> > +Unit Test Miniflow Extract
> > +++++++++++++++++++++++++++
> > +
> > +Unit test can also be used to test the workflow mentioned above by running
> > +the following test-case in tests/system-dpdk.at ::
> > +
> > + make check-dpdk TESTSUITEFLAGS='-k MFEX'
> > + OVS-DPDK - MFEX Autovalidator
> > +
> > +The unit test uses mulitple traffic type to test the correctness of the
> > +implementaions.
> > +
> > +The MFEX commands can also be tested for negative and positive cases to
> > +verify that the MFEX set command does not allow for incorrect parameters.
> > +A user can directly run the following configuration test case in
> > +tests/system-dpdk.at ::
> > +
> > + make check-dpdk TESTSUITEFLAGS='-k MFEX'
> > + OVS-DPDK - MFEX Configuration
> > +
> > +Running Fuzzy test with Autovalidator
> > ++++++++++++++++++++++++++++++++++++++
> > +
> > +Fuzzy tests can also be done on miniflow extract with the help of
> > +auto-validator and Scapy. The steps below describes the steps to
> > +reproduce the setup with IP being fuzzed to generate packets.
> > +
> > +Scapy is used to create fuzzy IP packets and save them into a PCAP ::
> > +
> > + pkt = fuzz(Ether()/IP()/TCP())
> > +
> > +Set the miniflow extract to autovalidator using ::
> > +
> > + $ ovs-appctl dpif-netdev/miniflow-parser-set autovalidator
> > +
> > +OVS is configured to receive the generated packets ::
> > +
> > + $ ovs-vsctl add-port br0 pcap0 -- \
> > + set Interface pcap0 type=dpdk options:dpdk-devargs=net_pcap0
> > + "rx_pcap=fuzzy.pcap"
> > +
> > +With this workflow, the autovalidator will ensure that all MFEX
> > +implementations are classifying each packet in exactly the same way.
> > +If an optimized MFEX implementation causes a different miniflow to be
> > +generated, the autovalidator has ovs_assert and logging statements that
> > +will inform about the issue.
> > +
> > +Unit Fuzzy test with Autovalidator
> > ++++++++++++++++++++++++++++++++++++++
> > +
> > +Unit test can also be used to test the workflow mentioned above by running
> > +the following test-case in tests/system-dpdk.at ::
> > +
> > + make check-dpdk TESTSUITEFLAGS='-k MFEX'
> > + OVS-DPDK - MFEX Autovalidator Fuzzy
> > diff --git a/tests/.gitignore b/tests/.gitignore
> > index 45b4f67b2..a3d927e5d 100644
> > --- a/tests/.gitignore
> > +++ b/tests/.gitignore
> > @@ -11,6 +11,7 @@
> > /ovsdb-cluster-testsuite
> > /ovsdb-cluster-testsuite.dir/
> > /ovsdb-cluster-testsuite.log
> > +/pcap/
> > /pki/
> > /system-afxdp-testsuite
> > /system-afxdp-testsuite.dir/
> > diff --git a/tests/automake.mk b/tests/automake.mk
> > index f45f8d76c..a6c15ba55 100644
> > --- a/tests/automake.mk
> > +++ b/tests/automake.mk
> > @@ -143,6 +143,11 @@ $(srcdir)/tests/fuzz-regression-list.at:
> tests/automake.mk
> > echo "TEST_FUZZ_REGRESSION([$$basename])"; \
> > done > $@.tmp && mv $@.tmp $@
> >
> > +EXTRA_DIST += $(MFEX_AUTOVALIDATOR_TESTS)
> > +MFEX_AUTOVALIDATOR_TESTS = \
> > + tests/pcap/mfex_test.pcap \
> > + tests/mfex_fuzzy.py
> > +
> > OVSDB_CLUSTER_TESTSUITE_AT = \
> > tests/ovsdb-cluster-testsuite.at \
> > tests/ovsdb-execution.at \
> > @@ -512,6 +517,7 @@ tests_test_type_props_SOURCES = tests/test-type-
> props.c
> > CHECK_PYFILES = \
> > tests/appctl.py \
> > tests/flowgen.py \
> > + tests/mfex_fuzzy.py \
> > tests/ovsdb-monitor-sort.py \
> > tests/test-daemon.py \
> > tests/test-json.py \
> > diff --git a/tests/mfex_fuzzy.py b/tests/mfex_fuzzy.py
> > new file mode 100755
> > index 000000000..5b056bb48
> > --- /dev/null
> > +++ b/tests/mfex_fuzzy.py
> > @@ -0,0 +1,33 @@
> > +#!/usr/bin/python3
> > +try:
> > + from scapy.all import RandMAC, RandIP, PcapWriter, RandIP6, RandShort,
> fuzz
> > + from scapy.all import IPv6, Dot1Q, IP, Ether, UDP, TCP
> > +except ModuleNotFoundError as err:
> > + print(err + ": Scapy")
> > +import sys
> > +
> > +path = str(sys.argv[1]) + "/pcap/fuzzy.pcap"
> > +pktdump = PcapWriter(path, append=False, sync=True)
> > +
> > +for i in range(0, 2000):
> > +
> > + # Generate random protocol bases, use a fuzz() over the combined packet
> > + # for full fuzzing.
> > + eth = Ether(src=RandMAC(), dst=RandMAC())
> > + vlan = Dot1Q()
> > + ipv4 = IP(src=RandIP(), dst=RandIP())
> > + ipv6 = IPv6(src=RandIP6(), dst=RandIP6())
> > + udp = UDP(dport=RandShort(), sport=RandShort())
> > + tcp = TCP(dport=RandShort(), sport=RandShort())
> > +
> > + # IPv4 packets with fuzzing
> > + pktdump.write(fuzz(eth / ipv4 / udp))
> > + pktdump.write(fuzz(eth / ipv4 / tcp))
> > + pktdump.write(fuzz(eth / vlan / ipv4 / udp))
> > + pktdump.write(fuzz(eth / vlan / ipv4 / tcp))
> > +
> > + # IPv6 packets with fuzzing
> > + pktdump.write(fuzz(eth / ipv6 / udp))
> > + pktdump.write(fuzz(eth / ipv6 / tcp))
> > + pktdump.write(fuzz(eth / vlan / ipv6 / udp))
> > + pktdump.write(fuzz(eth / vlan / ipv6 / tcp))
> > diff --git a/tests/pcap/mfex_test.pcap b/tests/pcap/mfex_test.pcap
> > new file mode 100644
> > index
> 0000000000000000000000000000000000000000..1aac67b8d643ecb016c758cb
> a4cc32212a80f52a
> > GIT binary patch
> > literal 416
> >
> zcmca|c+)~A1{MYw`2U}Qff2}Q<eHVR>K`M68ITRa|G at yFii5$Gfk6YL%z>@uY&}o
> |
> >
> z2s4N<1VH2&7y^V87$)XGOtD~MV$cFgfG~zBGGJ2#YtF$<F=a4i;9x8Q*<ZrSM6Uf
> z
> > xK>KST_NTIwYriok6N4Vm)gX-
> Q@<yO<!C`>c^{cp<7_5LgK^UuU{2>VS0RZ!RQ+EIW
> >
> > literal 0
> > HcmV?d00001
> >
> > diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at
> > index 802895488..a764c69ca 100644
> > --- a/tests/system-dpdk.at
> > +++ b/tests/system-dpdk.at
> > @@ -232,3 +232,132 @@ OVS_VSWITCHD_STOP(["\@does not exist. The
> Open vSwitch kernel module is probably
> > \@EAL: No free hugepages reported in hugepages-1048576kB at d"])
> > AT_CLEANUP
> > dnl --------------------------------------------------------------------------
> > +
> > +dnl --------------------------------------------------------------------------
> > +dnl Add standard DPDK PHY port
> > +AT_SETUP([OVS-DPDK - MFEX Autovalidator])
> > +AT_KEYWORDS([dpdk])
> > +
> > +OVS_DPDK_START()
> > +
> > +dnl Add userspace bridge and attach it to OVS
> > +AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev])
> > +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk
> options:dpdk-
> devargs=net_pcap1,rx_pcap=$srcdir/pcap/mfex_test.pcap,infinite_rx=1], [],
> [stdout], [stderr])
> > +AT_CHECK([ovs-vsctl show], [], [stdout])
> > +
> > +AT_SKIP_IF([! ovs-appctl dpif-netdev/miniflow-parser-get | sed 1,4d | grep
> "True"], [], [dnl
> > +])
> > +
> > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator], [0],
> [dnl
> > +Miniflow extract implementation set to autovalidator.
> > +])
> > +
> > +OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1 statistics | grep -oP
> 'rx_packets=\s*\K\d+'` -ge 1000])
> > +
> > +dnl Clean up
> > +AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr])
> > +AT_CLEANUP
> > +dnl --------------------------------------------------------------------------
> > +
> > +dnl --------------------------------------------------------------------------
> > +dnl Add standard DPDK PHY port
> > +AT_SETUP([OVS-DPDK - MFEX Autovalidator Fuzzy])
> > +AT_KEYWORDS([dpdk])
> > +AT_SKIP_IF([! pip3 list | grep scapy], [], [])
> > +AT_CHECK([$PYTHON3 $srcdir/mfex_fuzzy.py $srcdir], [], [stdout])
> > +OVS_DPDK_START()
> > +
> > +dnl Add userspace bridge and attach it to OVS
> > +AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev])
> > +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk
> options:dpdk-
> devargs=net_pcap1,rx_pcap=$srcdir/pcap/fuzzy.pcap,infinite_rx=1], [], [stdout],
> [stderr])
> > +AT_CHECK([ovs-vsctl show], [], [stdout])
> > +
> > +AT_SKIP_IF([! ovs-appctl dpif-netdev/miniflow-parser-get | sed 1,4d | grep
> "True"], [], [dnl
> > +])
> > +
> > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator], [0],
> [dnl
> > +Miniflow extract implementation set to autovalidator.
> > +])
> > +
> > +OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1 statistics | grep -oP
> 'rx_packets=\s*\K\d+'` -ge 100000])
> > +
> > +dnl Clean up
> > +AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr])
> > +AT_CLEANUP
> > +dnl --------------------------------------------------------------------------
> > +
> > +dnl --------------------------------------------------------------------------
> > +AT_SETUP([OVS-DPDK - MFEX Configuration])
> > +AT_KEYWORDS([dpdk])
> > +OVS_DPDK_START()
> > +dnl Add userspace bridge and attach it to OVS
> > +AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev])
> > +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk
> options:dpdk-
> devargs=net_pcap1,rx_pcap=$srcdir/pcap/mfex_test.pcap,infinite_rx=1], [],
> [stdout], [stderr])
> > +AT_CHECK([ovs-vsctl show], [], [stdout])
> > +
> > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set scalar 1], [2],
> > +[], [dnl
> > +Error: unknown argument 1.
> > +ovs-appctl: ovs-vswitchd: server returned an error
> > +])
> > +
> > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 3 study 300
> xyz], [2],
> > +[], [dnl
> > +Error: Invalid study_pkt_cnt value: xyz.
> > +ovs-appctl: ovs-vswitchd: server returned an error
> > +])
> > +
> > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set scalar abcd], [2],
> > +[], [dnl
> > +Error: unknown argument abcd.
> > +ovs-appctl: ovs-vswitchd: server returned an error
> > +])
> > +
> > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 0 scalar abcd],
> [2],
> > +[], [dnl
> > +Error: unknown argument abcd.
> > +ovs-appctl: ovs-vswitchd: server returned an error
> > +])
> > +
> > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd], [2],
> > +[], [dnl
> > +Error: -pmd option requires a thread id argument.
> > +ovs-appctl: ovs-vswitchd: server returned an error
> > +])
> > +
> > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set tudy abcd], [2],
> > +[], [dnl
> > +Error: unknown argument abcd.
> > +ovs-appctl: ovs-vswitchd: server returned an error
> > +])
> > +
> > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 3 study abcd],
> [2],
> > +[], [dnl
> > +Error: Invalid study_pkt_cnt value: abcd.
> > +ovs-appctl: ovs-vswitchd: server returned an error
> > +])
> > +
> > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 21 study], [0],
> [dnl
> > +Miniflow extract implementation set to study, on pmd thread 21, studying
> 128 packets.
> > +])
> > +
> > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 21 study 512],
> [0], [dnl
> > +Miniflow extract implementation set to study, on pmd thread 21, studying
> 512 packets.
> > +])
> > +
> > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set study 512], [0], [dnl
> > +Miniflow extract implementation set to study, studying 512 packets.
> > +])
> > +
> > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set study], [0], [dnl
> > +Miniflow extract implementation set to study, studying 128 packets.
> > +])
> > +
> > +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 21
> autovalidator], [0], [dnl
> > +Miniflow extract implementation set to autovalidator, on pmd thread 21.
> > +])
> > +
> > +dnl Clean up
> > +AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr])
> > +AT_CLEANUP dnl
> > +dnl --------------------------------------------------------------------------
> > --
>
> It’s missing some of the negative tests cases, which I have added based on the
> comment on the v6 patch on name changes.
> Here is the diff:
>
> diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at
> index a764c69ca..929bbf0a9 100644
> --- a/tests/system-dpdk.at
> +++ b/tests/system-dpdk.at
> @@ -337,6 +337,30 @@ Error: Invalid study_pkt_cnt value: abcd.
> ovs-appctl: ovs-vswitchd: server returned an error
> ])
>
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd zero study], [2],
> +[], [dnl
> +Error: Miniflow extract parser not changed, PMD thread argument passed is
> not valid: 'zero'. Pass a valid pmd thread ID.
> +ovs-appctl: ovs-vswitchd: server returned an error
> +])
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 1], [2],
> +[], [dnl
> +Error: no miniflow extract name provided. Output of miniflow-parser-get
> shows implementation list.
> +ovs-appctl: ovs-vswitchd: server returned an error
> +])
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 1 superstudy],
> [2],
> +[], [dnl
> +Error: unknown miniflow extract implementation superstudy.
> +ovs-appctl: ovs-vswitchd: server returned an error
> +])
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set superstudy], [2],
> +[], [dnl
> +Error: unknown miniflow extract implementation superstudy.
> +ovs-appctl: ovs-vswitchd: server returned an error
> +])
> +
> +AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 1 study -pmd],
> [2],
> +[], [dnl
> +Error: Invalid study_pkt_cnt value: -pmd.
> +ovs-appctl: ovs-vswitchd: server returned an error
> +])
> +
> AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set -pmd 21 study], [0], [dnl
> Miniflow extract implementation set to study, on pmd thread 21, studying 128
> packets.
> ])
More information about the dev
mailing list