[ovs-dev] Possible Regression due to "ossfuzz: Break flow test target into two targets to speed up fuzzing."

Lorenzo Bianconi lorenzo.bianconi at redhat.com
Mon Oct 15 21:31:55 UTC 2018


>
> On Fri, Oct 12, 2018 at 08:54:49AM -0700, Ben Pfaff wrote:
> > On Fri, Oct 12, 2018 at 03:44:51PM +0300, Ilya Maximets wrote:
> > > > Hi,
> > > >
> > > > it seems that travis-ci is failing due to a testsuite regression introduced
> > > > by 1adcbcee8f4c ("ossfuzz: Break flow test target into two targets to speed
> > > > up fuzzing.").
> > > >
> > > > https://travis-ci.org/openvswitch/ovs/jobs/439811394
> > >
> > > Hi Simon,
> > >
> > > As it was already said, this is not the patch that produces the issue.
> > > I'm able to reproduce '2649: ovn -- 3 HVs, 3 LS, 3 lports/LS, 1 LR'
> > > test failure locally by restricting to use only one CPU core with taskset.
> > > So, I bisected the issue and found that the fist patch that fails is:
> > >
> > >     2e5cdb4b1392 ("OVN: add buffering support for ip packets").
> > >
> > > Issue happens only if single core assigned:
> > >
> > >     taskset -c 2   make check TESTSUITEFLAGS='2649'  #FAILS
> > >     taskset -c 2-3 make check TESTSUITEFLAGS='2649'  #OK
> > >
> > > As Travis has no much CPU resources it fails all the time.
> > >
> > > I'm not much familiar with OVN code/tests, hope above information will
> > > be useful for your investigation.
> >
> > I can reproduce this too, with your hint (although it's test 3309, not
> > 2649).
> >
> > The failure is because a lot of expected packets don't show up:
> >
> >     ...
> >     rcv_n=212 exp_n=233
> >     ovn.at:12: wait failed after 10 seconds
> >
> > It needs more looking into.
>
> I don't have time to look into it, but we need the tests passing, so I
> sent a revert: https://patchwork.ozlabs.org/patch/984336/

Hi all,

I looked into it and it is probably a timing issue. In particular the
ARP replies generated during test 8
do not reach the router before the ip buffering gc time-out so we will
end up having less packets than
expected.
A trivial solution is to invert test 8 and 4,5,6 or maybe we can add a
unixctl command to disable ip
buffering.
Could you please try following patch? it fixes in my tests

Regards,
Lorenzo

diff --git a/tests/ovn.at b/tests/ovn.at
index 002ef0dfb..562d96533 100644
--- a/tests/ovn.at
+++ b/tests/ovn.at
@@ -2673,17 +2673,62 @@ test_arp() {
     fi
 }

+: > mac_bindings.expected
+# 4. Generate an ARP reply for each of the IP addresses ARPed for
+#    earlier as #3.
+#
+#    Here, the $s is the VIF that originated the ARP request and $d is
+#    the VIF that sends the ARP reply, which is somewhat backward but
+#    it means that $s and $d are the same as #3.
+for is in 1 2 3; do
+  for js in 1 2 3; do
+    for ks in 1 2 3; do
+      s=$is$js$ks
+      for id in 1 2 3; do
+        for jd in 1 2 3; do
+          if test $is$js = $id$jd; then
+            continue
+          fi
+
+          kd=1
+          d=$id$jd$kd
+
+          o4=`expr $is '*' 9 + $js '*' 3 + $ks + 10`
+          host_ip=`ip_to_hex 192 168 $id$jd $o4`
+          host_mac=8000000000$o4
+
+          lrmac=00000000ff$id$jd
+          lrip=`ip_to_hex 192 168 $id$jd 254`
+
+          arp=${lrmac}${host_mac}08060001080006040002${host_mac}${host_ip}${lrmac}${lrip}
+
+          echo
+          echo
+          echo
+          hv=hv`vif_to_hv $d`
+          as $hv ovs-appctl netdev-dummy/receive vif$d $arp
+          #as $hv ovs-appctl ofproto/trace br-int in_port=$d $arp
+          #as $hv ovs-ofctl dump-flows br-int table=19
+
+          host_ip_pretty=192.168.$id$jd.$o4
+          host_mac_pretty=80:00:00:00:00:$o4
+          echo lrp$id$jd,$host_ip_pretty,$host_mac_pretty >>
mac_bindings.expected
+        done
+      done
+    done
+  done
+done
+
 # Test router replies to ARP requests from all source ports:
 #
-# 4. Router replies to query for its MAC address from port's own IP address.
+# 5. Router replies to query for its MAC address from port's own IP address.
 #
-# 5. Router replies to query for its MAC address from any random IP address
+# 6. Router replies to query for its MAC address from any random IP address
 #    in its subnet.
 #
-# 6. No reply to query for IP address other than router IP.
+# 7. No reply to query for IP address other than router IP.
 #
-# 7. No reply to query from another subnet.
-: > mac_bindings.expected
+# 8. No reply to query from another subnet.
 for i in 1 2 3; do
   for j in 1 2 3; do
     for k in 1 2 3; do
@@ -2727,50 +2772,6 @@ for i in 1 2 3; do
   done
 done

-# 8. Generate an ARP reply for each of the IP addresses ARPed for
-#    earlier as #3.
-#
-#    Here, the $s is the VIF that originated the ARP request and $d is
-#    the VIF that sends the ARP reply, which is somewhat backward but
-#    it means that $s and $d are the same as #3.
-for is in 1 2 3; do
-  for js in 1 2 3; do
-    for ks in 1 2 3; do
-      s=$is$js$ks
-      for id in 1 2 3; do
-        for jd in 1 2 3; do
-          if test $is$js = $id$jd; then
-            continue
-          fi
-
-          kd=1
-          d=$id$jd$kd
-
-          o4=`expr $is '*' 9 + $js '*' 3 + $ks + 10`
-          host_ip=`ip_to_hex 192 168 $id$jd $o4`
-          host_mac=8000000000$o4
-
-          lrmac=00000000ff$id$jd
-          lrip=`ip_to_hex 192 168 $id$jd 254`
-
-          arp=${lrmac}${host_mac}08060001080006040002${host_mac}${host_ip}${lrmac}${lrip}
-
-          echo
-          echo
-          echo
-          hv=hv`vif_to_hv $d`
-          as $hv ovs-appctl netdev-dummy/receive vif$d $arp
-          #as $hv ovs-appctl ofproto/trace br-int in_port=$d $arp
-          #as $hv ovs-ofctl dump-flows br-int table=19
-
-          host_ip_pretty=192.168.$id$jd.$o4
-          host_mac_pretty=80:00:00:00:00:$o4
-          echo lrp$id$jd,$host_ip_pretty,$host_mac_pretty >>
mac_bindings.expected
-        done
-      done
-    done
-  done
-done

 # Allow some time for packet forwarding.
 # XXX This can be improved.


More information about the dev mailing list