[ovs-discuss] ASAN RCU use-after-free

[Eli Britstein]

Hello,

I am trying to use Address Sanitizer to detect issues.
With a simple code of use-after-free it works, but with postponed free,
there
is no detection of the problem.

Compilation is done with:
make -j CFLAGS="-O0 -g3 -Werror -fsanitize=address
-fno-omit-frame-pointer -fno-common"

Simple code:

--- a/lib/dpif-netdev.c
+++ b/lib/dpif-netdev.c
@@ -2742,6 +2742,12 @@ dp_netdev_flow_offload_main(void *data OVS_UNUSED)
                                   &dp_flow_offload.mutex);
               ovsrcu_quiesce_end();
           }
+        if (1) {
+            char *xx = xmalloc(80);
+            xx[0] = 'a';
+            free(xx);
+            xx[1] = 'b';
+        }
           list = ovs_list_pop_front(&dp_flow_offload.list);

Running a test:
make -j check TESTSUITEFLAGS=991

Problem is detected:
991: dpif-netdev - partial hw offload - dummy        FAILED
(ovs-macros.at:242)

tests/testsuite.dir/0991/asan.30479
=================================================================
==30479== ERROR: AddressSanitizer: heap-use-after-free on address
0x600e0001bfb1 at pc 0x20dcb79 bp 0x7fb60ca1e200 sp 0x7fb60ca1e1f0
WRITE of size 1 at 0x600e0001bfb1 thread T17
...

With RCU, if we force the free:
diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
index 300861ca5..71b48dbd8 100644
--- a/lib/dpif-netdev.c
+++ b/lib/dpif-netdev.c
@@ -2726,6 +2726,14 @@ err_free:
       return -1;
   }

+static volatile bool postpone_executed = false;
+static void
+flagged_free(void *p)
+{
+    free(p);
+    postpone_executed = true;
+}
+
   static void *
   dp_netdev_flow_offload_main(void *data OVS_UNUSED)
   {
@@ -2742,6 +2750,14 @@ dp_netdev_flow_offload_main(void *data OVS_UNUSED)
                                   &dp_flow_offload.mutex);
               ovsrcu_quiesce_end();
           }
+        if (1) {
+            char *xx = xmalloc(80);
+            xx[0] = 'a';
+            ovsrcu_postpone(flagged_free, xx);
+            ovsrcu_quiesce();
+            while (!postpone_executed);
+            xx[1] = 'b';
+        }
           list = ovs_list_pop_front(&dp_flow_offload.list);

The problem is also detected.

However, if removing the flag:
--- a/lib/dpif-netdev.c
+++ b/lib/dpif-netdev.c
@@ -2726,12 +2726,10 @@ err_free:
       return -1;
   }

-static volatile bool postpone_executed = false;
   static void
   flagged_free(void *p)
   {
       free(p);
-    postpone_executed = true;
   }

   static void *
@@ -2755,7 +2753,6 @@ dp_netdev_flow_offload_main(void *data OVS_UNUSED)
               xx[0] = 'a';
               ovsrcu_postpone(flagged_free, xx);
               ovsrcu_quiesce();
-            while (!postpone_executed);
               xx[1] = 'b';
           }
           list = ovs_list_pop_front(&dp_flow_offload.list);

The problem is not detected.

This way it is up to a race between the RCU thread and the write of xx[1].
Any thoughts of a better tool or technique that is more suitable?

Thanks,
Eli