[ovs-dev] ovs_assert_failure (where=where at entry=0x7ffff6f1d73f "lib/poll-loop.c:111", function=function at entry=0x7ffff6f1d7a0 <__func__.7424> "

Eric Li erictheone at gmail.com
Sat Jan 2 19:42:03 UTC 2021


Happy new year everyone, I believe the assert is benign and open the below
issue for tracking:

https://github.com/openvswitch/ovs-issues/issues/199

Thanks,
Eric

On Thu, Dec 24, 2020 at 9:38 AM Eric Li <erictheone at gmail.com> wrote:

> Hi All,
>
> My first post, so please be gentle on me :)
>
> We have an app that wants to add a lot of openflow rules quickly. To speed
> things up, the app is creating multiple threads to do its logic and then
> call into ovs library code directly. I am getting the below assert about
> 50% of time on fast machines. Need help to understand what that assert
> means or suggest a better approach.
>
> Questions in my mind:
>
> 1. I am thinking about using "bundle" to push in more openflow rules in
> one shot, what is a good number of rules to bundle together? 10, 100, 1000?
> 2. I tried to use a shared vonn in multiple threads but hitting other
> weird openflow errors, I don't think that's the right approach right?
>
> Thanks a lot!
> Eric
>
> poll-loop.c: line 111:
>
> static void
>
> poll_create_node(int fd, HANDLE wevent, short int events, const char
> *where)
>
> {
>
>     struct poll_loop *loop = poll_loop();
>
>     struct poll_node *node;
>
>
>     COVERAGE_INC(poll_create_node);
>
>
>     /* Both 'fd' and 'wevent' cannot be set. */
>
>     ovs_assert(!fd != !wevent);  <-- !!!assert hit here because wevent=0
> and fd=0!!!
>
>
>     /* Check for duplicate.  If found, "or" the events. */
>
>     node = find_poll_node(loop, fd, wevent);
>
>     if (node) {
>
>         node->pollfd.events |= events;
>
>     } else {
>
>         node = xzalloc(sizeof *node);
>
>         hmap_insert(&loop->poll_nodes, &node->hmap_node,
>
>                     hash_2words(fd, (uint32_t)wevent));
>
>         node->pollfd.fd = fd;
>
>         node->pollfd.events = events;
>
> #ifdef _WIN32
>
>         if (!wevent) {
>
>             wevent = CreateEvent(NULL, FALSE, FALSE, NULL);
>
>         }
>
> #endif
>
>         node->wevent = wevent;
>
>         node->where = where;
>
>     }
>
> }
>
> (gdb) bt
>
> #0  __GI_raise (sig=sig at entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
>
> #1  0x00007ffff61c18b1 in __GI_abort () at abort.c:79
>
> #2  0x00007ffff6ea1ade in ovs_abort_valist (err_no=err_no at entry=0,
> format=format at entry=0x7ffff6f21bd0 "%s: assertion %s failed in %s()",
>
>     args=args at entry=0x7fffeeffb890) at lib/util.c:419
>
> #3  0x00007ffff6ea9de0 in vlog_abort_valist (module_=<optimized out>,
> message=0x7ffff6f21bd0 "%s: assertion %s failed in %s()", args=args at entry
> =0x7fffeeffb890)
>
>     at lib/vlog.c:1227
>
> #4  0x00007ffff6ea9e74 in vlog_abort (module=module at entry=0x7ffff71b6960
> <this_module>, message=message at entry=0x7ffff6f21bd0 "%s: assertion %s
> failed in %s()")
>
>     at lib/vlog.c:1241
>
> #5  0x00007ffff6ea17bc in ovs_assert_failure (where=where at entry=0x7ffff6f1d73f
> "lib/poll-loop.c:111",
>
>     function=function at entry=0x7ffff6f1d7a0 <__func__.7424>
> "poll_create_node", condition=condition at entry=0x7ffff6f1d730 "!fd !=
> !wevent") at lib/util.c:86
>
> #6  0x00007ffff6e826b6 in poll_create_node (wevent=0,
> where=0x7ffff6f023a4 "lib/fatal-signal.c:224", events=1, fd=0) at
> lib/poll-loop.c:111
>
> #7  poll_fd_wait_at (fd=0, events=events at entry=1, where=where at entry=0x7ffff6f023a4
> "lib/fatal-signal.c:224") at lib/poll-loop.c:149
>
> #8  0x00007ffff6de6684 in fatal_signal_wait () at lib/fatal-signal.c:224
>
> #9  0x00007ffff6e82777 in poll_block () at lib/poll-loop.c:331
>
> #10 0x00007ffff6ea66eb in vconn_connect_block (vconn=0x7fffe0001bf0,
> timeout=<optimized out>) at lib/vconn.c:722
>
> #11 0x00005555558320c9 in ovs_control::OVS_Control::open_vconn
> (this=0x555555c32de8 <ovs_control::OVS_Control::get_instance()::instance>,
>
>
>     name=0x5555559869c8 "br-tun", vconnp=0x7fffeeffbbb0) at
> /root/aca-dev/src/ovs/ovs_control.cpp:642
>
> #12 0x00005555558317e7 in
> ovs_control::OVS_Control::open_vconn_for_flow_mod (this=0x555555c32de8
> <ovs_control::OVS_Control::get_instance()::instance>,
>
>     remote=0x5555559869c8 "br-tun", vconnp=0x7fffeeffbbb0,
>
>     usable_protocols=(OFPUTIL_P_OF10_NXM_TID | OFPUTIL_P_OF12_OXM |
> OFPUTIL_P_OF13_OXM | OFPUTIL_P_OF14_OXM | OFPUTIL_P_OF15_OXM))
>
>     at /root/aca-dev/src/ovs/ovs_control.cpp:501
>
> #13 0x000055555583157a in ovs_control::OVS_Control::flow_mod__
> (this=0x555555c32de8 <ovs_control::OVS_Control::get_instance()::instance>,
>
>
>     remote=0x5555559869c8 "br-tun", fms=0x7fffeeffbc40, n_fms=1,
>
>     usable_protocols=(OFPUTIL_P_OF10_NXM_TID | OFPUTIL_P_OF12_OXM |
> OFPUTIL_P_OF13_OXM | OFPUTIL_P_OF14_OXM | OFPUTIL_P_OF15_OXM))
>
>     at /root/aca-dev/src/ovs/ovs_control.cpp:442
>
> #14 0x0000555555831365 in ovs_control::OVS_Control::flow_mod
> (this=0x555555c32de8 <ovs_control::OVS_Control::get_instance()::instance>,
>
>
>     bridge=0x5555559869c8 "br-tun",
>
>     flow=0x7fffe0001c80
> "table=20,priority=50,dl_vlan=1,dl_dst:fa:16:3e:d7:f2:6c,actions=strip_vlan,load:20->NXM_NX_TUN_ID[],set_field:123.0.0.4->tun_dst,output:100",
> command=0) at /root/aca-dev/src/ovs/ovs_control.cpp:410
>
> #15 0x00005555558310dd in ovs_control::OVS_Control::add_flow
> (this=0x555555c32de8 <ovs_control::OVS_Control::get_instance()::instance>,
>
>
>     bridge=0x5555559869c8 "br-tun",
>
>     flow=0x7fffe0001c80
> "table=20,priority=50,dl_vlan=1,dl_dst:fa:16:3e:d7:f2:6c,actions=strip_vlan,load:20->NXM_NX_TUN_ID[],set_field:123.0.0.4->tun_dst,output:100")
> at /root/aca-dev/src/ovs/ovs_control.cpp:375
>
> ...
>


More information about the dev mailing list