[ovs-dev] [ovs-discuss] OVN nb-db and sb-db out of sync

Numan Siddique numans at ovn.org
Thu Jul 23 10:15:07 UTC 2020


On Thu, Jul 23, 2020 at 8:22 AM Tony Liu <tonyliu0592 at hotmail.com> wrote:

> Hi,
>
> I see why sb-db broke at 1568th port-binding.
> The 1568th datapath-binding in sb-db references the same
>
> _uuid               : 108cf745-db82-43c0-a9d3-afe27a41e4aa
> external_ids        :
> {logical-switch="8a5d1d3c-e9fc-4cbe-a461-98ff838e6473",
> name=neutron-e907dc17-f1e8-4217-a37d-86e9a98c86c2, name2=net-97-192}
> tunnel_key          : 1567
>
> _uuid               : d934ed92-2f3c-4b31-8a76-2a5047a3bb46
> external_ids        :
> {logical-switch="8a5d1d3c-e9fc-4cbe-a461-98ff838e6473",
> name=neutron-e907dc17-f1e8-4217-a37d-86e9a98c86c2, name2=net-97-192}
> tunnel_key          : 1568
>
> I don't believe this is supposed to happen. Any idea how could it happen?
> Then ovn-northd is stuck in trying to delete this duplication, and it
> ignores all the following updates.
> That caused out-of-sync between nb-db and sb-db.
> Any way I can fix it manually, like with ovn-sbctl to delete it?
>

If you delete the ovn sb db resources manually, ovn-northd should sync it
up.

But I'm surprised why ovn-northd didn't sync earlier. There's something
wrong related to raft going
on here. Not sure what.

Thanks
Numan



>
> Thanks!
>
> Tony
>
> ------------------------------
> *From:* dev <ovs-dev-bounces at openvswitch.org> on behalf of Tony Liu <
> tonyliu0592 at hotmail.com>
> *Sent:* July 22, 2020 11:33 AM
> *To:* ovs-dev <ovs-dev at openvswitch.org>
> *Subject:* [ovs-dev] OVN nb-db and sb-db out of sync
>
> Hi,
>
> During a scaling test where 4000 networks are created from OpenStack, I
> see that
> nb-db and sb-db are out of sync. All 4000 logical switches and 8000 LS
> ports
> (GW port and service port of each network) are created in nb-db. In sb-db,
> only 1567 port-bindings, 4000 is expected.
>
> [root at ovn-db-2 ~]# ovn-nbctl list nb_global
> _uuid               : b7b3aa05-f7ed-4dbc-979f-10445ac325b8
> connections         : []
> external_ids        : {"neutron:liveness_check_at"="2020-07-22
> 04:03:17.726917+00:00"}
> hv_cfg              : 312
> ipsec               : false
> name                : ""
> nb_cfg              : 2636
> options             : {mac_prefix="ca:e8:07",
> svc_monitor_mac="4e:d0:3a:80:d4:b7"}
> sb_cfg              : 2005
> ssl                 : []
>
> [root at ovn-db-2 ~]# ovn-sbctl list sb_global
> _uuid               : 3720bc1d-b0da-47ce-85ca-96fa8d398489
> connections         : []
> external_ids        : {}
> ipsec               : false
> nb_cfg              : 312
> options             : {mac_prefix="ca:e8:07",
> svc_monitor_mac="4e:d0:3a:80:d4:b7"}
> ssl                 : []
>
> Is there any way to force ovn-northd to rebuild sb-db to sync with nb-db,
> like manipulating nb_cfg or anything else? Note, it's 3-node RAFT cluster
> for both
> nb-db and sb-db.
>
> Is that "incremental update" implemented in 20.03?
> If not, in which release it's going to be available?
>
>
> Thanks!
>
> Tony
>
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> _______________________________________________
> discuss mailing list
> discuss at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>


More information about the dev mailing list