[ovs-discuss] OVN at scale in production

Numan Siddique numans at ovn.org
Wed Oct 6 20:44:15 UTC 2021


On Wed, Oct 6, 2021 at 2:49 PM Seena Fallah <seenafallah at gmail.com> wrote:
>
> I'm using these versions on a centos container:
> ovsdb-server (Open vSwitch) 2.15.2
> ovn-nbctl 21.06.0
> Open vSwitch Library 2.15.90
> DB Schema 5.32.0
>
> Today I see the election timed out too and I should increase ovsdb election timeout too. I saw the commits but I didn't find any related change to my problem.
> If I use ovn 21.09 with ovsdb 2.16 Is there still any need to increase election timeout and disable the inactivity probe?

Not sure on that.  It's worth a try if you have a test environment.

> Also is there any limitation on the number of ACLs that can OVN handle?

I don't think there is any limitation on the number of ACLs.  In
general as the size of the SB DB increases, we have seen issues.

Can you run the below command on each of your nodes where
ovn-controller runs and see if that helps ?

---
ovs-vsctl set open . external_ids:ovn-monitor-all=true
---

Thanks
Numan


>
> Thanks.
>
> On Wed, Oct 6, 2021 at 9:43 PM Numan Siddique <numans at ovn.org> wrote:
>>
>> On Wed, Oct 6, 2021 at 12:15 PM Seena Fallah <seenafallah at gmail.com> wrote:
>> >
>> > Hi,
>> >
>> > I use ovn for OpenStack neutron plugin for my production. After days I see issues about losing a leader in ovsdb. It seems it was because of the failing inactivity probe and because I had 17k acls. After I disable the inactivity probe it works fine but when I did a scale test on it (about 40k ACLS) again it fails the leader.
>> > I saw many docs about ovn at scale issues that were raised by both RedHat and eBay and seems the solution is to rewrite ovn with ddlog. I checked it with northd-ddlog but nothing changes.
>> >
>> > My question is should I wait more for ovn to be stable for high scale or is there any tuning I miss in my deployment?
>> > Also, will the ovn-nb/sb rewrite with ddlog and can help the issues at a high scale? if yes is there any due time?
>>
>> What is the ovsdb-server version you're using ?  There are many
>> improvements in the ovsdb-server in 2.16.
>> Maybe that would help in your deployment.  And also there were many
>> improvements which went into OVN 21.09
>> if you want to test it out.
>>
>> Thanks
>> Numan
>>
>> >
>> > Thanks.
>> > _______________________________________________
>> > discuss mailing list
>> > discuss at openvswitch.org
>> > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>
> _______________________________________________
> discuss mailing list
> discuss at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss


More information about the discuss mailing list