[ovs-dev] Scaling of Logical_Flows and MAC_Binding tables

[Dumitru Ceara]

On 11/25/20 3:30 PM, Renat Nurgaliyev wrote:
> Hello folks,
> 

Hi Renat,

> we run a lab where we try to evaluate scalability potential of OVN with
> OpenStack as CMS.
> Current lab setup is following:
> 
> 500 networks
> 500 routers
> 1500 VM ports (3 per network/router)
> 1500 Floating IPs (one per VM port)
> 
> There is an external network, which is bridged to br-provider on gateway
> nodes. There are 2000 ports
> connected to this external network (1500 Floating IPs + 500 SNAT router
> ports). So the setup is not
> very big we'd say, but after applying this configuration via ML2/OVN
> plugin, northd kicks in and does
> its job, and after its done, Logical_Flow table gets 645877 entries,
> which is way too much. But ok,
> we move on and start one controller on the gateway chassis, and here
> things get really messy.
> MAC_Binding table grows from 0 to 999088 entries in one moment, and
> after its done, the size of SB
> biggest tables look like this:
> 
> 999088 MAC_Binding
> 645877 Logical_Flow
> 4726 Port_Binding
> 1117 Multicast_Group
> 1068 Datapath_Binding
> 1046 Port_Group
> 551 IP_Multicast
> 519 DNS
> 517 HA_Chassis_Group
> 517 HA_Chassis
> ...
> 
> MAC binding table gets huge, basically it now has an entry for every
> port that is connected to external
> network * number of datapaths, which roughly makes it one million
> entries. This table by itself increases
> the size of the SB by 200 megabytes. Logical_Flow table also gets very
> heavy, we have already played a bit
> with logical datapath patches that Ilya Maximets submitted, and it looks
> much better, but the size of
> the MAC_Binding table still feels inadequate.
> 
> We would like to start to work at least on MAC_Binding table
> optimisation, but it is a bit difficult
> to start working from scratch. Can someone help us with ideas how this
> could be optimised?
> 
> Maybe it would also make sense to group entries in MAC_Binding table in
> the same way like it is proposed
> for logical flows in Ilya's patch?
> 

Maybe it would work but I'm not really sure how, right now.  However,
what if we change the way MAC_Bindings are created?

Right now a MAC Binding is created for each logical router port but in
your case there are a lot of logical router ports connected to the
single provider logical switch and they all learn the same ARPs.

What if we instead store MAC_Bindings per logical switch?  Basically
sharing all these MAC_Bindings between all router ports connected to the
same LS.

Do you see any problem with this approach?

Thanks,
Dumitru