[ovs-discuss] [OVN] flow explosion in lr_in_arp_resolve table

[Girish Moodalbail]

On Sat, May 16, 2020 at 12:36 AM Han Zhou <zhouhan at gmail.com> wrote:

>
>
> On Tue, May 5, 2020 at 11:57 AM Han Zhou <hzhou at ovn.org> wrote:
> >
> >
> >
> > On Fri, May 1, 2020 at 2:14 PM Dan Winship <danwinship at redhat.com>
> wrote:
> > >
> > > On 5/1/20 12:37 PM, Girish Moodalbail wrote:
> > > > If we now look at table=12 (lr_in_arp_resolve) in the ingress
> pipeline
> > > > of Gateway Router-1, then you will see that there will be 2000
> logical
> > > > flow entries...
> > >
> > > > In the topology above, the only intended path is North-South between
> > > > each gateway router and the logical router. There is no east-west
> > > > traffic between the gateway routers
> > >
> > > > Is there an another way to solve the above problem with just keeping
> the
> > > > single join logical switch?
> > >
> > > Two thoughts:
> > >
> > > 1. In openshift-sdn, the bridge doesn't try to handle ARP itself. It
> > > just lets ARP requests pass through normally, and lets ARP replies pass
> > > through normally as long as they are correct (ie, it doesn't let
> > > spoofing through). This means fewer flows but more traffic. Maybe
> that's
> > > the right tradeoff?
> > >
> > The 2M entries here is not for ARP responder, but more equivalent to the
> neighbour table (or ARP cache), on each LR. The ARP responder resides in
> the LS (join logical switch), which is O(n) instead of O(n^2), so it is not
> a problem here.
> >
> > However, a similar idea may works here to avoid the O(n^2) scale issue.
> For the neighbour table, actually OVN has two parts, one is statically
> build, which is the 2M entires mentioned in this case, and the other is the
> dynamic ARP resolve - the mac_binding table, which is dynamically populated
> by handling ARP messages. To solve the problem here, it is possible to
> change OVN to support configuring a LR to avoid static neighbour table, and
> relies only on dynamic ARP resolving. In this case, all the gateway routers
> can be configured as not using static ARP resolving, and eventually there
> will be only 2 entries (one for IPv4 and one for IPv6) for each gateway
> router in mac_binding table for the north-south traffic to the join router.
> (of source there will be still same amount of mac_bindings in each router
> for the external traffic on the other side of the gateway routers).
> >
> > This change seems straightforward, but I am not sure if there is any
> corner cases.
>
> Hi Girish,
>
> I've sent a RFC patch here for the above proposal:
> https://patchwork.ozlabs.org/project/openvswitch/patch/1589614395-99499-1-git-send-email-hzhou@ovn.org/
> For this use case, just set options:dynamic_neigh_routes=true for all the
> Gateway Routers. Could you try it in your scale environment and see if it
> solves the problem?
>
> Thanks,
> Han
>
> >
> > > 2. In most places in ovn-kubernetes, our MAC addresses are
> > > programmatically related to the corresponding IP addresses, and in
> > > places where that's not currently true, we could try to make it true,
> > > and then perhaps the thousands of rules could just be replaced by a
> > > single rule?
> > >
> > This may be a good idea, but I am not sure how to implement in OVN to
> make it generic, since most OVN users can't make such assumption.
> >
> > On the other hand, why wouldn't splitting the join logical switch to
> 1000 LSes solve the problem? I understand that there will be 1000 more
> datapaths, and 1000 more LRPs, but these are all O(n), which is much more
> efficient than the O(n^2) exploding. What's the other scale issues created
> by this?
> >
> > In addition, Girish, for the external LS, I am not sure why can't it be
> shared, if all the nodes are connected to a single L2 network. (If they are
> connected to separate L2 networks, different external LSes should be
> created, at least according to current OVN model).
>

Thanks Han for the patch. Will give it a try and let you know.

Regards,
~Girish


> >
> > Thanks,
> > Han
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20200516/af55d2ba/attachment.html>