[ovs-discuss] [OVN] Aging mechanism for MAC_Binding table

Han Zhou zhouhan at gmail.com
Wed Aug 21 01:11:12 UTC 2019


On Tue, Aug 20, 2019 at 4:57 PM Ben Pfaff <blp at ovn.org> wrote:
>
> Let me see if I'm following this correctly.  This is what currently
> happens:
>
> - HV1 needs a MAC address for an IP so it broadcasts an ARP request.
>
> - The port with the IP address, on HV2, causes the MAC_Binding to be
>   inserted.
>
> - Every ovn-controller inserts an OF flow for the binding.  HV1 and
>   perhaps other ovn-controllers use this flow to populate the MAC
>   address for subsequent packets destined to the IP address in question.
>
> This proposal augments that with:
>
> - After a while, the binding goes idle and isn't used.  The
>   ovn-controllers gradually notice this and delete their OF flows for
>   it.
>
> - HV3 eventually needs the binding again.  It broadcasts an ARP request.
>
> - The port with the IP address causes the MAC_Binding to be inserted.
>   This might still be on HV2 if the port hasn't moved, or it might be on
>   HV4 if it has.
>
> Is that what you mean?  It might work OK.
>
> Please do update the lifetime description in ovn-sb(5) under the
> MAC_Binding table regardless of what you implement.
>
> Thanks,
>
> Ben.
>
> On Tue, Aug 20, 2019 at 09:03:57AM +0200, Daniel Alvarez Sanchez wrote:
> > Hi folks,
> >
> > Reviving this thread as we're seeing this more and more problematic.
> > Combining the ideas mentioned up thread, Dumitru, Numan, Lucas and I
> > had some internal discussion where we came up with a possible approach
> > and we'd love to get feedback from you:
> >
> > - Local ovn-controller will always insert an OF rule per MAC_Binding
> > entry to match on src_ip + src_mac that will be sampled with a meter
> > to ovn-controller.
> > - When ovn-controller sees that one entry has not been hit "for a
> > while", it'll delete the OpenFlow rule in table 65 that fills the
> > eth.dst field with the MAC_Binding info.

I assume the rules in table 65 can be "extended" for this purpose, instead
of adding extra rules for this.

> > - This will result in further ARP requests from the instance(s) that
> > will refresh the MAC_Binding entries in the database.
> >
> > This could make troubleshooting a bit harder so at some point it'll be
> > great to have a mechanism in OVS where we could disable a flow instead
> > of deleting it. This way, one can tell that the flows in table 65 have
> > been disabled due to the aging mechanism in the local node.

Sorry that I didn't understand this. Why do you want the flow being
disabled instead of deleted? I think if we want to avoid stale entries, we
do want to delete them, so that the stale data doesn't occupy the space in
flow table, neither in SB DB. It may be ok to add debug log for deleting a
aged entry in ovn-controller, for trouble shooting purpose?

> >
> > Thoughts? Is there any performance consideration regarding the extra
> > flows and meters?

Are you proposing shared meters or one meter per mac-binding? If it is per
mac-binding, I would be worried about the scalability considering that we
may have >10k of mac-bindings. Or should I be worried? Maybe Justin and Ben
can comment on the meter scalability. If it is a concern, I would suggest
the feature be configurable (i.e. enable/disable), so that it can be
enabled in environments where aging is required but number of mac-bindings
are not very high.

> >
> > Thanks a lot!
> > Daniel
> >
> >
> > On Tue, Jul 9, 2019 at 7:19 AM Ben Pfaff <blp at ovn.org> wrote:
> > >
> > > On Mon, Jul 08, 2019 at 06:19:23PM -0700, Han Zhou wrote:
> > > > On Thu, Jun 27, 2019 at 6:44 AM Ben Pfaff <blp at ovn.org> wrote:
> > > > >
> > > > > On Tue, Jun 25, 2019 at 01:05:21PM +0200, Daniel Alvarez Sanchez
wrote:
> > > > > > Lately we've been trying to solve certain issues related to
stale
> > > > > > entries in the MAC_Binding table (e.g. [0]). On the other hand,
for
> > > > > > the OpenStack + Octavia (Load Balancing service) use case, we
see that
> > > > > > a reused VIP can be as well affected by stale entries in this
table
> > > > > > due to the fact that it's never bound to a VIF so
ovn-controller won't
> > > > > > claim it and send the GARPs to update the neighbors.
> > > > > >
> > > > > > I'm not sure if other scenarios may suffer from this issue but
seems
> > > > > > reasonable to have an aging mechanism (as we discussed at some
point
> > > > > > in the past) that makes unused/old entries to expire. After
talking to
> > > > > > Numan on IRC, since a new pinctrl thread has been introduced
recently
> > > > > > [1], it'd be nice to implement this aging mechanism there.
> > > > > > At the same time we'd be also reducing the amount of entries
for long
> > > > > > lived systems as it'd grow indefinitely.
> > > > > >
> > > > > > Any thoughts?
> > > > > >
> > > > > > Thanks!
> > > > > > Daniel
> > > > > >
> > > > > > PS. With regards to the 'unused' vs 'old' entries I think it
has to be
> > > > > > 'old' rather than 'unused' as I don't see a way to reset the
TTL of a
> > > > > > MAC_Binding entry when we see packets coming. The implication
is that
> > > > > > we'll be seeing ARPs sent out more often when perhaps they're
not
> > > > > > needed. This also leads to the discussion of making the cache
timeout
> > > > > > configurable.
> > > > >
> > > > > I've always considered the MAC_Binding implementation incomplete
because
> > > > > of this issue and others.  ovn/TODO.rst says:
> > > > >
> > > > >     * Dynamic IP to MAC binding enhancements.
> > > > >
> > > > >       OVN has basic support for establishing IP to MAC bindings
> > > > dynamically, using
> > > > >       ARP.
> > > > >
> > > > >       * Ratelimiting.
> > > > >
> > > > >         From casual observation, Linux appears to generate at
most one
> > > > ARP per
> > > > >         second per destination.
> > > > >
> > > > >         This might be supported by adding a new OVN logical
action for
> > > > >         rate-limiting.
> > > > >
> > > > >       * Tracking queries
> > > > >
> > > > >          It's probably best to only record in the database
responses to
> > > > queries
> > > > >          actually issued by an L3 logical router, so somehow they
have to
> > > > be
> > > > >          tracked, probably by putting a tentative binding without
a MAC
> > > > address
> > > > >          into the database.
> > > > >
> > > > >       * Renewal and expiration.
> > > > >
> > > > >         Something needs to make sure that bindings remain valid
and
> > > > expire those
> > > > >         that become stale.
> > > > >
> > > > >         One way to do this might be to add some support for time
to the
> > > > database
> > > > >         server itself.
> > > > >
> > > > >       * Table size limiting.
> > > > >
> > > > >         The table of MAC bindings must not be allowed to grow
> > > > unreasonably large.
> > > > >
> > > > >       * MTU handling (fragmentation on output)
> > > > >
> > > > > So, what do we do about it?  First, I think that adding support
for time
> > > > > to the database server is a terrible idea (even though I think I
wrote
> > > > > the above originally).  Let's not do that.  The following is some
> > > > > "thinking out loud" on the subject.
> > > > >
> > > > > I think there's a challenge around which ovn-controller should
take care
> > > > > of a given MAC_Binding.  We don't want every ovn-controller
expiring
> > > > > every binding.  Ideally, we want exactly one ovn-controller
expiring a
> > > > > binding.  One way would be to add an owner column (but it would be
> > > > > better if we don't need it).
> > > > >
> > > > > If we want to keep track of "unused" bindings, I can imagine a
> > > > > statistical mechanism to do that.  Any user of a binding
occasionally
> > > > > and probabilistically changes a serial number column that we'd
introduce
> > > > > into the MAC_Binding table (this could be optimized to not bother
if it
> > > > > has changed recently).  The owner checks the serial number every
so
> > > > > often and if it hasn't changed then it deletes the row.
> > > > >
> > > >
> > > > Thanks Ben for the advice. Since the user of a binding is simply a
OpenFlow
> > > > rule matching, I guess we will need "controller" action to trigger
the
> > > > serial number column update in ovn-controller, combined with a
meter action
> > > > so that only small number of packets trigger the update. Is this
what you
> > > > are suggesting?
> > >
> > > I had not thought that far ahead!  That approach would work, although
> > > the trigger percentage would be difficult to figure out--it seems like
> > > really we'd want "every Nth second", not "every Nth packet".  Another
> > > approach that might work would be for ovn-controller to notice the
> > > statistics on appropriate OpenFlow flows changing, or to use "learn"
> > > actions as a way to make a controller action trigger only every so
> > > often.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20190820/9ab6468e/attachment.html>


More information about the discuss mailing list