[ovs-dev] Recirculation with milti-threaded miss handling

[Ethan Jackson]

> In short, it would make my life much easier if facets could be looked up
> execute_flow_miss(). And I am interested to know if you have any plans in
> that regards.

This actually is in the exact opposite of the direction I'm planning
to move.  Facets don't scale to tens of thousands of datapath flows,
are confusing, and are difficult to maintain, so I'm in the process of
cleaning up patches I've written which do away with them entirely.  I
don't have a lot of context on this recirculation stuff, but it sounds
like you'll need some thread-safe global state, completely independent
of facets, which can be used for your purposes.  A giant global hash
table of some sort perhaps.

Comments inline.

>    However, my understanding is that with your recent changes facets aren't
>    available in the thread where misses are executed which seems to be when
>    action translation occurs and the rule is required.

Correct, and indeed facets are going away.

>    I had planned to get around this by creating a new recirculator
>    structure which would hold a recirculation id and flow. The flow
>    holds the parent recirculation id and it should be possible to
>    use this to find the top-most recirculator. This would provide
>    the thus the top-most flow which could be use for rule lookup.

Sounds like the right approach, but again I don't have much context.

>    However I see two problems with this scheme:
>
>    i.  If multiple misses are in-flight but handled in different
>        flow_miss_batches then multiple recirculators would be created for
>        what is in fact the same flow. This doesn't map comfortably to the
>        idea associate a recirculator with a facet. Though in a pinch it
>        ought to be possible to associate multiple recirculators with a
>        facet.

When facets go away flow miss batches will go away to.  A miss handler
will forward the packet and then be done with it, a new thread called
a "revalidator" will later clean up unused flows in the datapath
created by the miss handlers.

>        Another solution would be to allow recirculators to be looked up be
>        classifier. But it seems this would require re-working or augmenting
>        the thread-safety of classifiers as in the scheme I propose
>        recirculators are created when translation occurs and this is not in
>        the main thread where write-access to classifiers is thread-safe.

Perhaps they could be given they're own classifier?  The classifier
itself is thread safe if used within the context of a rwlock.

>    ii.  A more acute problem seems to that it seems that revalidation
>         should occur when rule lookup for a recirculated packet occurs.
>         This is because the scheme described above involves using offsets,
>         calculated during previous action translations, to determine which
>         part of a rules ofpacts should be used when translating actions
>         for a recirculated packet.

Revalidation is also going away.  There will only be the equivalent of
the "expire" function.  If the datapath actions happen to be wrong, of
course we'll fix them, but this won't have any special machinery.

If you'd like I could send you a prototype version of what I have in
the works so you could start planning.  It's a couple of weeks away
from being ready for an official posting on the dev list, but you can
get a sense of the jist.

Ethan