[ovs-dev] sFlow extension for tunnels / MPLS - question about user-space flow-cache

[Neil McKee]

I found the thread where we discussed option (3) back in August 2011.  I
had forgotten that in the earlier implementations we really did send the
actions to userspace with the sample upcall.  It looks like this was taken
out as part of the process of simplifying the kernel module so it could be
upstreamed.  Has anything changed since that might cause this to be
revisited?   In those days we only really wanted one or two fields (egress
port and vlan) but now we want tunnels, mpls labels and more.   In general
sFlow wants to say "Here's the packet,  and here's what I did with it",  so
knowing the full list of actions that were actually applied to that packet
by the datapath seems like the perfect mechanism.

http://openvswitch.org/pipermail/dev/2011-August/010598.html

In the discussion it was proposed that a user-space cache was the way to
go.   That translates to option (1),  but it really does seem to have all
the problems that Jesse and Ben anticipated in 2011.

Neil



------
Neil McKee
InMon Corp.
http://www.inmon.com

On Wed, Apr 1, 2015 at 10:14 PM, Neil McKee <neil.mckee at inmon.com> wrote:

> I've been looking at filling in the sFlow structures to report on tunnel
> encap and other transformations.   sFlow sampling is best done on ingress
> only,  so I can't use the egress-sampling action that the IPFIX
> implementation uses to get the tunnel info.   So how should I look up the
> list of actions for a flow when an sFlow sample appears in
> ofproto-dpif-upcall.c:process_upcall()?  There is not enough room for all
> the fields we need in the (8-byte) userdata-cookie that goes with the
> compiled actions into the kernel and is passed back on the upcall.   But it
> should be possible to look up the whole list of actions in user-space when
> we process the upcall.   Here are some possible ways.   Please comment:
>
> 1)  We can use the concurrent hash-map of flows that the revalidator
> threads are sharing.  The upcall knows the 128-bit ufid for the flow,  so
> we can get directly to the compiled actions like this:
>
> struct udpif_key *ukey = ukey_lookup(updif, upcall->ufid);
> if(ukey) {
>     struct nladdr *actions = ofpbuf_data(ukey->actions);
>     ....
>  }
>
> This seems to work most of the time,  but the way the revalidator threads
> are spinning and flushing seems to make ukey_lookup() fail sometimes and
> return NULL.  Is there a different way to query that would avoid this
> problem?   If this method was reliable we could probably eliminate all the
> code that builds and stores the sFlow userdata-cookie,  and we would have a
> mechanism that would work not just for tunnels,  but for MPLS and other
> actions too.
>
> 2)  look up the rule based on the packet header fields:
>
>   rule_dpif_lookup(upcall->ofproto, flow, .....)
>
> This is not tempting.   Partly because it might match a new rule that had
> only just been inserted and bears no relation to the compiled-rule that
> actually handled the packet we sampled,  and partly because it leads only
> to the rules as they are before compilation (struct ofpacts) so to infer
> exactly what happened to the packet would require compiling them again.
>
> 3) have the kernel pass the actions up with the sFlow upcall.
>
> This seems like it would be a great way to do it (why didn't we do it that
> way from the start?)  --  but of course we don't want to change the kernel
> module if we can avoid it.
>
> 4) If the flow-cache used by the revalidator threads is too ephemeral,
> then maybe the sFlow module can keep it's own cache of active flows,  key'd
> by the same 128-bit ufid and storing only the relevant actions - behaving
> like variable-length cookies.   The hard part there would probably be
> knowing when to flush the rules to keep the hash table as small as
> possible.  Could the revalidator threads tell us when a rule is being
> deleted from the kernel?  Is that worth pursuing?
>
> 5) something else?
>
> Neil
>
>
> ------
> Neil McKee
> InMon Corp.
> http://www.inmon.com
>