[ovs-dev] [PATCH 11/11] pkt-metadata: Avoid introducing overhead for userspace tunnels.

[Jesse Gross]

The addition of Geneve metadata requires a large amount of additional
space to handle the maximum set of options. In most cases, this is
not a big deal since it is only temporary storage on the stack or
can be automatically stripped out for miniflows. However, userspace
tunnels need to deal with this on a per-packet basis, so we should
avoid introducing additional overhead if possible. Two small changes
are aimed at this:

 * Move struct flow_tnl to the end of the packet metadata. Since
   the Geneve metadata is already at the end of flow_tnl and pkt_metadata
   is at the end of struct dp_packet, this avoids putting a large
   amount metadata (which might be empty) in hot cache lines.

 * Only push the new metadata into a miniflow if any options are present
   during miniflow_extract(). This does not necessarily provide the
   most fine-grained flow generation but it is a quick check and
   the userspace implementation of Geneve does not currently support
   options anyways.

Signed-off-by: Jesse Gross <jesse at nicira.com>
---
 lib/flow.c    | 7 ++++++-
 lib/packets.h | 2 +-
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/lib/flow.c b/lib/flow.c
index 7350a17..6bfe738 100644
--- a/lib/flow.c
+++ b/lib/flow.c
@@ -438,7 +438,12 @@ miniflow_extract(struct dp_packet *packet, struct miniflow *dst)
     /* Metadata. */
     if (md->tunnel.ip_dst) {
         miniflow_push_words(mf, tunnel, &md->tunnel,
-                            sizeof md->tunnel / sizeof(uint64_t));
+                            offsetof(struct flow_tnl, metadata) /
+                            sizeof(uint64_t));
+        if (md->tunnel.metadata.opt_map) {
+            miniflow_push_words(mf, tunnel.metadata, &md->tunnel.metadata,
+                                 sizeof md->tunnel.metadata / sizeof(uint64_t));
+        }
     }
     if (md->skb_priority || md->pkt_mark) {
         miniflow_push_uint32(mf, skb_priority, md->skb_priority);
diff --git a/lib/packets.h b/lib/packets.h
index 688e7373..04ee914 100644
--- a/lib/packets.h
+++ b/lib/packets.h
@@ -63,10 +63,10 @@ struct pkt_metadata {
                                    received from the wire. */
     uint32_t dp_hash;           /* hash value computed by the recirculation
                                    action. */
-    struct flow_tnl tunnel;     /* Encapsulating tunnel parameters. */
     uint32_t skb_priority;      /* Packet priority for QoS. */
     uint32_t pkt_mark;          /* Packet mark. */
     union flow_in_port in_port; /* Input port. */
+    struct flow_tnl tunnel;     /* Encapsulating tunnel parameters. */
 };
 
 #define PKT_METADATA_INITIALIZER(PORT) \
-- 
2.1.0