[ovs-git] [openvswitch/ovs] d09543: revalidator: Use 'cmap' for storing ukeys.

GitHub noreply at github.com
Wed Oct 22 00:56:42 UTC 2014


  Branch: refs/heads/per_pmd_cls_10_21
  Home:   https://github.com/openvswitch/ovs
  Commit: d09543d61c86500c5cefffc17c6344172bc1e0cf
      https://github.com/openvswitch/ovs/commit/d09543d61c86500c5cefffc17c6344172bc1e0cf
  Author: Joe Stringer <joestringer at nicira.com>
  Date:   2014-10-06 (Mon, 06 Oct 2014)

  Changed paths:
    M ofproto/ofproto-dpif-upcall.c

  Log Message:
  -----------
  revalidator: Use 'cmap' for storing ukeys.

Signed-off-by: Joe Stringer <joestringer at nicira.com>
Acked-by: Ben Pfaff <blp at nicira.com>
---
v4-v7: No change.
v3: Rebase.
v2: Call ovsrcu_quiesce() unconditionally.
RFC: Initial Post.


  Commit: 26e46a182fffe3b3aa23010ca4b6574cec542ff3
      https://github.com/openvswitch/ovs/commit/26e46a182fffe3b3aa23010ca4b6574cec542ff3
  Author: Joe Stringer <joestringer at nicira.com>
  Date:   2014-10-06 (Mon, 06 Oct 2014)

  Changed paths:
    M ofproto/ofproto-dpif-upcall.c

  Log Message:
  -----------
  revalidator: Protect ukeys with a mutex.

Currently, udpif_keys are protected during revalidator_sweep__() as only
one thread accesses the ukey at a time. This is ensured using barriers:
all revalidators will be in the GC phase, so they will only access their
own ukey collection.

A future patch will change the access patterns to allow these ukey
collections to be read or modified while a revalidator is garbage
collecting it. To protect the ukeys, this patch adds locking on the ukey
collection.

Signed-off-by: Joe Stringer <joestringer at nicira.com>
Acked-by: Ben Pfaff <blp at nicira.com>
---
v4-v7: No change.
v3: Rebase.
v2: No change.
RFC: First post.


  Commit: db27d98f2a665c3d53ac1ea1e7cf181430bfa323
      https://github.com/openvswitch/ovs/commit/db27d98f2a665c3d53ac1ea1e7cf181430bfa323
  Author: Joe Stringer <joestringer at nicira.com>
  Date:   2014-10-06 (Mon, 06 Oct 2014)

  Changed paths:
    M ofproto/ofproto-dpif-upcall.c

  Log Message:
  -----------
  udpif: Separate udpif_key maps from revalidators.

An upcoming patch will change the access patterns for ukey maps to
increase the number of writers, and shift write-access from revalidator
threads to upcall handler threads. As such, it no longer makes sense to
tie these maps to revalidators in a 1:1 relationship.

This patch separates the ukey maps from the revalidators, and increases
the number of maps used to store ukeys, to reduce contention.

Signed-off-by: Joe Stringer <joestringer at nicira.com>
Acked-by: Ben Pfaff <blp at nicira.com>
---
v6-v7: No change.
v4: Increase N_UMAPS to 512.
v3: Rebase.
v2: #define N_UMAPS.
    Simplify upcall_unixctl_show() element counting.
    Mention that 'ukeys' contains elements of type 'struct udpif_key'.
RFC: First post.


  Commit: aa0e3ad5d409e9a832eedf3e2f8bbd5f486d427b
      https://github.com/openvswitch/ovs/commit/aa0e3ad5d409e9a832eedf3e2f8bbd5f486d427b
  Author: Joe Stringer <joestringer at nicira.com>
  Date:   2014-10-06 (Mon, 06 Oct 2014)

  Changed paths:
    M ofproto/ofproto-dpif-upcall.c

  Log Message:
  -----------
  upcall: Rename dump_op -> ukey_op.

Future patches will make use of the 'struct dump_op' in a broader sense,
so this patch renames it to make things a bit clearer.

Signed-off-by: Joe Stringer <joestringer at nicira.com>
Acked-by: Ben Pfaff <blp at nicira.com>
---
v7: Rebase.
v4-v6: No change.
v3: Rebase.
v2: No change.
RFC: First post.


  Commit: 92d47a49b2f5554c0536b35e3cd46326eae48895
      https://github.com/openvswitch/ovs/commit/92d47a49b2f5554c0536b35e3cd46326eae48895
  Author: Joe Stringer <joestringer at nicira.com>
  Date:   2014-10-06 (Mon, 06 Oct 2014)

  Changed paths:
    M ofproto/ofproto-dpif-upcall.c

  Log Message:
  -----------
  upcall: Create ukeys in handler threads.

Currently, when a revalidator thread first dumps a flow, it creates a
'udpif_key' object and caches a copy of a kernel flow key. This allows
us to perform lookups in the classifier to attribute stats and validate
the correctness of the datapath flow.

This patch sets up this cache from the handler threads, during flow
setup. While this patch alone causes a decrease in revalidation
performance, it allows future patches increase performance by reducing
the cost of flow dumping.

Revalidators will continue to create ukeys if a flow is dumped that has
no corresponding ukey. This may happen in corner cases such as when
ovs-vswitchd is restarted (and flows remain in the datapath) or a user
installs a flow using ovs-dpctl.

Signed-off-by: Joe Stringer <joestringer at nicira.com>
Acked-by: Ben Pfaff <blp at nicira.com>
---
v7: Create ukeys in revalidator threads in corner cases.
v6: Always call poll_block() in udpif_upcall_handler().
    Reduce memory zeroing in upcall/ukey init code.
    Fix up extraneous netflow_unref().
    Don't fetch 'recirc' in paths that always support recirculation.
    Log and don't delete unrecognized flows.
    ukey_install() style fixes.
    Add comment for dpif-netdev ukey_new() path.
v2-v5: Rebase.
RFC: First post.


  Commit: 53418be29095baa6ed01fd2374c008cce678c172
      https://github.com/openvswitch/ovs/commit/53418be29095baa6ed01fd2374c008cce678c172
  Author: Joe Stringer <joestringer at nicira.com>
  Date:   2014-10-06 (Mon, 06 Oct 2014)

  Changed paths:
    M ofproto/ofproto-dpif-upcall.c

  Log Message:
  -----------
  upcall: Revalidate using cache of mask, actions.

This allows us to ignore most fields of a flow_dump, requiring only the
flow key for looking up the ukey. Fetching flows can also be avoided in
the corner case where a flow is missed from a dump but revalidation is
required.

A future patch will modify the datapath interface to allow datapaths to
skip dumping these fields, so this cache will be used instead.

Signed-off-by: Joe Stringer <joestringer at nicira.com>
Acked-by: Ben Pfaff <blp at nicira.com>
---
v7: Rebase against ukey_create() changes in previous patch.
v6: Use atomic_read_relaxed() in ukey_create().
    Simplify delete_op_init().
v5: Switch back to checking that the mask is more specific.
v4: No change.
v3: Rebase.
v2: Rebase.
RFC: First post.


  Commit: 333c38e31361113079f72d4fbc264cadffe902b8
      https://github.com/openvswitch/ovs/commit/333c38e31361113079f72d4fbc264cadffe902b8
  Author: Joe Stringer <joestringer at nicira.com>
  Date:   2014-10-06 (Mon, 06 Oct 2014)

  Changed paths:
    M include/openvswitch/types.h
    M lib/hash.c
    M lib/hash.h
    M tests/test-hash.c

  Log Message:
  -----------
  hash: Add 128-bit murmurhash.

Add the 128-bit murmurhash by Austin Appleby, r150 from:
http://code.google.com/p/smhasher/source/browse/trunk/MurmurHash3.cpp

Signed-off-by: Joe Stringer <joestringer at nicira.com>
Acked-by: Ben Pfaff <blp at nicira.com>
---
v7: Replace getblock*() with get_unaligned_u*().
    Add ovs_u128_equal() function rather than using memcmp.
    Only build the version with word length matching host architecture.
v6: Add version that uses 64-bit operations.
    Add tests for x86 and x64 versions of mhash128.
    More style fixes.
v5: No change.
v4: Minor comments and style fixes.
v3: Rebase.


  Commit: 6fd74994538abffeec0b9f070de846d387bb037f
      https://github.com/openvswitch/ovs/commit/6fd74994538abffeec0b9f070de846d387bb037f
  Author: Joe Stringer <joestringer at nicira.com>
  Date:   2014-10-06 (Mon, 06 Oct 2014)

  Changed paths:
    M lib/dpif-netdev.c
    M lib/dpif-netlink.c
    M lib/dpif.c
    M lib/dpif.h
    M ofproto/ofproto-dpif-upcall.c

  Log Message:
  -----------
  dpif: Generate flow_hash for revalidators in dpif.

This patch shifts the responsibility for determining the hash for a flow
from the revalidation logic down to the dpif layer. This assists in
handling backward-compatibility for revalidation with the upcoming
unique identifier "UID" patches, and allows ovs-dpctl to automatically
determine flow hashes while keeping dpif logic separate from the
ofproto-dpif logic.

A 128-bit UID was selected to minimize the likelihood of hash conflicts.
Handler threads will not install a flow that has an identical UID as
another flow, to prevent misattribution of stats and to ensure that the
correct flow key cache is used for revalidation.

The UID for a flow is a 128-bit hash of the flow key. For datapaths that
do not support UID, which is currently all datapaths, the dpif will
generate the UID and pass it up during upcall and flow_dump. This is
generated based on the datapath flow key.

Later patches will add support for datapaths to store and interpret this
UID, in which case the dpif has a responsibility to pass it through
transparently.

Signed-off-by: Joe Stringer <joestringer at nicira.com>
Acked-by: Ben Pfaff <blp at nicira.com>
---
v7: Use ovs_u128_equal() for UID comparison.
v6: Split out from "dpif: Add Unique flow identifiers."
    Store flow hash seed staticly for all dpifs.
    Rebase.
v5: Always pass flow_key down to dpif, to improve error logging.
    Rebase.
v4: Generate UID in dpif layer and pass up to ofproto-dpif-upcall.
    Skip sending flow key in flow_del.
    Combine dpif,upcall,dpif-netdev,dpif-linux changes into one patch.
v3: Rebase.


  Commit: ea5fd7958e9feb29380f74bf99d9e641608d8b27
      https://github.com/openvswitch/ovs/commit/ea5fd7958e9feb29380f74bf99d9e641608d8b27
  Author: Joe Stringer <joestringer at nicira.com>
  Date:   2014-10-06 (Mon, 06 Oct 2014)

  Changed paths:
    M datapath/datapath.c
    M datapath/flow.h
    M datapath/flow_netlink.c
    M datapath/flow_netlink.h
    M datapath/flow_table.c
    M datapath/flow_table.h
    M datapath/linux/compat/include/linux/openvswitch.h

  Log Message:
  -----------
  datapath: Add support for unique flow identifiers.

If a datapath is created with the flag OVS_DP_F_INDEX_BY_UID, then an
additional table_instance is added to the flow_table, which is indexed
by unique identifiers ("UID"). Userspace implementations can specify a
UID of up to 128 bits along with a flow operation as shorthand for the
key. This allows revalidation performance improvements of up to 50%.

If a datapath is created using OVS_DP_F_INDEX_BY_UID and a UID is not
specified at flow setup time, then that operation will fail. If
OVS_UID_F_* flags are specified for an operation, then they will modify
what is returned through the operation. For instance, OVS_UID_F_SKIP_KEY
allows the datapath to skip returning the key (eg, during dump to reduce
memory copy).

Signed-off-by: Joe Stringer <joestringer at nicira.com>
---
v7: Remove OVS_DP_F_INDEX_BY_UID.
    Rework UID serialisation for variable-length UID.
    Log error if uid not specified and OVS_UID_F_SKIP_KEY is set.
    Rebase against "probe" logging changes.
v6: Fix documentation for supporting UIDs between 32-128 bits.
    Minor style fixes.
    Rebase.
v5: No change.
v4: Fix memory leaks.
    Log when triggering the older userspace issue above.
v3: Initial post.

squash! datapath: Add support for unique flow identifiers.


  Commit: 51b8e47ba2b4cf8a333f052d2f4e67100994bff0
      https://github.com/openvswitch/ovs/commit/51b8e47ba2b4cf8a333f052d2f4e67100994bff0
  Author: Joe Stringer <joestringer at nicira.com>
  Date:   2014-10-06 (Mon, 06 Oct 2014)

  Changed paths:
    M lib/dpctl.c
    M lib/dpif-netdev.c
    M lib/dpif-netlink.c
    M lib/dpif.c
    M lib/dpif.h
    M lib/odp-util.c
    M lib/odp-util.h
    M ofproto/ofproto-dpif-upcall.c
    M ofproto/ofproto-dpif.c
    M tests/dpif-netdev.at
    M tests/ofproto-dpif.at
    M tests/ofproto-macros.at

  Log Message:
  -----------
  dpif: Index flows using unique identifiers.

This patch modifies the dpif interface to allow flows to be manipulated
using a 128-bit identifier. This allows revalidator threads to perform
datapath operations faster, as they do not need to serialise the entire
flow key for operations like flow_get and flow_delete. In conjunction
with a future patch to simplify the dump interface, this provides a
significant performance benefit for revalidation.

When handlers assemble flow_put operations, they specify a unique
identifier (UID) for each flow as it is passed down to the datapath to
be stored with the flow. The UID is currently provided to handlers
by the dpif during upcall processing.

When revalidators assemble flow_get or flow_del operations, they specify
the UID for the flow along with the key, and a boolean for whether to
send the request using only a UID or to send the request using the UID
and flow key. The former is preferred for newer datapaths that support
UID, while the latter is used for backwards compatibility.

Signed-off-by: Joe Stringer <joestringer at nicira.com>
---
v7: Shift UID below the dpif layer
    Remove OVS_DP_F_INDEX_BY_UID
    Use ovs_u128_equal().
    Don't use 'static bool' for dpif_netlink_init_uid().
    Rebase.
v6: Split out from "dpif: Add Unique flow identifiers."
    Make sure that actions are fetched if terse=false.
    Add additional dpif.h documentation to describe the interface.
    Rebase.
v5: Always pass flow_key down to dpif, to improve error logging.
    Fix extraneous netflow_unref.
    Fix testsuite failure.
    Rebase.
v4: Skip sending flow key in flow_del.
    Fix race conditions with tests.
    Combine dpif,upcall,dpif-netdev,dpif-linux changes into one patch.
    Log UID in flow_del/flow_get/flow_dump messages.
v3: Rebase.


  Commit: c10ba5e8d11466a56b6557d966b09cc2761e871e
      https://github.com/openvswitch/ovs/commit/c10ba5e8d11466a56b6557d966b09cc2761e871e
  Author: Joe Stringer <joestringer at nicira.com>
  Date:   2014-10-06 (Mon, 06 Oct 2014)

  Changed paths:
    M lib/dpctl.c
    M lib/dpif-netdev.c
    M lib/dpif-netlink.c
    M lib/dpif-provider.h
    M lib/dpif.c
    M lib/dpif.h
    M ofproto/ofproto-dpif-upcall.c
    M ofproto/ofproto-dpif.c
    M tests/dpif-netdev.at
    M tests/ofproto-dpif.at

  Log Message:
  -----------
  dpif: Minimize memory copy for revalidation.

One of the limiting factors on the number of flows that can be supported
in the datapath is the overhead of assembling flow dump messages in the
datapath. This patch modifies the flow_dump interface to allow
revalidators to skip dumping the key, mask and actions from the
datapath, by making use of the unique identifiers introduced in earlier
patches.

For each flow dump, the dpif user specifies whether to skip these
attributes, allowing the common case to only dump a pair of 128-bit ID
and flow stats. This increases the number of flows that a revalidator
can handle per second by 50% or more.

Signed-off-by: Joe Stringer <joestringer at nicira.com>
---
v7: Shift UID below the dpif layer.
    Shift UID support detection to earlier patch.
    Handle ukey_creation when key is missing from flow dump.
    Use ovs_u128_equal().
v6: Split out from "dpif: Add Unique flow identifiers."
    Rebase.
v5: Always pass flow_key down to dpif, to improve error logging.
    Fix extraneous netflow_unref.
    Rebase.
v4: Fix bug where UID-based terse dump wasn't enabled by default.
    Fix race conditions with tests.
    Combine dpif,upcall,dpif-netdev,dpif-linux changes into one patch.
    Display whether terse dump is enabled in ovs-appctl upcall/show.
v3: Rebase.


  Commit: f471b0ad29f0f7aa0cf0bd6cb173ec8397819ab2
      https://github.com/openvswitch/ovs/commit/f471b0ad29f0f7aa0cf0bd6cb173ec8397819ab2
  Author: Alex Wang <alexw at nicira.com>
  Date:   2014-10-13 (Mon, 13 Oct 2014)

  Changed paths:
    M lib/dpif-netdev.c

  Log Message:
  -----------
  dpif-netdev: Add function to get pmd using core id.

This commit adds the function dp_netdev_get_pmd() which allows
users to get 'struct dp_netdev_pmd_thread' based on the core id.

Signed-off-by: Alex Wang <alexw at nicira.com>


  Commit: 019c815d998b8d480c8df8da04a1fd73f4a2d4a1
      https://github.com/openvswitch/ovs/commit/019c815d998b8d480c8df8da04a1fd73f4a2d4a1
  Author: Alex Wang <alexw at nicira.com>
  Date:   2014-10-21 (Tue, 21 Oct 2014)

  Changed paths:
    M lib/dpif-netdev.c
    M lib/dpif.c
    M lib/dpif.h
    M ofproto/ofproto-dpif-upcall.c

  Log Message:
  -----------
  Per_pmd_cls:

TODO:
1. How to extend 128-hash to contain the poller-id.
[solution] do not need to care.  'struct flow' must be unique for
     each pmd thread.

2. Cherry-pick flow dump logic.
[progress] done.

3. Take care of dpctl flow add/del.
[idea]     get the numa id from in-port => apply operation to all
     pmds.

4. Flush revalidator when reconfigure pmd.
[progress] not started yet.


Compare: https://github.com/openvswitch/ovs/compare/d09543d61c86^...019c815d998b


More information about the git mailing list