[ovs-dev] [PATCH v3 00/16] Userspace (DPDK) connection tracker

[Daniele Di Proietto]

This series aims to implement the ct() action for the dpif-netdev datapath.
The bulk of the code is in the new conntrack module: it contains some packet
parsing code, some lookup tables and the logic to implements all the ct bits.

The conntrack module is helped by conntrack-tcp, for TCP window and flags
tracking: the bulk of the code of this submodule is from the FreeBSD's pf
subsystem, therefore is BSD licensed.

The rest of the series integrates the connection tracker with the rest of
OVS: the ct() action is implemented in dpif-netdev, and the debugging
interfaces required by dpctl/{dump,flush}-conntrack are implemented.

Besides adding some unit tests, this series ports the existing conntrack
system test to the userspace datapath.  Some small modifications are
required to pass the testsuite, and some tests still have to be skipped.

This can also be downloaded at:

https://github.com/ddiproietto/ovs/tree/userconntrack_20160516

Any feedback is appreciated, thanks.

v2 -> v3:
* Rebased.
* Squashed commits for flushing (in dpif-netdev and conntrack).
* Squashed commits for dumping (in dpif-netdev and conntrack).
* Use adaptive mutex instead of spinlock: this prevents livelock
  if the cleanup thread is executed on the same CPU as a forwarding
  thread.  Performance impact in minimal.
* Validate L3 and L4 checksum.
* Use proper L3 and L4 checksum in hardcoded packets in system and unit
  tests.
* Consider ICMPv6 as well as ICMP in l4_protos and conn_key_to_tuple.
* Mention conntrack in NEWS and FAQ.md.
* Use uint16_t for ct_state.
* Fix possible NULL dereference for conn in process_one().
* Add OVS_U128_MIN, OVS_U128_ZERO.
* Use HMAP_FOR_EACH_POP.
* Check that UDP length is valid.
* Style fix: prefer 'sizeof *object' instead of 'sizeof type'
* Don't accept packets from/to UDP/TCP port 0.
* Use defines for timeouts.
* Check expiration inside lookup loop in conn_key_lookup().
* Limit the number of connections.
* Simplify case if tcp_get_wscale().
* Introduce general INT_MOD_* macros for comparisons in modular arithmetic.
* Improve comments.
* New cleanup mechanism: we keep connections in an ordered list and we have
  a separate thread to performs the cleanup.  This doesn't block the main
  thread for long intervals anymore.
* Correctly fill UDP length and UDP/TCP/ICMP checksums in flow_compose():
  it's useful to write testcases for the connection tracker.
* Added system test with ICMP traffic through the connection tracker.
* Track ICMP type and code.

v1 -> v2:
* Fixed bug in tcp_get_wscale(), related to TCP options parsing.
* Changed names of ICMP constants: now they're different from Linux and
  FreeBSD.
* Fixed bug in parse_ipv6_ext_hdrs().
* Used ALWAYS_INLINE in parse_vlan and parse_ethertype, to avoid a
  performance regression in miniflow_extract().
* Updated copyright info in COPYING and debian/copyright.in.
* Rebased.
* Changed batching strategy in conntrack_execute() to allow a newly
  created connection to be picked up by packets in the same batch.
* Added an ovs-test module to throw pcap files at the connection tracker.
* Added a workaround for the userspace testsuite on new kernels and a tcp
  non-conntrack test.


Daniele Di Proietto (16):
  packets: Define ICMP types.
  flow: Export parse_ipv6_ext_hdrs().
  flow: Introduce parse_dl_type().
  conntrack: New userspace connection tracker.
  conntrack: Periodically delete expired connections.
  tests: Add very simple conntrack benchmark.
  tests: Add test-conntrack pcap test.
  dpif-netdev: Execute conntrack action.
  dpif-netdev: Implement conntrack dump functions.
  dpif-netdev: Implement conntrack flush interface.
  flow: Fill udp_len in flow_compose_l4().
  flow: Generate checksum in flow_compose().
  tests: Add conntrack ofproto-dpif tests.
  system-tests: Run conntrack tests with userspace.
  system-tests: Add ping through conntrack test.
  conntrack: Track ICMP type and code.

 COPYING                          |    1 +
 FAQ.md                           |    2 +-
 NEWS                             |    2 +
 debian/copyright.in              |    4 +
 include/openvswitch/types.h      |    4 +
 lib/automake.mk                  |    6 +
 lib/conntrack-icmp.c             |  105 ++++
 lib/conntrack-other.c            |   86 +++
 lib/conntrack-private.h          |  113 ++++
 lib/conntrack-tcp.c              |  499 +++++++++++++++
 lib/conntrack.c                  | 1237 ++++++++++++++++++++++++++++++++++++++
 lib/conntrack.h                  |  201 +++++++
 lib/ct-dpif.c                    |   24 +-
 lib/ct-dpif.h                    |    3 +-
 lib/dpif-netdev.c                |  133 +++-
 lib/flow.c                       |  201 ++++---
 lib/flow.h                       |    4 +
 lib/netlink-conntrack.c          |    2 +-
 lib/packets.h                    |   14 +-
 lib/util.h                       |    9 +
 tests/automake.mk                |    1 +
 tests/dpif-netdev.at             |   14 +-
 tests/ofproto-dpif.at            |  896 +++++++++++++++++++++++----
 tests/system-kmod-macros.at      |   28 +
 tests/system-traffic.at          |  149 ++++-
 tests/system-userspace-macros.at |   45 +-
 tests/test-conntrack.c           |  236 ++++++++
 27 files changed, 3780 insertions(+), 239 deletions(-)
 create mode 100644 lib/conntrack-icmp.c
 create mode 100644 lib/conntrack-other.c
 create mode 100644 lib/conntrack-private.h
 create mode 100644 lib/conntrack-tcp.c
 create mode 100644 lib/conntrack.c
 create mode 100644 lib/conntrack.h
 create mode 100644 tests/test-conntrack.c

-- 
2.1.4