[ovs-dev] [PATCH 0/6 V2] netdev-dpdk: Use intermediate queue during packet transmission.

[Bhanuprakash Bodireddy]

After packet classification, packets are queued in to batches depending
on the matching netdev flow. Thereafter each batch is processed to
execute the related actions. This becomes particularly inefficient if
there are few packets in each batch as rte_eth_tx_burst() incurs expensive
MMIO writes.

This patch series implements intermediate queue for DPDK and vHost User ports.
Packets are queued and burst when the packet count exceeds threshold. Also
drain logic is implemented to handle cases where packets can get stuck in
the tx queues at low rate traffic conditions. Care has been taken to see
that latency is well with in the acceptable limits. Testing shows significant
performance gains with this implementation.

This path series combines the earlier 2 patches posted below.
  DPDK patch: https://mail.openvswitch.org/pipermail/ovs-dev/2017-April/331039.html
  vHost User patch: https://mail.openvswitch.org/pipermail/ovs-dev/2017-May/332271.html

Also this series proposes to disable the retries on vHost User ports and make it 
configurable via ovsdb.(controversial?)

Performance Numbers with intermediate queue:

                      DPDK ports
                     ===========

  Throughput for P2P scenario, for two 82599ES 10G port with 64 byte packets

  Number
  flows       MASTER     With PATCH
  ======    =========    =========
    10       10727283    13393844
    32        7042253    11228799
    50        7515491     9607791
   100        5838699     9430730
   500        5285066     7845807
  1000        5226477     7135601

   Latency test

   MASTER
   =======
   Pkt size  min(ns)  avg(ns)  max(ns)
    512      4,631      5,022    309,914
   1024      5,545      5,749    104,294
   1280      5,978      6,159     45,306
   1518      6,419      6,774    946,850

   PATCH
   =====
   Pkt size  min(ns)  avg(ns)  max(ns)
    512      4,711      5,064    182,477
   1024      5,601      5,888    701,654
   1280      6,018      6,491    533,037
   1518      6,467      6,734    312,471

               vHost User ports
              ==================

  Throughput for PV scenario, with 64 byte packets

   Number
   flows       MASTER    With PATCH
  ========  =========   =========
    10        5945899     7833914
    32        3872211     6530133
    50        3283713     6618711
   100        3132540     5857226
   500        2964499     5273006
  1000        2931952     5178038

  Latency test.

  MASTER
  =======
  Pkt size  min(ns)  avg(ns)  max(ns)
   512      10,011   12,100   281,915
  1024       7,870    9,313   193,116
  1280       7,862    9,036   194,439
  1518       8,215    9,417   204,782

  PATCH
  =======
  Pkt size  min(ns)  avg(ns)  max(ns)
   512      10,492   13,655   281,538
  1024       8,407    9,784   205,095
  1280       8,399    9,750   194,888
  1518       8,367    9,722   196,973

Performance number reported by Eelco Chaudron <eechaudro at redhat.com> at
  https://mail.openvswitch.org/pipermail/ovs-dev/2017-June/333949.html
  https://mail.openvswitch.org/pipermail/ovs-dev/2017-May/332271.html
  https://mail.openvswitch.org/pipermail/ovs-dev/2017-April/331039.html

-----------------------------------
v1->v2
  * Rebased on master due to HW offload changes.
  * Introduced union for packet count and buffers and changed the variable
    names appropriately.
  * No functional change changes.

Bhanuprakash Bodireddy (6):
  netdev: Add netdev_txq_flush function.
  netdev-dpdk: Add netdev_dpdk_txq_flush function.
  netdev-dpdk: Add intermediate queue support.
  dpif-netdev: Flush the packets in intermediate queue.
  netdev-dpdk: Add netdev_dpdk_vhost_txq_flush function.
  netdev-dpdk: Enable intermediate queue for vHost User port.

 lib/dpif-netdev.c     |  49 ++++++++++++-
 lib/netdev-bsd.c      |   1 +
 lib/netdev-dpdk.c     | 194 ++++++++++++++++++++++++++++++++++++++++++--------
 lib/netdev-dummy.c    |   1 +
 lib/netdev-linux.c    |   1 +
 lib/netdev-provider.h |   8 +++
 lib/netdev-vport.c    |   2 +-
 lib/netdev.c          |   9 +++
 lib/netdev.h          |   1 +
 9 files changed, 235 insertions(+), 31 deletions(-)

-- 
2.4.11