[ovs-dev] [dpdk patch 0/8] DKDP PMD multithreading.

Alex Wang alexw at nicira.com
Tue Aug 12 04:56:35 UTC 2014


This series of patches adds the multithreading functionality
to OVS dpdk module.  Users are able to create multiple pmd
threads and set their cpu affinity via specifying a cpu mask
string similar to the EAL '-c COREMASK' option.  Also, the
number of rx queues for each dpdk interface can be configured.

Due to the fact that other ongoing ovs dpdk development can
significantly affect the ovs dpdk performance.  Limited tests
were conducted with this series on top of master.  Here are
some results.

1. Spirent: 32 bi-directional udp flows (64 byte) between 2 dpdk
            ifaces on same cpu socket.


MASTER:
+-------------------+-------------+-------------+
|       stats       | port1 //6/1 | port2 //6/2 |
+-------------------+-------------+-------------+
|  Rx.SigFrameCount |  451719219  |  451716683  |
| Rx.MinFrameLength |      64     |      64     |
| Rx.MaxFrameLength |     198     |     198     |
+-------------------+-------------+-------------+
pktgen: INFO: TX results summary 
+--------------------+---------------+---------------+
|       stats        |     port1     |     port2     |
+--------------------+---------------+---------------+
|   Tx.ParentPort    |      917      |      1666     |
| Tx.TotalFrameCount |   1487311447  |   1487318647  |
|  Tx.TotalBitCount  |  761503460864 |  761507147264 |
|   Tx.L1BitCount    |  999473292384 |  999478130784 |
|   Tx.TxDuration    |     100.0     |     100.0     |
|   Tx.StreamBlock   |   32 udp 64   |  32 udp 64-2  |
|      Tx.Load       |      100      |      100      |
|   Tx.Saturation    | 99.9473292384 | 99.9478130784 |
|    Tx.DropRate     | 69.6286303355 | 69.6286303355 |
+--------------------+---------------+---------------+


3 PMD THREADS ON EACH CPU SOCKET, 6 RX QUEUES PER DPDK IFACE:
+-------------------+-------------+-------------+
|       stats       | port1 //6/1 | port2 //6/2 |
+-------------------+-------------+-------------+
|  Rx.SigFrameCount |  1136484925 |  1140549704 |
| Rx.MinFrameLength |      64     |      64     |
| Rx.MaxFrameLength |    65529    |     198     |
+-------------------+-------------+-------------+
pktgen: INFO: TX results summary 
+--------------------+---------------+---------------+
|       stats        |     port1     |     port2     |
+--------------------+---------------+---------------+
|   Tx.ParentPort    |      917      |      1666     |
| Tx.TotalFrameCount |   1488094078  |   1488094078  |
|  Tx.TotalBitCount  |  761904167936 |  761904167936 |
|   Tx.L1BitCount    |  999999220416 |  999999220416 |
|   Tx.TxDuration    |     100.0     |     100.0     |
|   Tx.StreamBlock   |   32 udp 64   |  32 udp 64-2  |
|      Tx.Load       |      100      |      100      |
|   Tx.Saturation    | 99.9999220416 | 99.9999220416 |
|    Tx.DropRate     | 23.4915768209 | 23.4915768209 |
+--------------------+---------------+---------------+




2. Spirent: 32 bi-directional udp flows (64 byte) between 2 dpdk
            ifaces on different cpu socket.


MASTER:
+-------------------+-------------+-------------+
|       stats       | port1 //6/1 | port2 //6/2 |
+-------------------+-------------+-------------+
|  Rx.SigFrameCount |  326922953  |  326923196  |
| Rx.MinFrameLength |      64     |      64     |
| Rx.MaxFrameLength |    16492    |     198     |
+-------------------+-------------+-------------+
pktgen: INFO: TX results summary 
+--------------------+---------------+---------------+
|       stats        |     port1     |     port2     |
+--------------------+---------------+---------------+
|   Tx.ParentPort    |      917      |      1666     |
| Tx.TotalFrameCount |   1488094078  |   1488094078  |
|  Tx.TotalBitCount  |  761904167936 |  761904167936 |
|   Tx.L1BitCount    |  999999220416 |  999999220416 |
|   Tx.TxDuration    |     100.0     |     100.0     |
|   Tx.StreamBlock   |   32 udp 64   |  32 udp 64-2  |
|      Tx.Load       |      100      |      100      |
|   Tx.Saturation    | 99.9999220416 | 99.9999220416 |
|    Tx.DropRate     | 78.0307522667 | 78.0307522667 |
+--------------------+---------------+---------------+


3 PMD THREADS ON EACH CPU SOCKET, 6 RX QUEUES PER DPDK IFACE:

+-------------------+-------------+-------------+
|       stats       | port1 //6/1 | port2 //6/2 |
+-------------------+-------------+-------------+
|  Rx.SigFrameCount |  1487802957 |  1487807151 |
| Rx.MinFrameLength |      12     |      64     |
| Rx.MaxFrameLength |     198     |     198     |
+-------------------+-------------+-------------+
pktgen: INFO: TX results summary 
+--------------------+-------------------+-------------------+
|       stats        |       port1       |       port2       |
+--------------------+-------------------+-------------------+
|   Tx.ParentPort    |        917        |        1666       |
| Tx.TotalFrameCount |     1487811649    |     1487807527    |
|  Tx.TotalBitCount  |    761759564288   |    761757453824   |
|   Tx.L1BitCount    |    999809428128   |    999806658144   |
|   Tx.TxDuration    |       100.0       |       100.0       |
|   Tx.StreamBlock   |     32 udp 64     |    32 udp 64-2    |
|      Tx.Load       |        100        |        100        |
|   Tx.Saturation    |   99.9809428128   |   99.9806658144   |
|    Tx.DropRate     | 0.000304743297568 | 0.000304743297568 |
+--------------------+-------------------+-------------------+






Alex Wang (8):
  ovs-numa: Add ovs-numa.{c,h} for extracting and storing cpu socket   
     and cpu core info.
  netdev-dpdk: Add function for getting the socket_id of netdev-dpdk.
  netdev-dpdk: Make memory pool name contain the socket id.
  netdev-dpdk: Create 'number of dpdk ifaces on same cpu socket' rx    
    queues and 'number of cpu cores' tx queues for each dpdk interface.
  dpif-netdev: Create 'number of dpdk ifaces on cpu socket'     pmd
    threads for each cpu socket.
  netdev-dpdk: Add function for configuring rx queues for all dpdk    
    interfaces.
  ovs-numa: Add support for cpu-mask configuration.
  vswitchd: Add option to configure cpu-mask and number of dpdk
    rx_queues.

 lib/automake.mk            |    2 +
 lib/dpif-linux.c           |    1 +
 lib/dpif-netdev.c          |  354 +++++++++++++++++++++++++++++++++++++-------
 lib/dpif-netdev.h          |    3 +-
 lib/dpif-provider.h        |    7 +
 lib/dpif.c                 |   17 +++
 lib/dpif.h                 |    1 +
 lib/netdev-dpdk.c          |  150 ++++++++++++++++---
 lib/netdev-dpdk.h          |   30 ++++
 lib/ovs-numa.c             |  351 +++++++++++++++++++++++++++++++++++++++++++
 lib/ovs-numa.h             |  118 +++++++++++++++
 ofproto/ofproto-dpif.c     |    2 +
 ofproto/ofproto-provider.h |    6 +
 ofproto/ofproto.c          |   16 ++
 ofproto/ofproto.h          |    2 +
 tests/ofproto-macros.at    |    1 +
 vswitchd/bridge.c          |    5 +
 vswitchd/vswitch.xml       |   29 ++++
 18 files changed, 1017 insertions(+), 78 deletions(-)
 create mode 100644 lib/ovs-numa.c
 create mode 100644 lib/ovs-numa.h

-- 
1.7.9.5




More information about the dev mailing list