[ovs-git] [openvswitch/ovs] 324c83: dpif-netdev: XPS (Transmit Packet Steering) implem...

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/openvswitch/ovs
  Commit: 324c8374852a87b514db251051ec06ddcfff3e95
      https://github.com/openvswitch/ovs/commit/324c8374852a87b514db251051ec06ddcfff3e95
  Author: Ilya Maximets <i.maximets at samsung.com>
  Date:   2016-07-27 (Wed, 27 Jul 2016)

  Changed paths:
    M lib/dpif-netdev.c
    M lib/netdev-bsd.c
    M lib/netdev-dpdk.c
    M lib/netdev-dummy.c
    M lib/netdev-linux.c
    M lib/netdev-provider.h
    M lib/netdev.c
    M lib/netdev.h

  Log Message:
  -----------
  dpif-netdev: XPS (Transmit Packet Steering) implementation.

If CPU number in pmd-cpu-mask is not divisible by the number of queues and
in a few more complex situations there may be unfair distribution of TX
queue-ids between PMD threads.

For example, if we have 2 ports with 4 queues and 6 CPUs in pmd-cpu-mask
such distribution is possible:
<------------------------------------------------------------------------>
pmd thread numa_id 0 core_id 13:
  port: vhost-user1       queue-id: 1
  port: dpdk0     queue-id: 3
pmd thread numa_id 0 core_id 14:
  port: vhost-user1       queue-id: 2
pmd thread numa_id 0 core_id 16:
  port: dpdk0     queue-id: 0
pmd thread numa_id 0 core_id 17:
  port: dpdk0     queue-id: 1
pmd thread numa_id 0 core_id 12:
  port: vhost-user1       queue-id: 0
  port: dpdk0     queue-id: 2
pmd thread numa_id 0 core_id 15:
  port: vhost-user1       queue-id: 3
<------------------------------------------------------------------------>

As we can see above dpdk0 port polled by threads on cores:
	12, 13, 16 and 17.

By design of dpif-netdev, there is only one TX queue-id assigned to each
pmd thread. This queue-id's are sequential similar to core-id's. And
thread will send packets to queue with exact this queue-id regardless
of port.

In previous example:

	pmd thread on core 12 will send packets to tx queue 0
	pmd thread on core 13 will send packets to tx queue 1
	...
	pmd thread on core 17 will send packets to tx queue 5

So, for dpdk0 port after truncating in netdev-dpdk:

	core 12 --> TX queue-id 0 % 4 == 0
	core 13 --> TX queue-id 1 % 4 == 1
	core 16 --> TX queue-id 4 % 4 == 0
	core 17 --> TX queue-id 5 % 4 == 1

As a result only 2 of 4 queues used.

To fix this issue some kind of XPS implemented in following way:

	* TX queue-ids are allocated dynamically.
	* When PMD thread first time tries to send packets to new port
	  it allocates less used TX queue for this port.
	* PMD threads periodically performes revalidation of
	  allocated TX queue-ids. If queue wasn't used in last
	  XPS_TIMEOUT_MS milliseconds it will be freed while revalidation.
  * XPS is not working if we have enough TX queues.

Reported-by: Zhihong Wang <zhihong.wang at intel.com>
Signed-off-by: Ilya Maximets <i.maximets at samsung.com>
Signed-off-by: Daniele Di Proietto <diproiettod at vmware.com>


  Commit: 91364d18defab4885bddc8f241b88671ffff5217
      https://github.com/openvswitch/ovs/commit/91364d18defab4885bddc8f241b88671ffff5217
  Author: Ilya Maximets <i.maximets at samsung.com>
  Date:   2016-07-27 (Wed, 27 Jul 2016)

  Changed paths:
    M lib/dpif-netdev.c
    M lib/dpif-netlink.c
    M lib/dpif-provider.h
    M lib/dpif.c
    M lib/dpif.h
    M ofproto/ofproto-dpif.c
    M ofproto/ofproto-provider.h
    M ofproto/ofproto.c
    M ofproto/ofproto.h
    M vswitchd/bridge.c

  Log Message:
  -----------
  bridge: Pass interface's configuration to datapath.

This commit adds functionality to pass value of 'other_config' column
of 'Interface' table to datapath.

This may be used to pass not directly connected with netdev options and
configure behaviour of the datapath for different ports.
For example: pinning of rx queues to polling threads in dpif-netdev.

Signed-off-by: Ilya Maximets <i.maximets at samsung.com>
Acked-by: Daniele Di Proietto <diproiettod at vmware.com>


  Commit: a6a426d69ac90e6b222fd620b1c8f988770af725
      https://github.com/openvswitch/ovs/commit/a6a426d69ac90e6b222fd620b1c8f988770af725
  Author: Ilya Maximets <i.maximets at samsung.com>
  Date:   2016-07-27 (Wed, 27 Jul 2016)

  Changed paths:
    M lib/dpif-netdev.c

  Log Message:
  -----------
  dpif-netdev: Add reconfiguration request to dp_netdev.

Next patches will add new conditions when reconfiguration will be
required. It'll be simpler to have common way to request reconfiguration.

Signed-off-by: Ilya Maximets <i.maximets at samsung.com>
Acked-by: Daniele Di Proietto <diproiettod at vmware.com>


  Commit: 3eb67853c481d0c9584cba351b4d7a67491ec5e8
      https://github.com/openvswitch/ovs/commit/3eb67853c481d0c9584cba351b4d7a67491ec5e8
  Author: Ilya Maximets <i.maximets at samsung.com>
  Date:   2016-07-27 (Wed, 27 Jul 2016)

  Changed paths:
    M INSTALL.DPDK.md
    M NEWS
    M lib/dpif-netdev.c
    M tests/pmd.at
    M vswitchd/vswitch.xml

  Log Message:
  -----------
  dpif-netdev: Introduce pmd-rxq-affinity.

New 'other_config:pmd-rxq-affinity' field for Interface table to
perform manual pinning of RX queues to desired cores.

This functionality is required to achieve maximum performance because
all kinds of ports have different cost of rx/tx operations and
only user can know about expected workload on different ports.

Example:
	# ./bin/ovs-vsctl set interface dpdk0 options:n_rxq=4 \
             other_config:pmd-rxq-affinity="0:3,1:7,3:8"
	Queue #0 pinned to core 3;
	Queue #1 pinned to core 7;
	Queue #2 not pinned.
	Queue #3 pinned to core 8;

It's decided to automatically isolate cores that have rxq explicitly
assigned to them because it's useful to keep constant polling rate on
some performance critical ports while adding/deleting other ports
without explicit pinning of all ports.

Signed-off-by: Ilya Maximets <i.maximets at samsung.com>
Signed-off-by: Daniele Di Proietto <diproiettod at vmware.com>


Compare: https://github.com/openvswitch/ovs/compare/7adc92ac590e...3eb67853c481