[ovs-discuss] ovs-dpdk: How to achieve predictable throughput?

[Mattias Johansson G]

Hi,

I've been doing throughput measurements of OVS DPDK version 2.4 using two PMD
threads for the datapath and I'm facing a behavior that I would like to discuss. 

Two blades are involved in the test setup. One blade is the traffic generator and the other
one has the OVS and the VM to be tested. Traffic is sent from  the traffic generator to the
VM via OVS which echoes the traffic back to the traffic generator using the same interface
as the traffic arrived on. OVS is configured to have two PMD threads located on the same
numa node. In total OVS serves two ports; one physical port (dpdk0) and one vNIC
(vhostuser-x).

With this configuration one PMD thread does Rx from the physical port and Tx to the VM.
The other PMD thread does Rx from the VM and Tx to the physical port. This works fine.

What is the expected behavior (and can I control it) when I start another VM with
one vNIC?

Sometimes one PMD thread will do Rx and Tx of both ports included in my throughput test
while the other PMD thread does Rx for the newly added vNIC. So the impact of starting
the second VM decreases the throughput a lot.

Sometimes when I start the second VM with one vNIC I get the two ports included in the
throughput test on different PMD threads and then there is no impact on the throughput.

Is there a way to control this behavior? Or having the ports distributed over the PMD
threads based on the amount of traffic they are having? This implies redistribution
of ports with a certain interval depending on the variation in the amount of traffic being
sent on each the port.

I've looked in netdev-dpdk.c and if I interpret it correct this happens when a port is
added:
- In OVS 2.4 it's the order of the ports in a hash table that will decide which PMD thread
  that does Rx for a port.
- In OVS latest/master it's the PMD thread with the smallest amount of queues/ports
  that will be chosen when a new port is added (not the one with least amount of traffic).

BR,
Mattias