[ovs-dev] [PATCH] dpif-netdev: Report overhead busy cycles per pmd.

[Aaron Conole]

Hi David,

David Marchand <david.marchand at redhat.com> writes:

> When setting PMD auto load balance parameters and observing the
> feature (and the per rxq statistics) in action, you can easily get
> rebalances while the sum of per rxq pmd usage is below the pmd load
> threshold you had set.
>
> This is because the dpif-netdev/pmd-rxq-show command only reports "pure"
> rxq cycles while some cycles are used in the pmd mainloop and adds up to
> the total pmd load.
>
> dpif-netdev/pmd-stats-show does report per pmd load usage.
> This load is measured since the last dpif-netdev/pmd-stats-clear call.
> On the other hand, the auto load balance feature uses the pmd load on a 10s
> sliding window which makes it non trivial to correlate.
>
> Gather per pmd busy cycles with the same periodicity and report the
> difference as overhead in dpif-netdev/pmd-rxq-show so that we have all
> info in a single command.
>
> Example:
> $ ovs-appctl dpif-netdev/pmd-rxq-show
> pmd thread numa_id 0 core_id 4:
>   isolated : false
>   port: dpdk0             queue-id:  0 (enabled)   pmd usage: 37 %
>   port: dpdk1             queue-id:  0 (enabled)   pmd usage: 36 %
>   port: vhost3            queue-id:  0 (enabled)   pmd usage:  0 %
>   port: vhost6            queue-id:  0 (enabled)   pmd usage:  0 %
>   port: vhost7            queue-id:  0 (enabled)   pmd usage:  0 %
>   overhead:  4 %
> pmd thread numa_id 0 core_id 18:
>   isolated : false
>   port: vhost0            queue-id:  0 (enabled)   pmd usage: 37 %
>   port: vhost1            queue-id:  0 (enabled)   pmd usage: 39 %
>   port: vhost2            queue-id:  0 (enabled)   pmd usage:  0 %
>   port: vhost4            queue-id:  0 (enabled)   pmd usage:  0 %
>   port: vhost5            queue-id:  0 (enabled)   pmd usage:  0 %
>   overhead:  5 %
>
> Signed-off-by: David Marchand <david.marchand at redhat.com>
> ---

... snip ...

> diff --git a/tests/pmd.at b/tests/pmd.at
> index cc5371d5a5..256adb83f0 100644
> --- a/tests/pmd.at
> +++ b/tests/pmd.at
> @@ -72,6 +72,7 @@ CHECK_PMD_THREADS_CREATED()
>  AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | sed SED_NUMA_CORE_PATTERN], [0], [dnl
>  pmd thread numa_id <cleared> core_id <cleared>:
>    isolated : false
> +  usage: NOT AVAIL

I don't see this added anywhere, but I do see "overhead:" added.  I
guess this is leftover from an earlier version.  It causes test case failures.

>    port: p0                queue-id:  0 (enabled)   pmd usage: NOT AVAIL
>  ])
>  
> @@ -103,6 +104,7 @@ dummy at ovs-dummy: hit:0 missed:0
>  AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | sed SED_NUMA_CORE_PATTERN], [0], [dnl
>  pmd thread numa_id <cleared> core_id <cleared>:
>    isolated : false
> +  usage: NOT AVAIL
>    port: p0                queue-id:  0 (enabled)   pmd usage: NOT AVAIL
>    port: p0                queue-id:  1 (enabled)   pmd usage: NOT AVAIL
>    port: p0                queue-id:  2 (enabled)   pmd usage: NOT AVAIL
> @@ -134,6 +136,7 @@ dummy at ovs-dummy: hit:0 missed:0
>  AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | sed SED_NUMA_CORE_PATTERN], [0], [dnl
>  pmd thread numa_id <cleared> core_id <cleared>:
>    isolated : false
> +  usage: NOT AVAIL
>    port: p0                queue-id:  0 (enabled)   pmd usage: NOT AVAIL
>    port: p0                queue-id:  1 (enabled)   pmd usage: NOT AVAIL
>    port: p0                queue-id:  2 (enabled)   pmd usage: NOT AVAIL
> @@ -149,13 +152,13 @@ TMP=$(cat ovs-vswitchd.log | wc -l | tr -d [[:blank:]])
>  AT_CHECK([ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x3])
>  CHECK_PMD_THREADS_CREATED([2], [], [+$TMP])
>  
> -AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | awk '/AVAIL$/ { printf("%s\t", $0); next } 1' | parse_pmd_rxq_show_group | sort], [0], [dnl
> +AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | awk '/\<port:.*AVAIL$/ { printf("%s\t", $0); next } 1' | parse_pmd_rxq_show_group | sort], [0], [dnl
>  port: p0 queue-id: 0 3 4 7
>  port: p0 queue-id: 1 2 5 6
>  ])
>  
>  AT_CHECK([ovs-vsctl set Open_vSwitch . other_config:pmd-rxq-assign=roundrobin])
> -AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | awk '/AVAIL$/ { printf("%s\t", $0); next } 1' | parse_pmd_rxq_show_group | sort], [0], [dnl
> +AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | awk '/\<port:.*AVAIL$/ { printf("%s\t", $0); next } 1' | parse_pmd_rxq_show_group | sort], [0], [dnl
>  port: p0 queue-id: 0 2 4 6
>  port: p0 queue-id: 1 3 5 7
>  ])
> @@ -167,6 +170,7 @@ CHECK_PMD_THREADS_CREATED([1], [], [+$TMP])
>  AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | sed SED_NUMA_CORE_PATTERN], [0], [dnl
>  pmd thread numa_id <cleared> core_id <cleared>:
>    isolated : false
> +  usage: NOT AVAIL
>    port: p0                queue-id:  0 (enabled)   pmd usage: NOT AVAIL
>    port: p0                queue-id:  1 (enabled)   pmd usage: NOT AVAIL
>    port: p0                queue-id:  2 (enabled)   pmd usage: NOT AVAIL