[ovs-dev] [PATCH v6 0/7] Output packet batching.
Jan Scheurich
jan.scheurich at ericsson.com
Mon Dec 4 23:21:16 UTC 2017
Hi Ilya,
I have retested your "Output patches batching" v6 in our standard PVP L3-VPN/VXLAN benchmark setup [1]. The configuration is a single PMD serving a physical 10G port and a VM running DPDK testpmd as IP reflector with 4 equally loaded vhostuser ports. The tests are run with 64 byte packets. Below are Mpps values averaged over four 10 second runs:
master patch patch
Flows Mpps tx-flush-interval=0 tx-flush-interval=50
8 4.419 4.342 -1.7% 4.749 7.5%
100 4.026 3.956 -1.7% 4.281 6.3%
1000 3.630 3.632 0.1% 3.760 3.6%
2000 3.394 3.390 -0.1% 3.490 2.8%
5000 2.989 2.938 -1.7% 2.994 0.2%
10000 2.756 2.711 -1.6% 2.746 -0.4%
20000 2.641 2.598 -1.6% 2.622 -0.7%
50000 2.604 2.558 -1.8% 2.579 -1.0%
100000 2.598 2.552 -1.8% 2.572 -1.0%
500000 2.598 2.550 -1.8% 2.571 -1.0%
As expected output batching within rx bursts (tx-flush-interval=0) provides little or no benefit in this scenario. The test results reflect roughly a 1.7% performance penalty due to the tx batching overhead. This overhead is measurable, but should in my eyes not be a blocker for merging this patch series.
Interestingly, tests with time-based tx batching and a minimum flush interval of 50 microseconds show a consistent and significant performance increase for small number of flows (in the regime where EMC is effective) and a reduced penalty of 1% for many flows. I don't have a good explanation yet for this phenomenon. I would be interested to see if other benchmark results support the general positive impact of time-based tx batching on throughput also for synthetic DPDK applications in the VM. The average Ping RTT increases by 20-30 us as expected.
We will also retest the performance improvement of time-based tx batching on interrupt driven Linux kernel applications (such as iperf3).
BR, Jan
> -----Original Message-----
> From: Ilya Maximets [mailto:i.maximets at samsung.com]
> Sent: Friday, 01 December, 2017 16:44
> To: ovs-dev at openvswitch.org; Bhanuprakash Bodireddy <bhanuprakash.bodireddy at intel.com>
> Cc: Heetae Ahn <heetae82.ahn at samsung.com>; Antonio Fischetti <antonio.fischetti at intel.com>; Eelco Chaudron
> <echaudro at redhat.com>; Ciara Loftus <ciara.loftus at intel.com>; Kevin Traynor <ktraynor at redhat.com>; Jan Scheurich
> <jan.scheurich at ericsson.com>; Ian Stokes <ian.stokes at intel.com>; Ilya Maximets <i.maximets at samsung.com>
> Subject: [PATCH v6 0/7] Output packet batching.
>
> This patch-set inspired by [1] from Bhanuprakash Bodireddy.
> Implementation of [1] looks very complex and introduces many pitfalls [2]
> for later code modifications like possible packet stucks.
>
> This version targeted to make simple and flexible output packet batching on
> higher level without introducing and even simplifying netdev layer.
>
> Basic testing of 'PVP with OVS bonding on phy ports' scenario shows
> significant performance improvement.
>
> Test results for time-based batching for v3:
> https://mail.openvswitch.org/pipermail/ovs-dev/2017-September/338247.html
>
> Test results for v4:
> https://mail.openvswitch.org/pipermail/ovs-dev/2017-October/339624.html
>
> [1] [PATCH v4 0/5] netdev-dpdk: Use intermediate queue during packet transmission.
> https://mail.openvswitch.org/pipermail/ovs-dev/2017-August/337019.html
>
> [2] For example:
> https://mail.openvswitch.org/pipermail/ovs-dev/2017-August/337133.html
>
> Version 6:
> * Rebased on current master:
> - Added new patch to refactor dp_netdev_pmd_thread structure
> according to following suggestion:
> https://mail.openvswitch.org/pipermail/ovs-dev/2017-November/341230.html
>
> NOTE: I still prefer reverting of the padding related patch.
> Rebase done to not block acepting of this series.
> Revert patch and discussion here:
> https://mail.openvswitch.org/pipermail/ovs-dev/2017-November/341153.html
>
> * Added comment about pmd_thread_ctx_time_update() usage.
>
> Version 5:
> * pmd_thread_ctx_time_update() calls moved to different places to
> call them only from dp_netdev_process_rxq_port() and main
> polling functions:
> pmd_thread_main, dpif_netdev_run and dpif_netdev_execute.
> All other functions should use cached time from pmd->ctx.now.
> It's guaranteed to be updated at least once per polling cycle.
> * 'may_steal' patch returned to version from v3 because
> 'may_steal' in qos is a completely different variable. This
> patch only removes 'may_steal' from netdev API.
> * 2 more usec functions added to timeval to have complete public API.
> * Checking of 'output_cnt' turned to assertion.
>
> Version 4:
> * Rebased on current master.
> * Rebased on top of "Keep latest measured time for PMD thread."
> (Jan Scheurich)
> * Microsecond resolution related patches integrated.
> * Time-based batching without RFC tag.
> * 'output_time' renamed to 'flush_time'. (Jan Scheurich)
> * 'flush_time' update moved to 'dp_netdev_pmd_flush_output_on_port'.
> (Jan Scheurich)
> * 'output-max-latency' renamed to 'tx-flush-interval'.
> * Added patch for output batching statistics.
>
> Version 3:
>
> * Rebased on current master.
> * Time based RFC: fixed assert on n_output_batches <= 0.
>
> Version 2:
>
> * Rebased on current master.
> * Added time based batching RFC patch.
> * Fixed mixing packets with different sources in same batch.
>
>
> Ilya Maximets (7):
> dpif-netdev: Refactor PMD thread structure for further extension.
> dpif-netdev: Keep latest measured time for PMD thread.
> dpif-netdev: Output packet batching.
> netdev: Remove unused may_steal.
> netdev: Remove useless cutlen.
> dpif-netdev: Time based output batching.
> dpif-netdev: Count sent packets and batches.
>
> lib/dpif-netdev.c | 412 +++++++++++++++++++++++++++++++++++++-------------
> lib/netdev-bsd.c | 6 +-
> lib/netdev-dpdk.c | 30 ++--
> lib/netdev-dummy.c | 6 +-
> lib/netdev-linux.c | 8 +-
> lib/netdev-provider.h | 7 +-
> lib/netdev.c | 12 +-
> lib/netdev.h | 2 +-
> vswitchd/vswitch.xml | 16 ++
> 9 files changed, 349 insertions(+), 150 deletions(-)
>
> --
> 2.7.4
More information about the dev
mailing list