[ovs-dev] [PATCH v6 0/6] OVS-DPDK rxq to pmd assignment improvements.

Kevin Traynor ktraynor at redhat.com
Wed Sep 13 15:36:19 UTC 2017


On 09/12/2017 12:09 AM, Darrell Ball wrote:
> 
> 
> On 9/11/17, 12:25 PM, "Federico Iezzi" <fiezzi at redhat.com> wrote:
> 
>     On Fri, Aug 25, 2017 at 10:36 AM, Darrell Ball <dball at vmware.com> wrote:
>     > I applied the series to https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_darball_ovs_commits_dpdk-5Fmerge&d=DwIBaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=C5lG7aon2zE9j1Xvdo9MBQuwqOjQOBRrUcTTFdcxT6s&s=Pb67xViu5aZuBs9GL7Vp2uG4WXE_TT35X-JlmaDd_yQ&e= 
>     
>     I believe this series is worth a mention in the NEWS, reengineering an
>     important OVS-DPDK limitation.
> 
> 
> Part of the enhancement is not easily user visible, as an underlying algorithm was changed, which has a probabilistic benefit.
> So, I guess you refer to the rebalance command addition ?
> We had identified some enhancements that were to be done to guide the use of the rebalance command, which is probably the
> main reason why we had not added to NEWS yet.
> However, we can still add to NEWS for 2.8.1, if that is the consensus ?
> 

I don't really mind either way, but I'd lean towards leaving it out for
the time being for the reasons Darrell mentioned.

>     
>     >
>     >
>     >
>     > On 8/24/17, 4:37 PM, "Kevin Traynor" <ktraynor at redhat.com> wrote:
>     >
>     >     For the DPDK datapath, by default rxqs are assigned to available pmds
>     >     in round robin order with no weight or priority.
>     >
>     >     It can happen that some very busy queues are handled by one pmd which
>     >     does not have enough cycles to prevent packets being dropped on them.
>     >     While at the same time another pmd which handles queues with no traffic
>     >     on them is essentially idle.
>     >
>     >     Rxq to pmd assignment happens as a result of a number of events and
>     >     when it does, the same unweighted round robin approach is applied
>     >     each time.
>     >
>     >     This patchset proposes to improve the round robin nature of rxq to pmd
>     >     assignment by counting the processing cycles used by the rxqs during
>     >     their operation and incorporating that data into assignment.
>     >
>     >     Before assigning in a round robin manner, the rxqs will be sorted in
>     >     order of the processing cycles they have been consuming. Assuming
>     >     multiple pmds, this ensures that the rxqs measured to be using the
>     >     most processing cycles will be assigned to different cores.
>     >
>     >     In some cases the measured cycles for an rxq may be not available as
>     >     the rxq is new or may not be useful for assignment as traffic patterns
>     >     may change.  In those cases the code will essentially fallback to being
>     >     round round similar to what currently exists. However, in the case
>     >     where data is available and a reliable indication of future rxq cycles
>     >     consumption, rxq to pmd distribution will be much improved.
>     >
>     >     V5 -> V6
>     >     ========
>     >     Minor changes to 2/6, 3/6, 4/6, 5/6 from Darrell's review comments.
>     >
>     >     V4 -> V5
>     >     ========
>     >     Changed history of rxq considered during assignment to 1 min. In order
>     >     to have data available quicker than 1 min and not to be using up to
>     >     1 min old data, introduced storing of data in multiple intervals
>     >     similar to suggestion by Darrell. Some minor name changes to reflect
>     >     this.
>     >
>     >     2/6 Added storage for multiple intervals
>     >     3/6 Store cycles into interval
>     >     4/6 Sum cycles from intervals and use total for assignment
>     >
>     >     V3 -> V4
>     >     ========
>     >     4/6
>     >     Rebased to accomodate new cross numa assigment.
>     >
>     >     V2 -> V3
>     >     ========
>     >     Dropped v2 1/7 as not reusing dpcls optimisation interval anymore
>     >
>     >     2/6
>     >     Moved unused functions to 3/6 to avoid compiler warning
>     >
>     >     3/6
>     >     Made pmd rxq interval independent from dpcls opt interval
>     >
>     >     4/6
>     >     Moved docs about rebalance command to when it is available in 6/6
>     >     Added logging info for pmd to rxq assignment
>     >
>     >     5/6
>     >     Added an example to docs
>     >
>     >     6/6
>     >     Noted in commit msg that Jan requested this for testing purposes
>     >
>     >     V1 -> V2
>     >     ========
>     >     Dropped Ciara's patch to change how pmd cycles are counted as it merged.
>     >
>     >     6/7: Rebased unit tests.
>     >
>     >
>     >     Kevin Traynor (6):
>     >       dpif-netdev: Change polled_queue to use dp_netdev_rxq.
>     >       dpif-netdev: Add rxq processing cycle counters.
>     >       dpif-netdev: Count the rxq processing cycles for an rxq.
>     >       dpif-netdev: Change rxq_scheduling to use rxq processing cycles.
>     >       dpif-netdev: Change pmd selection order.
>     >       dpif-netdev: Add ovs-appctl dpif-netdev/pmd-rxq-rebalance.
>     >
>     >      Documentation/howto/dpdk.rst |  26 ++++
>     >      lib/dpif-netdev.c            | 298 ++++++++++++++++++++++++++++++++++++-------
>     >      tests/pmd.at                 |   2 +-
>     >      vswitchd/ovs-vswitchd.8.in   |   2 +
>     >      4 files changed, 284 insertions(+), 44 deletions(-)
>     >
>     >     --
>     >     1.8.3.1
>     >
>     >
>     >
>     >
>     >
>     > _______________________________________________
>     > dev mailing list
>     > dev at openvswitch.org
>     > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwIBaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=C5lG7aon2zE9j1Xvdo9MBQuwqOjQOBRrUcTTFdcxT6s&s=mJU1HPcKumQmctVZgLs-cRJ-_3DDawlR3b67lBPcel0&e= 
>     
> 



More information about the dev mailing list