[ovs-dev] [PATCH] dpdk: Support running PMD threads on cores > RTE_MAX_LCORE.

Flavio Leitner fbl at sysclose.org
Wed Dec 4 20:21:37 UTC 2019


On Wed, Dec 04, 2019 at 09:09:42PM +0100, David Marchand wrote:
> On Wed, Dec 4, 2019 at 8:52 PM Flavio Leitner <fbl at sysclose.org> wrote:
> >
> > On Mon, Dec 02, 2019 at 05:03:30PM +0100, David Marchand wrote:
> > > Most DPDK components make the assumption that rte_lcore_id() returns
> > > a valid lcore_id in [0..RTE_MAX_LCORE] range (with the exception of
> > > the LCORE_ID_ANY special value).
> > >
> > > OVS does not currently check which value is set in
> > > RTE_PER_LCORE(_lcore_id) which exposes us to potential crashes on DPDK
> > > side.
> > >
> > > Introduce a lcore allocator in OVS for PMD threads and map them to
> > > unused lcores from DPDK à la --lcores.
> > >
> > > The physical cores on which the PMD threads are running still
> > > constitutes an important information when debugging, so still keep
> > > those in the PMD thread names but add a new debug log when starting
> > > them.
> > >
> > > Synchronize DPDK internals on numa and cpuset for the PMD threads by
> > > registering them via the rte_thread_set_affinity() helper.
> > >
> > > Signed-off-by: David Marchand <david.marchand at redhat.com>
> > > ---
> >
> > I liked the idea because maybe we could lower RTE_MAX_LCORE and save
> > some memory. It's unusual to require the default amount anyways.
> 
> About decreasing RTE_MAX_LCORE, I had that in mind, but the problem is
> existing deployments with -c/-l in ovsdb.
> We could try to parse this, reformat and only pass --lcores to dpdk.
> Still feels a bit dangerous.

Always the legacy stuff... :)

> > I changed RTE_MAX_LCORE to 4, and ran some tests with CPU 8:
> > 2019-12-04T18:27:22.797Z|00001|dpdk(pmd-c08/id:3)|INFO|Initialised lcore 1 for core 8
> >
> > I didn't find any issues with either the patch or with the tests.
> >
> > Acked-by: Flavio Leitner <fbl at sysclose.org>
> 
> Thanks for the review and test!

BTW, it would be nice to have an ovs-appctl command to dump a list
of IDs and CPUs in case the logs are rotated, config changed and
whatnot. For instance, sos could get an accurate report dumping a
list.

-- 
fbl


More information about the dev mailing list