[ovs-dev] [PATCH 2/2] Adding configuration option to whitelist DPDK physical ports.

[Chandran, Sugesh]

Regards
_Sugesh

> -----Original Message-----
> From: O Mahony, Billy
> Sent: Thursday, December 7, 2017 11:47 AM
> To: Chandran, Sugesh <sugesh.chandran at intel.com>; dev at openvswitch.org;
> blp at ovn.org
> Subject: RE: [ovs-dev] [PATCH 2/2] Adding configuration option to whitelist
> DPDK physical ports.
> 
> Hi Sugesh,
> 
> > -----Original Message-----
> > From: Chandran, Sugesh
> > Sent: Wednesday, December 6, 2017 6:23 PM
> > To: O Mahony, Billy <billy.o.mahony at intel.com>; dev at openvswitch.org;
> > blp at ovn.org
> > Subject: RE: [ovs-dev] [PATCH 2/2] Adding configuration option to
> > whitelist DPDK physical ports.
> >
> > Thank you Billy for the review.
> > Please find below my reply.
> >
> > Regards
> > _Sugesh
> >
> >
> > > -----Original Message-----
> > > From: O Mahony, Billy
> > > Sent: Wednesday, December 6, 2017 5:31 PM
> > > To: Chandran, Sugesh <sugesh.chandran at intel.com>;
> > > dev at openvswitch.org; blp at ovn.org
> > > Subject: RE: [ovs-dev] [PATCH 2/2] Adding configuration option to
> > > whitelist DPDK physical ports.
> > >
> > > Hi Sugesh,
> > >
> > > This is definitely a very useful feature. I'm looking forward to
> > > running trex on the same DUT as my ovs-dpdk.
> > >
> > > However I'd suggest adding an sscanf or some such to verify that the
> > > domain is also specified for each whitelist member. And either add
> > > the default of '0000' or complain loudly if the domain is absent.
> > [Sugesh] Will throw an error in that case then .
> >
> > >
> > > Currently (without this patch) you must specify the domain when adding
> ports:
> > >    Vsctl add-port ... options:dpdk-devargs=0000:05:00.0 Or else an
> > > error such as 'Cannot find unplugged device (05:00.0)'  is reported.
> > >
> > > And with the patch if you include the domain in the other_config (e.g.
> > > other_config:dpdk-whitelist-pci-ids="0000:05:00.0") everything works
> > > just as before.
> > >
> > > However with the patch if you add the whitelist *without* a domain e.g.
> > > 	ovs-vsctl --no-wait set Open_vSwitch .
> > > other_config:dpdk-whitelist-pci- ids="05:00.0"
> > >
> > > There is no immediate error. However later when doing add-port if
> > > you include the domain (current required practice) you will get an error.
> > > If you omit the domain all is well.
> > [Sugesh] It looks to me, the dpdk-devargs need the PCI id with the '0000'.
> > But to bind and PCI scan its not necessary.
> > So to keep it consistent, I would add check for PCI-ID in whitelist
> > config too, and throw error incase pci-id are mentioned wrong(means without
> '0000'.
> > Does it looks OK to you?
> 
> [[BO'M]] I think the error is the right thing to do. It would be tempting to insert
> the default '0000' if the domain is omitted but then you would have a confusing
> inconsistency in that it would be ok to omit the domain in one place (whitelist)
> but not in the other (add-port).
> 
[Sugesh] Thank you Billy,Will add the check and error out accordingly.
Will release the next patch version by incorporating the comments, once you have complete the
review of second part of the patch too.
> > >
> > > It's a little bit strange as regardless of domain or no domain in
> > > the other_config the PCI probe always reports the NIC as expected:
> > >     2017-12-06T16:55:27Z|00013|dpdk|INFO|EAL: PCI device
> > > 0000:05:00.0 on NUMA socket -1
> > >     2017-12-06T16:55:27Z|00014|dpdk|WARN|EAL:   Invalid NUMA socket,
> > > default to 0
> > >     2017-12-06T16:55:27Z|00015|dpdk|INFO|EAL:   probe driver: 8086:1572
> > > net_i40e
> > >
> > > I'll be using the other patch in this series "isolate rte-mempool
> > > allocation" over the next few days so I'll review that in due course.
> > >
> > > Thanks,
> > > Billy.
> > >
> > > > -----Original Message-----
> > > > From: ovs-dev-bounces at openvswitch.org [mailto:ovs-dev-
> > > > bounces at openvswitch.org] On Behalf Of Sugesh Chandran
> > > > Sent: Friday, November 10, 2017 1:29 AM
> > > > To: dev at openvswitch.org; blp at ovn.org
> > > > Subject: [ovs-dev] [PATCH 2/2] Adding configuration option to
> > > > whitelist DPDK physical ports.
> > > >
> > > > Adding a OVS configuration option to whitelist DPDK physical ports.
> > > > By default running multiple instances of DPDK on a single platform
> > > > cannot use physical ports at the same time even though they are distinct.
> > > >
> > > > The eal init scans all the ports that are bound to DPDK and
> > > > initialize the drivers accordingly. This happens for every DPDK process init.
> > > > On a multi instance deployment usecase, it causes issues for using
> > > > physical NIC ports.
> > > > Consider a two DPDK process that are running on a single platform,
> > > > the second DPDK primary process will try to initialize the drivers
> > > > for all the physical ports even though it may be used in first DPDK process.
> > > >
> > > > To avoid this situation user can whitelist the ports for each DPDK
> application.
> > > > Whitelisting of ports/PCI-ID in a DPDK process will limit the
> > > > eal-init only on those ports.
> > > >
> > > > To whitelist two physical ports "0000:06:00.0" and "0000:06:00.1",
> > > > the configuration option in OVS would be
> > > >   ovs-vsctl  set Open_vSwitch . other_config:dpdk-whitelist-pci-
> > > > ids="0000:06:00.0,0000:06:00.1"
> > > >
> > > > To update the whitelist ports, OVS daemon has to be restarted.
> > > >
> > > > Signed-off-by: Sugesh Chandran <sugesh.chandran at intel.com>
> > > > ---
> > > >  lib/dpdk.c           | 29 +++++++++++++++++++++++++++++
[snip]
> > > >
> > > > _______________________________________________
> > > > dev mailing list
> > > > dev at openvswitch.org
> > > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev