[ovs-dev] [PATCH v1] doc: Add "Representors" topic document

[Flavio Leitner]

On Thu, Feb 14, 2019 at 01:47:27PM +0300, Ilya Maximets wrote:
> On 13.02.2019 19:39, Ian Stokes wrote:
> > On 2/13/2019 8:14 AM, Ophir Munk wrote:
> >>
> >>
> >>> -----Original Message-----
> >>>> -----Original Message-----
> >>>> From: Ian Stokes <ian.stokes at intel.com>
> >>>> Sent: Tuesday, February 12, 2019 7:17 PM
> >>>> To: Ilya Maximets <i.maximets at samsung.com>; Ophir Munk
> >>>> <ophirmu at mellanox.com>; ovs-dev at openvswitch.org
> >>>> Cc: Olga Shern <olgas at mellanox.com>; Kevin Traynor
> >>>> <ktraynor at redhat.com>; Asaf Penso <asafp at mellanox.com>; Thomas
> >>>> Monjalon <thomas at monjalon.net>
> >>>> Subject: Re: [PATCH v1] doc: Add "Representors" topic document
> >>>>
> >>>> On 2/12/2019 1:15 PM, Ilya Maximets wrote:
> >>>>> On 11.02.2019 3:01, Ophir Munk wrote:
> >>>>>> This details how to configure representors ports.
> >>>>>>
> >>>>>> Signed-off-by: Ophir Munk <ophirmu at mellanox.com>
> >>>>>> ---
> >>>>>>    Documentation/topics/dpdk/phy.rst | 80
> >>>> +++++++++++++++++++++++++++++++++++++++
> >>>>>>    1 file changed, 80 insertions(+)
> >>>>>>
> >> ...
> >>>>>> +
> >>>>>> +Representors
> >>>>>> +------------
> >>> .....
> >>>>>> +
> >>>>>> +.. important::
> >>>>>> +
> >>>>>> +   Representors ports are configured prior to OVS invocation and
> >>>> independently
> >>>>>> +   of it, or by other means as well. Please consult a NIC vendor
> >>>> instructions
> >>>>>> +   on how to establish representors.
> >>>>>
> >>>>> It'll be good to have configuration example for at least one
> >>>>> commonly used NIC (ixgbe/i40e ?). Or maybe a link to the docs where
> >>>>> the process
> >>>> described.
> >>>>>
> >>>>> What do you think ?
> >>>>> Ian, maybe you could add some example, since you have already tried
> >>>>> it in
> >>>> practice?
> >>>>>
> >>>>
> >>>> Good call, I'll draw up an incremental and post here, if acceptable we
> >>>> can roll it into the same patch.
> >>>>
> >>>
> >>> In addition to Ian drawing - please find a link which details how to create VFs
> >>> for ConnectX-4 5 or 6 NICs:
> >>> https://docs.openstack.org/neutron/rocky/admin/config-ovs-offload.html
> >>> section "Create Compute virtual functions".
> >>> I will send an updated v2 with this reference.
> >>>
> >>>> Ian
> >>
> >> After giving it more thought - IMHO it's better to add a new document under howto/ where NIC vendors could specify their configuration for representors setup.
> >> We can then refer from this document to the new one.
> >> Ilay, Ian - what do you think?
> >>
> > 
> > Hmmm, for ixgbe/i40e that are bound to a dpdk compatible driver it's just a case of creating the vf with something similar to below
> > 
> > echo 1 > /sys/bus/pci/devices/0000\:05\:00.0/max_vfs
> > 
> > And then adding it as described above
> > 
> > ovs-vsctl add-port br0 dpdk-rep5 -- set Interface dpdk-rep0 type=dpdk \
> > options:dpdk-devargs=0000:05:00.0,representor=[0]
> > 
> > I'm not sure that it would warrant it's own document for those devices, how does it compare to the mellanox creation of vfs? Is that more complex or different?
> 
> I know that to make representors appear as linux interfaces (for using them
> in OVS kernel datapath, for example) you need to turn the PF into "switchdev"
> mode. Probably, the same needed with DPDK.
> 
> Anyway, The instructions for most of NICs, I guess, should be "just echo number
> of VFs to 'max_vfs' sysfs knob". We may add this information right here and
> make a remark that some NICs may require additional steps. We may also point
> to the openstack ovs-offload guide, first parts of it describes how to configure
> representors with Mellanox NICs.
> IMHO, this amount of information should be enough and it does not deserve a
> separate manual.

I agree it would be very useful to have instructions on how to put the
card in the correct state to use the representors.  However, I'd avoid
pointing to other guides outside of OVS because we don't control them,
so they can change and move and our side will break. Also that usually
they include more instruction/details than actually is needed to make
it work, which can be confusing.

I think people building OSP, for instance (but could be any other
project), would look after OVS documentation on how to build their
side and not vice-versa :-).

fbl