[ovs-dev] [PATCH ovs v3 0/2] Introduce dpdkvdpa netdev

[Ilya Maximets]

On 17.10.2019 13:16, Noa Ezra wrote:
> There are two approaches to communicate with a guest, using virtIO or
> SR-IOV.
> SR-IOV allows working with port representor which is attached to the
> OVS and a matching VF is given with pass-through to the VM.
> HW rules can process packets from up-link and direct them to the VF
> without going through SW (OVS) and therefore SR-IOV gives the best
> performance.
> However, SR-IOV architecture requires that the guest will use a driver
> which is specific to the underlying HW. Specific HW driver has two main
> drawbacks:
> 1. Breaks virtualization in some sense (VM aware of the HW), can also
>     limit the type of images supported.
> 2. Less natural support for live migration.
> 
> Using virtIO interface solves both problems, but reduces performance and
> causes losing of some functionality, for example, for some HW offload,
> working directly with virtIO cannot be supported.
> In order to solve this conflict, we created a new netdev type-dpdkvdpa.
> The new netdev is basically similar to a regular dpdk netdev, but it
> has some additional functionality for transferring packets from virtIO
> guest (VM) to a VF and vice versa. With this solution we can benefit
> both SR-IOV and virtIO.
> vDPA netdev is designed to support both SW and HW use-cases.
> HW mode will be used to configure vDPA capable devices. The support
> for this mode is on progress in the dpdk community.
> SW acceleration is used to leverage SR-IOV offloads to virtIO guests
> by relaying packets between VF and virtio devices and as a pre-step for
> supporting vDPA in HW mode.
> 
> Running example:
> 1. Configure OVS bridge and ports:
> ovs-vsctl add-br br0-ovs -- set bridge br0-ovs datapath_type=netdev
> ovs-vsctl add-port br0-ovs pf -- set Interface pf type=dpdk options: \
>          dpdk-devargs=<pf pci id>
> ovs-vsctl add-port br0 vdpa0 -- set Interface vdpa0 type=dpdkvdpa \
>          options:vdpa-socket-path=<sock path> \
>          options:vdpa-accelerator-devargs=<vf pci id> \
>          options:dpdk-devargs=<pf pci id>,representor=[id]
> 2. Run a virtIO guest (VM) in server mode that creates the socket of
>     the vDPA port.
> 3. Send traffic.
> 
> Noa Ezra (2):
>    netdev-dpdk-vdpa: Introduce dpdkvdpa netdev
>    netdev-dpdk: Add dpdkvdpa port
> 
>   Documentation/automake.mk           |   1 +
>   Documentation/topics/dpdk/index.rst |   1 +
>   Documentation/topics/dpdk/vdpa.rst  |  90 +++++
>   NEWS                                |   1 +
>   lib/automake.mk                     |   4 +-
>   lib/netdev-dpdk-vdpa.c              | 750 ++++++++++++++++++++++++++++++++++++
>   lib/netdev-dpdk-vdpa.h              |  54 +++
>   lib/netdev-dpdk.c                   | 162 ++++++++
>   vswitchd/vswitch.xml                |  25 ++
>   9 files changed, 1087 insertions(+), 1 deletion(-)
>   create mode 100644 Documentation/topics/dpdk/vdpa.rst
>   create mode 100755 lib/netdev-dpdk-vdpa.c
>   create mode 100644 lib/netdev-dpdk-vdpa.h
> 

Hi, everyone.

So, I have a few questions (mostly to Roni?):

1. What happened to idea of implementing this as a DPDK vdev?

2. What was the results of "direct output optimization" patch [1] testing?
    At this point I also have to mention that OVS is going to have TSO support
    in one of the next releases (at least we have some progress on that, thanks
    to idea of using extbuf).  This way in combine with patch [1] there should
    be no any benefits from having separate netdev for forwarding purposes.
    Even without this patch, havin TSO alone should provide good performance
    benefits making it considerable to not have any extra netdev.

    [1] https://patchwork.ozlabs.org/patch/1105880/


3. Regarding implementation, I expected less code duplication. It's unclear
    why you're not reusing hotplug capabilities of exiting dpdk netdev.
    I expected that this new netdev will call netdev_open() like a 3 times
    with a few tweaks for TSO enabling or stripping/adding some options.

4. This code seem doesn't make sense in sw mode without full HW offloading.
    i.e. it doesn't make sense to use this netdev while VF representor is
    attached to userspace datapath.  This is just because, according to the
    solution architecture, all the traffic will be copied firstly from the
    VM to VF (parsed on the fly to enable TSO), and will immediately appear
    on VF representor to be handled by OVS (same parsing with partial offload
    + actions execution).  I'm completely unsure if it any faster than just
    sending packet from VM to VF withoout bypassing OVS processing.  Maybe
    even slower and like 3 times heavier for PCI bandwidth.

5. "HW mode will be used to configure vDPA capable devices. The support
     for this mode is on progress in the dpdk community."
    AFAIU, host side of vDPA support is done already for some time in DPDK.
    API is prepared and there is a vhost-vdpa example that allows you to
    check the functionality.  As I understand, the missing part in dpdk right
    now is a DPDK virtio driver for guest, but this is not a limitation for
    not implementing vDPA support from the host side.  Am I missing something?

Best regards, Ilya Maximets.