[ovs-dev] [PATCH v1 1/1] DPDK: Remove support for vhost-user zero-copy.

Kevin Traynor ktraynor at redhat.com
Wed Sep 9 10:34:09 UTC 2020


On 08/09/2020 16:26, Ian Stokes wrote:
> Support for vhost-user dequeue zero-copy was deprecated in OVS 2.14 with
> the aim of removing it for OVS 2.15. Support for zero-copy will also be
> removed from DPDK 20.11. As such remove support from OVS.
> 

Maybe you can add some links to the DPDK patches and OVS discussions
seen as they are being requested on the ML.

> Signed-off-by: Ian Stokes <ian.stokes at intel.com>
> ---
>  Documentation/topics/dpdk/vhost-user.rst | 72 --------------------------------
>  NEWS                                     |  2 +
>  lib/netdev-dpdk.c                        | 25 -----------
>  vswitchd/vswitch.xml                     | 11 -----
>  4 files changed, 2 insertions(+), 108 deletions(-)
> 
> diff --git a/Documentation/topics/dpdk/vhost-user.rst b/Documentation/topics/dpdk/vhost-user.rst
> index 4af738d11..595e40cde 100644
> --- a/Documentation/topics/dpdk/vhost-user.rst
> +++ b/Documentation/topics/dpdk/vhost-user.rst
> @@ -553,78 +553,6 @@ shown with::
>  
>    $ ovs-vsctl get Interface dpdkvhostclient0 statistics:ovs_tx_retries
>  
> -vhost-user Dequeue Zero Copy (experimental)
> --------------------------------------------
> -
> -.. warning::
> -
> -   vhost-user Dequeue Zero Copy is deprecated in OVS and will be removed in
> -   the next release.
> -
> -Normally when dequeuing a packet from a vHost User device, a memcpy operation
> -must be used to copy that packet from guest address space to host address
> -space. This memcpy can be removed by enabling dequeue zero-copy like so::
> -
> -    $ ovs-vsctl add-port br0 dpdkvhostuserclient0 -- set Interface \
> -        dpdkvhostuserclient0 type=dpdkvhostuserclient \
> -        options:vhost-server-path=/tmp/dpdkvhostclient0 \
> -        options:dq-zero-copy=true
> -
> -With this feature enabled, a reference (pointer) to the packet is passed to
> -the host, instead of a copy of the packet. Removing this memcpy can give a
> -performance improvement for some use cases, for example switching large packets
> -between different VMs. However additional packet loss may be observed.
> -
> -Note that the feature is disabled by default and must be explicitly enabled
> -by setting the ``dq-zero-copy`` option to ``true`` while specifying the
> -``vhost-server-path`` option as above. If you wish to split out the command
> -into multiple commands as below, ensure ``dq-zero-copy`` is set before
> -``vhost-server-path``::
> -
> -    $ ovs-vsctl set Interface dpdkvhostuserclient0 options:dq-zero-copy=true
> -    $ ovs-vsctl set Interface dpdkvhostuserclient0 \
> -        options:vhost-server-path=/tmp/dpdkvhostclient0
> -
> -The feature is only available to ``dpdkvhostuserclient`` port types.
> -
> -A limitation exists whereby if packets from a vHost port with
> -``dq-zero-copy=true`` are destined for a ``dpdk`` type port, the number of tx
> -descriptors (``n_txq_desc``) for that port must be reduced to a smaller number,
> -128 being the recommended value. This can be achieved by issuing the following
> -command::
> -
> -    $ ovs-vsctl set Interface dpdkport options:n_txq_desc=128
> -
> -Note: The sum of the tx descriptors of all ``dpdk`` ports the VM will send to
> -should not exceed 128. For example, in case of a bond over two physical ports
> -in balance-tcp mode, one must divide 128 by the number of links in the bond.
> -
> -Refer to :ref:`dpdk-queues-sizes` for more information.
> -
> -The reason for this limitation is due to how the zero copy functionality is
> -implemented. The vHost device's 'tx used vring', a virtio structure used for
> -tracking used ie. sent descriptors, will only be updated when the NIC frees
> -the corresponding mbuf. If we don't free the mbufs frequently enough, that
> -vring will be starved and packets will no longer be processed. One way to
> -ensure we don't encounter this scenario, is to configure ``n_txq_desc`` to a
> -small enough number such that the 'mbuf free threshold' for the NIC will be hit
> -more often and thus free mbufs more frequently. The value of 128 is suggested,
> -but values of 64 and 256 have been tested and verified to work too, with
> -differing performance characteristics. A value of 512 can be used too, if the
> -virtio queue size in the guest is increased to 1024 (available to configure in
> -QEMU versions v2.10 and greater). This value can be set like so::
> -
> -    $ qemu-system-x86_64 ... -chardev socket,id=char1,path=<sockpath>,server
> -      -netdev type=vhost-user,id=mynet1,chardev=char1,vhostforce
> -      -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,
> -      tx_queue_size=1024
> -
> -Because of this limitation, this feature is considered 'experimental'.
> -
> -.. note::
> -
> -   Post-copy Live Migration is not compatible with dequeue zero copy.

There's a reference to this in the post-copy section too that can be
removed.

> -
>  Further information can be found in the
>  `DPDK documentation
>  <https://doc.dpdk.org/guides-19.11/prog_guide/vhost_lib.html>`__
> diff --git a/NEWS b/NEWS
> index 2f67d5047..2db246d1a 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -1,5 +1,7 @@
>  Post-v2.14.0
>  ---------------------
> +   - DPDK:
> +     * Deprecated vhost-user dequeue zero-copy support removed.
>  

Nit, not sure there's a need to say it was deprecated, and it reads a
little funny as it is not being deprecated it in this release.

>  
>  v2.14.0 - 17 Aug 2020
> diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
> index 18c4adcc7..515ad612e 100644
> --- a/lib/netdev-dpdk.c
> +++ b/lib/netdev-dpdk.c
> @@ -2032,12 +2032,6 @@ netdev_dpdk_vhost_client_set_config(struct netdev *netdev,
>          if (!nullable_string_is_equal(path, dev->vhost_id)) {
>              free(dev->vhost_id);
>              dev->vhost_id = nullable_xstrdup(path);
> -            /* check zero copy configuration */
> -            if (smap_get_bool(args, "dq-zero-copy", false)) {
> -                dev->vhost_driver_flags |= RTE_VHOST_USER_DEQUEUE_ZERO_COPY;
> -            } else {
> -                dev->vhost_driver_flags &= ~RTE_VHOST_USER_DEQUEUE_ZERO_COPY;
> -            }
>              netdev_request_reconfigure(netdev);
>          }
>      }
> @@ -5028,7 +5022,6 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev *netdev)
>      int err;
>      uint64_t vhost_flags = 0;
>      uint64_t vhost_unsup_flags;
> -    bool zc_enabled;
>  
>      ovs_mutex_lock(&dev->mutex);
>  
> @@ -5054,19 +5047,6 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev *netdev)
>              vhost_flags |= RTE_VHOST_USER_POSTCOPY_SUPPORT;
>          }
>  
> -        zc_enabled = dev->vhost_driver_flags
> -                     & RTE_VHOST_USER_DEQUEUE_ZERO_COPY;
> -        /* Enable zero copy flag, if requested */
> -        if (zc_enabled) {
> -            vhost_flags |= RTE_VHOST_USER_DEQUEUE_ZERO_COPY;
> -            /* DPDK vHost library doesn't allow zero-copy with linear buffers.
> -             * Hence disable Linear buffer.
> -             */
> -            vhost_flags &= ~RTE_VHOST_USER_LINEARBUF_SUPPORT;
> -            VLOG_WARN("Zero copy enabled, disabling linear buffer"
> -                      " check for vHost port %s", dev->up.name);
> -        }
> -
>          /* Enable External Buffers if TCP Segmentation Offload is enabled. */
>          if (userspace_tso_enabled()) {
>              vhost_flags |= RTE_VHOST_USER_EXTBUF_SUPPORT;
> @@ -5083,11 +5063,6 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev *netdev)
>              VLOG_INFO("vHost User device '%s' created in 'client' mode, "
>                        "using client socket '%s'",
>                        dev->up.name, dev->vhost_id);
> -            if (zc_enabled) {
> -                VLOG_INFO("Zero copy enabled for vHost port %s", dev->up.name);
> -                VLOG_WARN("Zero copy support is deprecated and will be "
> -                          "removed in the next OVS release.");
> -            }
>          }
>  
>          err = rte_vhost_driver_callback_register(dev->vhost_id,
> diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml
> index 81c84927f..07da2ee8c 100644
> --- a/vswitchd/vswitch.xml
> +++ b/vswitchd/vswitch.xml
> @@ -3236,17 +3236,6 @@ ovs-vsctl add-port br0 p0 -- set Interface p0 type=patch options:peer=p1 \
>          </p>
>        </column>
>  
> -      <column name="options" key="dq-zero-copy"
> -              type='{"type": "boolean"}'>
> -        <p>
> -          The value specifies whether or not to enable dequeue zero copy on
> -          the given interface.
> -          Must be set before vhost-server-path is specified.
> -          Only supported by dpdkvhostuserclient interfaces.
> -          The feature is considered experimental.
> -        </p>
> -      </column>
> -
>        <column name="options" key="tx-retries-max"
>                type='{"type": "integer", "minInteger": 0, "maxInteger": 32}'>
>          <p>
> 

Other than the minor comments above, LGTM, thanks.



More information about the dev mailing list