[ovs-dev] [PATCH v1 1/1] DPDK: Remove support for vhost-user zero-copy.

Xu, Chenjie chenjie.xu at intel.com
Wed Sep 9 01:58:02 UTC 2020

Hi all,
I'm wondering why vhost-user dequeue zero-copy is removed. Could you please help me understand this question or let me know where I can find the discussions about this question?

Best Regards,
Xu, Chenjie

-----Original Message-----
From: dev <ovs-dev-bounces at openvswitch.org> On Behalf Of Ian Stokes
Sent: Tuesday, September 8, 2020 11:27 PM
To: dev at openvswitch.org
Cc: maxime.coquelin at redhat.com; i.maximets at ovn.org
Subject: [ovs-dev] [PATCH v1 1/1] DPDK: Remove support for vhost-user zero-copy.

Support for vhost-user dequeue zero-copy was deprecated in OVS 2.14 with the aim of removing it for OVS 2.15. Support for zero-copy will also be removed from DPDK 20.11. As such remove support from OVS.

Signed-off-by: Ian Stokes <ian.stokes at intel.com>
 Documentation/topics/dpdk/vhost-user.rst | 72 --------------------------------
 NEWS                                     |  2 +
 lib/netdev-dpdk.c                        | 25 -----------
 vswitchd/vswitch.xml                     | 11 -----
 4 files changed, 2 insertions(+), 108 deletions(-)

diff --git a/Documentation/topics/dpdk/vhost-user.rst b/Documentation/topics/dpdk/vhost-user.rst
index 4af738d11..595e40cde 100644
--- a/Documentation/topics/dpdk/vhost-user.rst
+++ b/Documentation/topics/dpdk/vhost-user.rst
@@ -553,78 +553,6 @@ shown with::
   $ ovs-vsctl get Interface dpdkvhostclient0 statistics:ovs_tx_retries
-vhost-user Dequeue Zero Copy (experimental)
-.. warning::
-   vhost-user Dequeue Zero Copy is deprecated in OVS and will be removed in
-   the next release.
-Normally when dequeuing a packet from a vHost User device, a memcpy operation -must be used to copy that packet from guest address space to host address -space. This memcpy can be removed by enabling dequeue zero-copy like so::
-    $ ovs-vsctl add-port br0 dpdkvhostuserclient0 -- set Interface \
-        dpdkvhostuserclient0 type=dpdkvhostuserclient \
-        options:vhost-server-path=/tmp/dpdkvhostclient0 \
-        options:dq-zero-copy=true
-With this feature enabled, a reference (pointer) to the packet is passed to -the host, instead of a copy of the packet. Removing this memcpy can give a -performance improvement for some use cases, for example switching large packets -between different VMs. However additional packet loss may be observed.
-Note that the feature is disabled by default and must be explicitly enabled -by setting the ``dq-zero-copy`` option to ``true`` while specifying the -``vhost-server-path`` option as above. If you wish to split out the command -into multiple commands as below, ensure ``dq-zero-copy`` is set before
-    $ ovs-vsctl set Interface dpdkvhostuserclient0 options:dq-zero-copy=true
-    $ ovs-vsctl set Interface dpdkvhostuserclient0 \
-        options:vhost-server-path=/tmp/dpdkvhostclient0
-The feature is only available to ``dpdkvhostuserclient`` port types.
-A limitation exists whereby if packets from a vHost port with -``dq-zero-copy=true`` are destined for a ``dpdk`` type port, the number of tx -descriptors (``n_txq_desc``) for that port must be reduced to a smaller number,
-128 being the recommended value. This can be achieved by issuing the following
-    $ ovs-vsctl set Interface dpdkport options:n_txq_desc=128
-Note: The sum of the tx descriptors of all ``dpdk`` ports the VM will send to -should not exceed 128. For example, in case of a bond over two physical ports -in balance-tcp mode, one must divide 128 by the number of links in the bond.
-Refer to :ref:`dpdk-queues-sizes` for more information.
-The reason for this limitation is due to how the zero copy functionality is -implemented. The vHost device's 'tx used vring', a virtio structure used for -tracking used ie. sent descriptors, will only be updated when the NIC frees -the corresponding mbuf. If we don't free the mbufs frequently enough, that -vring will be starved and packets will no longer be processed. One way to -ensure we don't encounter this scenario, is to configure ``n_txq_desc`` to a -small enough number such that the 'mbuf free threshold' for the NIC will be hit -more often and thus free mbufs more frequently. The value of 128 is suggested, -but values of 64 and 256 have been tested and verified to work too, with -differing performance characteristics. A value of 512 can be used too, if the -virtio queue size in the guest is increased to 1024 (available to configure in -QEMU versions v2.10 and greater). This value can be set like so::
-    $ qemu-system-x86_64 ... -chardev socket,id=char1,path=<sockpath>,server
-      -netdev type=vhost-user,id=mynet1,chardev=char1,vhostforce
-      -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,
-      tx_queue_size=1024
-Because of this limitation, this feature is considered 'experimental'.
-.. note::
-   Post-copy Live Migration is not compatible with dequeue zero copy.
 Further information can be found in the  `DPDK documentation  <https://doc.dpdk.org/guides-19.11/prog_guide/vhost_lib.html>`__
diff --git a/NEWS b/NEWS
index 2f67d5047..2db246d1a 100644
--- a/NEWS
+++ b/NEWS
@@ -1,5 +1,7 @@
+   - DPDK:
+     * Deprecated vhost-user dequeue zero-copy support removed.
 v2.14.0 - 17 Aug 2020
diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 18c4adcc7..515ad612e 100644
--- a/lib/netdev-dpdk.c
+++ b/lib/netdev-dpdk.c
@@ -2032,12 +2032,6 @@ netdev_dpdk_vhost_client_set_config(struct netdev *netdev,
         if (!nullable_string_is_equal(path, dev->vhost_id)) {
             dev->vhost_id = nullable_xstrdup(path);
-            /* check zero copy configuration */
-            if (smap_get_bool(args, "dq-zero-copy", false)) {
-                dev->vhost_driver_flags |= RTE_VHOST_USER_DEQUEUE_ZERO_COPY;
-            } else {
-                dev->vhost_driver_flags &= ~RTE_VHOST_USER_DEQUEUE_ZERO_COPY;
-            }
@@ -5028,7 +5022,6 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev *netdev)
     int err;
     uint64_t vhost_flags = 0;
     uint64_t vhost_unsup_flags;
-    bool zc_enabled;
@@ -5054,19 +5047,6 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev *netdev)
             vhost_flags |= RTE_VHOST_USER_POSTCOPY_SUPPORT;
-        zc_enabled = dev->vhost_driver_flags
-                     & RTE_VHOST_USER_DEQUEUE_ZERO_COPY;
-        /* Enable zero copy flag, if requested */
-        if (zc_enabled) {
-            vhost_flags |= RTE_VHOST_USER_DEQUEUE_ZERO_COPY;
-            /* DPDK vHost library doesn't allow zero-copy with linear buffers.
-             * Hence disable Linear buffer.
-             */
-            vhost_flags &= ~RTE_VHOST_USER_LINEARBUF_SUPPORT;
-            VLOG_WARN("Zero copy enabled, disabling linear buffer"
-                      " check for vHost port %s", dev->up.name);
-        }
         /* Enable External Buffers if TCP Segmentation Offload is enabled. */
         if (userspace_tso_enabled()) {
             vhost_flags |= RTE_VHOST_USER_EXTBUF_SUPPORT; @@ -5083,11 +5063,6 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev *netdev)
             VLOG_INFO("vHost User device '%s' created in 'client' mode, "
                       "using client socket '%s'",
                       dev->up.name, dev->vhost_id);
-            if (zc_enabled) {
-                VLOG_INFO("Zero copy enabled for vHost port %s", dev->up.name);
-                VLOG_WARN("Zero copy support is deprecated and will be "
-                          "removed in the next OVS release.");
-            }
         err = rte_vhost_driver_callback_register(dev->vhost_id,
diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml index 81c84927f..07da2ee8c 100644
--- a/vswitchd/vswitch.xml
+++ b/vswitchd/vswitch.xml
@@ -3236,17 +3236,6 @@ ovs-vsctl add-port br0 p0 -- set Interface p0 type=patch options:peer=p1 \
-      <column name="options" key="dq-zero-copy"
-              type='{"type": "boolean"}'>
-        <p>
-          The value specifies whether or not to enable dequeue zero copy on
-          the given interface.
-          Must be set before vhost-server-path is specified.
-          Only supported by dpdkvhostuserclient interfaces.
-          The feature is considered experimental.
-        </p>
-      </column>
       <column name="options" key="tx-retries-max"
               type='{"type": "integer", "minInteger": 0, "maxInteger": 32}'>

dev mailing list
dev at openvswitch.org

More information about the dev mailing list