[ovs-dev] [PATCH RFC v6 1/1] netdev-dpdk: add dpdk vhost ports
Kevin Traynor
kevin.traynor at intel.com
Thu Jan 8 23:05:02 UTC 2015
This patch adds support for a new port type to userspace datapath
called dpdkvhost. This allows KVM (QEMU) to offload the servicing
of virtio-net devices to its associated dpdkvhost port. Instructions
for use are in INSTALL.DPDK.
This has been tested on Intel multi-core platforms and with clients
that have virtio-net interfaces.
ver 6:
- rebased with master
- modified to use DPDK v1.8.0 vhost library
- reworked for review comments
ver 5:
- rebased against latest master
ver 4:
- added eventfd_link.h and eventfd_link.c to EXTRA_DIST in
utilities/automake.mk
- rebased with master to work with DPDK 1.7 ver 3:
- rebased with master
ver 2:
- rebased with master
Signed-off-by: Ciara Loftus <ciara.loftus at intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor at intel.com>
Signed-off-by: Maryam Tahhan <maryam.tahhan at intel.com>
---
INSTALL.DPDK.md | 236 +++++++++++++++++
Makefile.am | 4 +
lib/automake.mk | 1 +
lib/netdev-dpdk.c | 649 +++++++++++++++++++++++++++++++++++++++--------
lib/netdev.c | 3 +-
utilities/automake.mk | 3 +-
utilities/qemu-wrap.py | 389 ++++++++++++++++++++++++++++
vswitchd/ovs-vswitchd.c | 4 +-
8 files changed, 1177 insertions(+), 112 deletions(-)
mode change 100644 => 100755 lib/netdev-dpdk.c
create mode 100755 utilities/qemu-wrap.py
diff --git a/INSTALL.DPDK.md b/INSTALL.DPDK.md
index 2cc7636..da8116d 100644
--- a/INSTALL.DPDK.md
+++ b/INSTALL.DPDK.md
@@ -17,6 +17,7 @@ Building and Installing:
------------------------
Required DPDK 1.7
+Optional `fuse`, `fuse-devel`
1. Configure build & install DPDK:
1. Set `$DPDK_DIR`
@@ -264,6 +265,241 @@ A general rule of thumb for better performance is that the client
application should not be assigned the same dpdk core mask "-c" as
the vswitchd.
+DPDK vHost:
+-----------
+
+Prerequisites:
+1. DPDK 1.8 with vHost support enabled and recompile OVS as above.
+
+ Update `config/common_linuxapp` so that DPDK is built with vHost
+ libraries:
+
+ `CONFIG_RTE_LIBRTE_VHOST=y`
+
+2. Insert the Fuse module:
+
+ `modprobe fuse`
+
+3. Build and insert the `eventfd_link` module:
+
+ `cd $DPDK_DIR/lib/librte_vhost/eventfd_link/`
+ `make`
+ `insmod $DPDK_DIR/lib/librte_vhost/eventfd_link.ko`
+
+4. Remove /dev/vhost-net character device:
+
+ `rm -rf /dev/vhost-net`
+
+Following the steps above to create a bridge, you can now add DPDK vHost
+as a port to the vswitch.
+
+`ovs-vsctl add-port br0 dpdkvhost0 -- set Interface dpdkvhost0 type=dpdkvhost`
+
+Unlike DPDK ring ports, DPDK vHost ports can have arbitrary names:
+
+`ovs-vsctl add-port br0 port123ABC -- set Interface port123ABC type=dpdkvhost`
+
+However, please note that when attaching userspace devices to QEMU, the
+name provided during the add-port operation must match the ifname parameter
+on the QEMU command line.
+
+DPDK vHost VM configuration:
+----------------------------
+
+1. Configure virtio-net adaptors:
+ The guest must be configured with virtio-net adapters and offloads
+ MUST BE DISABLED. This means the following parameters should be passed
+ to the QEMU binary:
+
+ ```
+ -netdev tap,id=<id>,script=no,downscript=no,ifname=<name>,vhost=on
+ -device virtio-net-pci,netdev=net1,mac=<mac>,csum=off,gso=off,
+ guest_tso4=off,guest_tso6=off,guest_ecn=off
+ ```
+
+ Repeat the above parameters for multiple devices.
+
+2. Configure huge pages:
+ QEMU must allocate the VM's memory on hugetlbfs. Vhost ports access a
+ virtio-net device's virtual rings and packet buffers mapping the VM's
+ physical memory on hugetlbfs. To enable vhost-ports to map the VM's
+ memory into their process address space, pass the following paramters
+ to QEMU:
+
+ `-mem-path /dev/hugepages -mem-prealloc`
+
+DPDK vHost with standard vHost:
+-------------------------------
+
+DPDK vHost ports use a Linux* character device to communicate with QEMU.
+By default it is set to `/dev/vhost-net`. This conflicts with the kernel
+vHost device, hence the need to remove `/dev/vhost-net` above. However,
+if you wish to use kernel vhost in parallel, you can specify an
+alternative basename on the vswitchd command line like so:
+
+ `./vswitchd/ovs-vswitchd --dpdk --basename my-vhost-net -c 0x1 ...`
+
+Note that the basename arguement and associated string must be the first
+arguements after `--dpdk` and come before the EAL arguements.
+
+DPDK vHost VM configuration with standard vHost:
+------------------------------------------------
+
+1. As with the "normal" (i.e. using `/dev/vhost-net`) DPDK vHost setup,
+the guest must be configured with virtio-net adapters and offloads
+MUST BE DISABLED. However, this time you must also pass in a `vhostfd`
+argument:
+
+ ```
+ -netdev tap,id=<id>,script=no,downscript=no,ifname=<name>,vhost=on,
+ vhostfd=<open_fd>
+ -device virtio-net-pci,netdev=net1,mac=<mac>,csum=off,gso=off,
+ guest_tso4=off,guest_tso6=off,guest_ecn=off
+ ```
+
+ The open file descriptor must be passed to QEMU running as a child
+ process.
+
+2. As above, QEMU must allocate the VM's memory on hugetlbfs:
+
+ `-mem-path /dev/hugepages -mem-prealloc`
+
+3. (Optional) If you are using libvirt, you must enable libvirt to access
+the userspace device file by adding it to controllers cgroup for libvirtd
+using the following steps:
+
+ 1. In `/etc/libvirt/qemu.conf` add/edit the following lines:
+
+ ```
+ 1) cgroup_controllers = [ ... "devices", ... ]
+ 2) clear_emulator_capabilities = 0
+ 3) user = "root"
+ 4) group = "root"
+ 5) cgroup_device_acl = [
+ "/dev/null", "/dev/full", "/dev/zero",
+ "/dev/random", "/dev/urandom",
+ "/dev/ptmx", "/dev/kvm", "/dev/kqemu",
+ "/dev/rtc", "/dev/hpet", "/dev/net/tun",
+ "/dev/<devbase-name>-<index>",
+ "/dev/hugepages"]
+ ```
+
+ 2. Disable SELinux or set to permissive mode
+ 3. Mount cgroup device controller:
+
+ ```
+ mkdir /dev/cgroup
+ mount -t cgroup none /dev/cgroup -o devices
+ ```
+
+ 4. Restart the libvirtd process
+ For example, on Fedora:
+
+ `systemctl restart libvirtd.service`
+
+The easiest way to setup a Guest that isn't using `/dev/vhost-net` is to
+use the `qemu-wrap.py` script located in utilities. This Python script
+automates the requirements specified above and can be used in conjunction
+with libvirt.
+
+DPDK vHost VM configuration with QEMU wrapper:
+----------------------------------------------
+
+The QEMU wrapper script automatically detects and calls QEMU with the
+necessary parameters required to integrate with the vhost sample code.
+It performs the following actions:
+
+ * Automatically detects the location of the hugetlbfs and inserts this
+ into the command line parameters.
+ * Automatically open file descriptors for each virtio-net device and
+ inserts this into the command line parameters.
+ * Disables offloads on each virtio-net device.
+ * Calls QEMU passing both the command line parameters passed to the
+ script itself and those it has auto-detected.
+
+Before use, you **must** edit the configuration parameters section of the
+script to point to the correct emulator location and set any additional
+options. Of these settings, `emul_path` and `us_vhost_path` **must** be
+set. All other parameters are optional.
+
+To use directly from the command line simply pass the wrapper some of the
+QEMU parameters: it will configure the rest. For example:
+
+```
+qemu-wrap.py -cpu host -boot c -hda <disk image> -m 4096 -smp 4
+ --enable-kvm -nographic -vnc none -net none -netdev tap,id=net1,
+ script=no,downscript=no,ifname=if1,vhost=on -device virtio-net-pci,
+ netdev=net1,mac=00:00:00:00:00:01
+```
+
+To use with libvirt, follow these steps:
+
+1. Place `qemu-wrap.py` in libvirtd's binary search PATH ($PATH)
+ Ideally in the same directory that the QEMU binary is located.
+
+2. Ensure that the script has the same owner/group and file permissions as the
+ QEMU binary.
+
+3. Update the VM xml file using "virsh edit VM.xml"
+
+ 1. Set the VM to use the launch script
+ Set the emulator path contained in the `<emulator><emulator/>` tags.
+ For example, replace:
+
+ `<emulator>/usr/bin/qemu-kvm<emulator/>`
+
+ with:
+
+ `<emulator>/usr/bin/qemu-wrap.py<emulator/>`
+
+ 2. Set the VM's devices to use vhost-net offload
+ ```
+ <interface type='network'>
+ <mac address='xx:xx:xx:xx:xx:xx'/>
+ <source network='default'/>
+ <model type='virtio'/>
+ <driver name='vhost'/>
+ <address type=.../>
+ </interface>
+ ```
+
+ 4. Enable libvirt to access our userpace device file by adding it
+ to controllers cgroup for libvirtd as specified above.
+
+ 5. (Optional) Set `hugetlbfs_mount` variable
+ VMs using userspace vhost must use hugepage backed memory. This
+ can be enabled in the libvirt XML config by adding a memory
+ backing section to the XML config:
+
+ ```
+ <memoryBacking>
+ <hugepages/>
+ </memoryBacking>
+ ```
+
+ This memory backing section should be added after the `<memory>`
+ and `<currentMemory>` sections. This will add the `-mem-prealloc`
+ and `-mem-path <path>` flags to the QEMU command line. The
+ `hugetlbfs_mount` variable can be used to override the default
+ `<path>` passed through by libvirt.
+
+ If `-mem-prealloc` or `-mem-path <path>` are not passed through
+ and a vhost device is detected then these options will be
+ automatically added by this script. This script will detect the
+ system hugetlbfs mount point to be used for `<path>`. The default
+ `<path>` for this script can be overidden by the `hugetlbfs_dir`
+ variable in the configuration section of this script.
+
+4. Restart the libvirtd system process
+ For example on Fedora:
+
+ `systemctl restart libvirtd.service`
+
+5. Edit the Configuration Parameters section of the script to point to
+the correct emulator location and set any additional options.
+
+6. Use virt-manager to launch the VM
+
Restrictions:
-------------
diff --git a/Makefile.am b/Makefile.am
index 6ad6fc2..c1fabdd 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -32,6 +32,10 @@ AM_CFLAGS = -Wstrict-prototypes
AM_CFLAGS += $(WARNING_FLAGS)
AM_CFLAGS += $(OVS_CFLAGS)
+if DPDK_NETDEV
+AM_CFLAGS += -D_FILE_OFFSET_BITS=64
+endif
+
if NDEBUG
AM_CPPFLAGS += -DNDEBUG
AM_CFLAGS += -fomit-frame-pointer
diff --git a/lib/automake.mk b/lib/automake.mk
index 2a5844b..42c133e 100644
--- a/lib/automake.mk
+++ b/lib/automake.mk
@@ -336,6 +336,7 @@ lib_libopenvswitch_la_SOURCES += \
endif
if DPDK_NETDEV
+lib_libopenvswitch_la_LDFLAGS += -lfuse
lib_libopenvswitch_la_SOURCES += \
lib/netdev-dpdk.c \
lib/netdev-dpdk.h
diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
old mode 100644
new mode 100755
index 6b77bc4..62a1a08
--- a/lib/netdev-dpdk.c
+++ b/lib/netdev-dpdk.c
@@ -30,6 +30,7 @@
#include <rte_config.h>
#include <rte_mbuf.h>
+#include <rte_virtio_net.h>
#include "dpif-netdev.h"
#include "list.h"
@@ -59,6 +60,7 @@ static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 20);
#define OVS_CACHE_LINE_SIZE CACHE_LINE_SIZE
#define OVS_VPORT_DPDK "ovs_dpdk"
+
/*
* need to reserve tons of extra space in the mbufs so we can align the
* DMA addresses to 4KB.
@@ -85,6 +87,12 @@ static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 20);
#define TX_HTHRESH 0 /* Default values of TX host threshold reg. */
#define TX_WTHRESH 0 /* Default values of TX write-back threshold reg. */
+#define MAX_BASENAME_SZ NAME_MAX /* Maximum character device basename size. */
+#define MAX_PKT_BURST 32 /* Max burst size for RX/TX */
+
+/* Character device basename. */
+char *dev_basename = NULL;
+
static const struct rte_eth_conf port_conf = {
.rxmode = {
.mq_mode = ETH_MQ_RX_RSS,
@@ -131,29 +139,30 @@ enum { MAX_TX_QUEUE_LEN = 384 };
enum { DPDK_RING_SIZE = 256 };
BUILD_ASSERT_DECL(IS_POW2(DPDK_RING_SIZE));
enum { DRAIN_TSC = 200000ULL };
+enum { DPDK = 0, VHOST };
static int rte_eal_init_ret = ENODEV;
-static struct ovs_mutex dpdk_mutex = OVS_MUTEX_INITIALIZER;
+static rte_spinlock_t dpdk_spinlock = RTE_SPINLOCK_INITIALIZER;
/* Contains all 'struct dpdk_dev's. */
-static struct ovs_list dpdk_list OVS_GUARDED_BY(dpdk_mutex)
+static struct ovs_list dpdk_list OVS_GUARDED_BY(dpdk_spinlock)
= OVS_LIST_INITIALIZER(&dpdk_list);
-static struct ovs_list dpdk_mp_list OVS_GUARDED_BY(dpdk_mutex)
+static struct ovs_list dpdk_mp_list OVS_GUARDED_BY(dpdk_spinlock)
= OVS_LIST_INITIALIZER(&dpdk_mp_list);
-/* This mutex must be used by non pmd threads when allocating or freeing
+/* This spinlock must be used by non pmd threads when allocating or freeing
* mbufs through mempools. Since dpdk_queue_pkts() and dpdk_queue_flush() may
- * use mempools, a non pmd thread should hold this mutex while calling them */
-struct ovs_mutex nonpmd_mempool_mutex = OVS_MUTEX_INITIALIZER;
+ * use mempools, a non pmd thread should hold this spinlock while calling them */
+rte_spinlock_t nonpmd_mempool_spinlock = RTE_SPINLOCK_INITIALIZER;
struct dpdk_mp {
struct rte_mempool *mp;
int mtu;
int socket_id;
int refcount;
- struct ovs_list list_node OVS_GUARDED_BY(dpdk_mutex);
+ struct ovs_list list_node OVS_GUARDED_BY(dpdk_spinlock);
};
/* There should be one 'struct dpdk_tx_queue' created for
@@ -170,7 +179,7 @@ struct dpdk_tx_queue {
so we have to keep them around once they've been created
*/
-static struct ovs_list dpdk_ring_list OVS_GUARDED_BY(dpdk_mutex)
+static struct ovs_list dpdk_ring_list OVS_GUARDED_BY(dpdk_spinlock)
= OVS_LIST_INITIALIZER(&dpdk_ring_list);
struct dpdk_ring {
@@ -179,17 +188,18 @@ struct dpdk_ring {
struct rte_ring *cring_rx;
int user_port_id; /* User given port no, parsed from port name */
int eth_port_id; /* ethernet device port id */
- struct ovs_list list_node OVS_GUARDED_BY(dpdk_mutex);
+ struct ovs_list list_node OVS_GUARDED_BY(dpdk_spinlock);
};
struct netdev_dpdk {
struct netdev up;
int port_id;
int max_packet_len;
+ int type; /* DPDK or VHOST */
struct dpdk_tx_queue *tx_q;
- struct ovs_mutex mutex OVS_ACQ_AFTER(dpdk_mutex);
+ rte_spinlock_t spinlock OVS_ACQ_AFTER(dpdk_spinlock);
struct dpdk_mp *dpdk_mp;
int mtu;
@@ -203,8 +213,11 @@ struct netdev_dpdk {
struct rte_eth_link link;
int link_reset_cnt;
+ /* virtio-net structure for vhost device */
+ OVSRCU_TYPE(struct virtio_net *) virtio_dev;
+
/* In dpdk_list. */
- struct ovs_list list_node OVS_GUARDED_BY(dpdk_mutex);
+ struct ovs_list list_node OVS_GUARDED_BY(dpdk_spinlock);
rte_spinlock_t dpdkr_tx_lock;
};
@@ -217,14 +230,18 @@ static bool thread_is_pmd(void);
static int netdev_dpdk_construct(struct netdev *);
+void *start_cuse_session_loop(void *dummy);
+
+struct virtio_net * netdev_dpdk_get_virtio(const struct netdev_dpdk *dev);
+
static bool
is_dpdk_class(const struct netdev_class *class)
{
return class->construct == netdev_dpdk_construct;
}
-/* XXX: use dpdk malloc for entire OVS. infact huge page shld be used
- * for all other sengments data, bss and text. */
+/* XXX: use dpdk malloc for entire OVS. in fact huge page should be used
+ * for all other segments data, bss and text. */
static void *
dpdk_rte_mzalloc(size_t sz)
@@ -239,7 +256,7 @@ dpdk_rte_mzalloc(size_t sz)
}
/* XXX this function should be called only by pmd threads (or by non pmd
- * threads holding the nonpmd_mempool_mutex) */
+ * threads holding the nonpmd_mempool_spinlock) */
void
free_dpdk_buf(struct dpif_packet *p)
{
@@ -290,7 +307,7 @@ ovs_rte_pktmbuf_init(struct rte_mempool *mp,
}
static struct dpdk_mp *
-dpdk_mp_get(int socket_id, int mtu) OVS_REQUIRES(dpdk_mutex)
+dpdk_mp_get(int socket_id, int mtu) OVS_REQUIRES(dpdk_spinlock)
{
struct dpdk_mp *dmp = NULL;
char mp_name[RTE_MEMPOOL_NAMESIZE];
@@ -378,13 +395,13 @@ dpdk_watchdog(void *dummy OVS_UNUSED)
pthread_detach(pthread_self());
for (;;) {
- ovs_mutex_lock(&dpdk_mutex);
+ rte_spinlock_lock(&dpdk_spinlock);
LIST_FOR_EACH (dev, list_node, &dpdk_list) {
- ovs_mutex_lock(&dev->mutex);
+ rte_spinlock_lock(&dev->spinlock);
check_link_status(dev);
- ovs_mutex_unlock(&dev->mutex);
+ rte_spinlock_unlock(&dev->spinlock);
}
- ovs_mutex_unlock(&dpdk_mutex);
+ rte_spinlock_unlock(&dpdk_spinlock);
xsleep(DPDK_PORT_WATCHDOG_INTERVAL);
}
@@ -392,7 +409,7 @@ dpdk_watchdog(void *dummy OVS_UNUSED)
}
static int
-dpdk_eth_dev_init(struct netdev_dpdk *dev) OVS_REQUIRES(dpdk_mutex)
+dpdk_eth_dev_init(struct netdev_dpdk *dev) OVS_REQUIRES(dpdk_spinlock)
{
struct rte_pktmbuf_pool_private *mbp_priv;
struct ether_addr eth_addr;
@@ -485,15 +502,15 @@ netdev_dpdk_alloc_txq(struct netdev_dpdk *netdev, unsigned int n_txqs)
static int
netdev_dpdk_init(struct netdev *netdev_, unsigned int port_no)
- OVS_REQUIRES(dpdk_mutex)
+ OVS_REQUIRES(dpdk_spinlock)
{
struct netdev_dpdk *netdev = netdev_dpdk_cast(netdev_);
int sid;
int err = 0;
- ovs_mutex_init(&netdev->mutex);
+ rte_spinlock_init(&netdev->spinlock);
- ovs_mutex_lock(&netdev->mutex);
+ rte_spinlock_lock(&netdev->spinlock);
/* If the 'sid' is negative, it means that the kernel fails
* to obtain the pci numa info. In that situation, always
@@ -502,6 +519,7 @@ netdev_dpdk_init(struct netdev *netdev_, unsigned int port_no)
netdev->socket_id = sid < 0 ? SOCKET0 : sid;
netdev_dpdk_alloc_txq(netdev, NR_QUEUE);
netdev->port_id = port_no;
+ netdev->type = DPDK;
netdev->flags = 0;
netdev->mtu = ETHER_MTU;
netdev->max_packet_len = MTU_TO_MAX_LEN(netdev->mtu);
@@ -526,7 +544,7 @@ unlock:
if (err) {
rte_free(netdev->tx_q);
}
- ovs_mutex_unlock(&netdev->mutex);
+ rte_spinlock_unlock(&netdev->spinlock);
return err;
}
@@ -546,6 +564,65 @@ dpdk_dev_parse_name(const char dev_name[], const char prefix[],
}
static int
+netdev_dpdk_vhost_construct(struct netdev *netdev_)
+{
+ struct netdev_dpdk *netdev = netdev_dpdk_cast(netdev_);
+ struct netdev_dpdk *tmp_netdev;
+ unsigned int port_no = 0;
+ int err = 0;
+ int sid = 0;
+
+ if (rte_eal_init_ret) {
+ return rte_eal_init_ret;
+ }
+ rte_spinlock_lock(&dpdk_spinlock);
+ rte_spinlock_init(&netdev->spinlock);
+ rte_spinlock_lock(&netdev->spinlock);
+
+ if (!list_is_empty(&dpdk_list)) {
+ LIST_FOR_EACH (tmp_netdev, list_node, &dpdk_list) {
+ if (tmp_netdev->type== VHOST) {
+ port_no++;
+ }
+ }
+ }
+
+ /* Retrieving socket id for master DPDK core */
+ sid = rte_lcore_to_socket_id(rte_get_master_lcore());
+ netdev->socket_id = sid < 0 ? SOCKET0 : sid;
+ netdev->port_id = port_no;
+ netdev->type = VHOST;
+
+ netdev->flags = 0;
+ netdev->mtu = ETHER_MTU;
+ netdev->max_packet_len = MTU_TO_MAX_LEN(netdev->mtu);
+
+ ovsrcu_set(&netdev->virtio_dev, NULL);
+ netdev->stats.rx_packets = 0;
+ netdev->stats.tx_packets = 0;
+
+ netdev->dpdk_mp = dpdk_mp_get(netdev->socket_id, netdev->mtu);
+ if (!netdev->dpdk_mp) {
+ err = ENOMEM;
+ goto unlock_dev;
+ }
+
+ netdev_->n_txq = NR_QUEUE;
+ netdev_->n_rxq = NR_QUEUE;
+
+ VLOG_INFO("%s is associated with VHOST port #%d\n", netdev_->name,
+ netdev->port_id);
+
+ list_push_back(&dpdk_list, &netdev->list_node);
+
+unlock_dev:
+ rte_spinlock_unlock(&netdev->spinlock);
+ rte_spinlock_unlock(&dpdk_spinlock);
+
+ return err;
+}
+
+static int
netdev_dpdk_construct(struct netdev *netdev)
{
unsigned int port_no;
@@ -561,9 +638,9 @@ netdev_dpdk_construct(struct netdev *netdev)
return err;
}
- ovs_mutex_lock(&dpdk_mutex);
+ rte_spinlock_lock(&dpdk_spinlock);
err = netdev_dpdk_init(netdev, port_no);
- ovs_mutex_unlock(&dpdk_mutex);
+ rte_spinlock_unlock(&dpdk_spinlock);
return err;
}
@@ -572,17 +649,32 @@ netdev_dpdk_destruct(struct netdev *netdev_)
{
struct netdev_dpdk *dev = netdev_dpdk_cast(netdev_);
- ovs_mutex_lock(&dev->mutex);
+ rte_spinlock_lock(&dev->spinlock);
rte_eth_dev_stop(dev->port_id);
- ovs_mutex_unlock(&dev->mutex);
+ rte_spinlock_unlock(&dev->spinlock);
- ovs_mutex_lock(&dpdk_mutex);
+ rte_spinlock_lock(&dpdk_spinlock);
rte_free(dev->tx_q);
list_remove(&dev->list_node);
dpdk_mp_put(dev->dpdk_mp);
- ovs_mutex_unlock(&dpdk_mutex);
+ rte_spinlock_unlock(&dpdk_spinlock);
+}
+
+static void
+netdev_dpdk_vhost_destruct(struct netdev *netdev_)
+{
+ struct netdev_dpdk *dev = netdev_dpdk_cast(netdev_);
- ovs_mutex_destroy(&dev->mutex);
+ /* Can't remove a port while a guest is attached to it. */
+ if (netdev_dpdk_get_virtio(dev) != NULL) {
+ VLOG_ERR("Can not remove port, vhost device still attached\n");
+ return;
+ }
+
+ rte_spinlock_lock(&dpdk_spinlock);
+ list_remove(&dev->list_node);
+ dpdk_mp_put(dev->dpdk_mp);
+ rte_spinlock_unlock(&dpdk_spinlock);
}
static void
@@ -598,11 +690,11 @@ netdev_dpdk_get_config(const struct netdev *netdev_, struct smap *args)
{
struct netdev_dpdk *dev = netdev_dpdk_cast(netdev_);
- ovs_mutex_lock(&dev->mutex);
+ rte_spinlock_lock(&dev->spinlock);
smap_add_format(args, "configured_rx_queues", "%d", netdev_->n_rxq);
smap_add_format(args, "configured_tx_queues", "%d", netdev_->n_txq);
- ovs_mutex_unlock(&dev->mutex);
+ rte_spinlock_unlock(&dev->spinlock);
return 0;
}
@@ -629,19 +721,42 @@ netdev_dpdk_set_multiq(struct netdev *netdev_, unsigned int n_txq,
return err;
}
- ovs_mutex_lock(&dpdk_mutex);
- ovs_mutex_lock(&netdev->mutex);
-
- rte_eth_dev_stop(netdev->port_id);
+ rte_spinlock_lock(&dpdk_spinlock);
+ rte_spinlock_lock(&netdev->spinlock);
netdev->up.n_txq = n_txq;
netdev->up.n_rxq = n_rxq;
+
+ rte_eth_dev_stop(netdev->port_id);
rte_free(netdev->tx_q);
netdev_dpdk_alloc_txq(netdev, n_txq);
err = dpdk_eth_dev_init(netdev);
- ovs_mutex_unlock(&netdev->mutex);
- ovs_mutex_unlock(&dpdk_mutex);
+ rte_spinlock_unlock(&netdev->spinlock);
+ rte_spinlock_unlock(&dpdk_spinlock);
+
+ return err;
+}
+
+static int
+netdev_dpdk_vhost_set_multiq(struct netdev *netdev_, unsigned int n_txq,
+ unsigned int n_rxq)
+{
+ struct netdev_dpdk *netdev = netdev_dpdk_cast(netdev_);
+ int err = 0;
+
+ if (netdev->up.n_txq == n_txq && netdev->up.n_rxq == n_rxq) {
+ return err;
+ }
+
+ rte_spinlock_lock(&dpdk_spinlock);
+ rte_spinlock_lock(&netdev->spinlock);
+
+ netdev->up.n_txq = n_txq;
+ netdev->up.n_rxq = n_rxq;
+
+ rte_spinlock_unlock(&netdev->spinlock);
+ rte_spinlock_unlock(&dpdk_spinlock);
return err;
}
@@ -666,9 +781,9 @@ netdev_dpdk_rxq_construct(struct netdev_rxq *rxq_)
struct netdev_rxq_dpdk *rx = netdev_rxq_dpdk_cast(rxq_);
struct netdev_dpdk *netdev = netdev_dpdk_cast(rx->up.netdev);
- ovs_mutex_lock(&netdev->mutex);
+ rte_spinlock_lock(&netdev->spinlock);
rx->port_id = netdev->port_id;
- ovs_mutex_unlock(&netdev->mutex);
+ rte_spinlock_unlock(&netdev->spinlock);
return 0;
}
@@ -712,9 +827,9 @@ dpdk_queue_flush__(struct netdev_dpdk *dev, int qid)
for (i = nb_tx; i < txq->count; i++) {
rte_pktmbuf_free_seg(txq->burst_pkts[i]);
}
- ovs_mutex_lock(&dev->mutex);
+ rte_spinlock_lock(&dev->spinlock);
dev->stats.tx_dropped += txq->count-nb_tx;
- ovs_mutex_unlock(&dev->mutex);
+ rte_spinlock_unlock(&dev->spinlock);
}
txq->count = 0;
@@ -732,6 +847,36 @@ dpdk_queue_flush(struct netdev_dpdk *dev, int qid)
dpdk_queue_flush__(dev, qid);
}
+/*
+ * The receive path for the vhost port is the TX path out from guest.
+ */
+static int
+netdev_dpdk_vhost_rxq_recv(struct netdev_rxq *rxq_,
+ struct dpif_packet **packets, int *c)
+{
+ struct netdev_rxq_dpdk *rx = netdev_rxq_dpdk_cast(rxq_);
+ struct netdev *netdev = rx->up.netdev;
+ struct netdev_dpdk *vhost_dev = netdev_dpdk_cast(netdev);
+ struct virtio_net *virtio_dev = netdev_dpdk_get_virtio(vhost_dev);
+ int qid = 1;
+ uint16_t nb_rx = 0;
+
+ if (virtio_dev != NULL && (virtio_dev->flags & VIRTIO_DEV_RUNNING)) {
+ nb_rx = rte_vhost_dequeue_burst(virtio_dev, qid,
+ vhost_dev->dpdk_mp->mp, (struct rte_mbuf **)packets,
+ MAX_PKT_BURST);
+ if (!nb_rx) {
+ return EAGAIN;
+ }
+
+ vhost_dev->stats.rx_packets += (uint64_t)nb_rx;
+ }
+
+ *c = (int) nb_rx;
+
+ return 0;
+ }
+
static int
netdev_dpdk_rxq_recv(struct netdev_rxq *rxq_, struct dpif_packet **packets,
int *c)
@@ -760,6 +905,44 @@ netdev_dpdk_rxq_recv(struct netdev_rxq *rxq_, struct dpif_packet **packets,
return 0;
}
+static int
+netdev_dpdk_vhost_send(struct netdev *netdev, int qid OVS_UNUSED, struct dpif_packet **pkts,
+ int cnt, bool may_steal)
+{
+ int i;
+ int tx_pkts;
+
+ struct netdev_dpdk *vhost_dev = netdev_dpdk_cast(netdev);
+ struct virtio_net *virtio_dev = netdev_dpdk_get_virtio(vhost_dev);
+
+ if (virtio_dev == NULL || !(virtio_dev->flags & VIRTIO_DEV_RUNNING)) {
+ VLOG_WARN_RL(&rl, "virtio_dev not added yet");
+
+ rte_spinlock_lock(&vhost_dev->spinlock);
+ vhost_dev->stats.tx_dropped+= cnt;
+ rte_spinlock_unlock(&vhost_dev->spinlock);
+ goto out;
+ }
+
+ /* Hardcoded until mq support */
+ qid = VIRTIO_RXQ;
+
+ tx_pkts = rte_vhost_enqueue_burst(virtio_dev, qid,
+ (struct rte_mbuf **)pkts, cnt);
+
+ vhost_dev->stats.tx_packets += tx_pkts;
+ vhost_dev->stats.tx_dropped += (cnt - tx_pkts);
+
+out:
+ if (may_steal) {
+ for (i = 0; i < cnt; i++) {
+ dpif_packet_delete(pkts[i]);
+ }
+ }
+
+ return 0;
+}
+
inline static void
dpdk_queue_pkts(struct netdev_dpdk *dev, int qid,
struct rte_mbuf **pkts, int cnt)
@@ -801,11 +984,11 @@ dpdk_do_tx_copy(struct netdev *netdev, int qid, struct dpif_packet ** pkts,
int newcnt = 0;
int i;
- /* If we are on a non pmd thread we have to use the mempool mutex, because
+ /* If we are on a non pmd thread we have to use the mempool spinlock, because
* every non pmd thread shares the same mempool cache */
if (!thread_is_pmd()) {
- ovs_mutex_lock(&nonpmd_mempool_mutex);
+ rte_spinlock_lock(&nonpmd_mempool_spinlock);
}
for (i = 0; i < cnt; i++) {
@@ -837,16 +1020,16 @@ dpdk_do_tx_copy(struct netdev *netdev, int qid, struct dpif_packet ** pkts,
}
if (OVS_UNLIKELY(dropped)) {
- ovs_mutex_lock(&dev->mutex);
+ rte_spinlock_lock(&dev->spinlock);
dev->stats.tx_dropped += dropped;
- ovs_mutex_unlock(&dev->mutex);
+ rte_spinlock_unlock(&dev->spinlock);
}
dpdk_queue_pkts(dev, qid, mbufs, newcnt);
dpdk_queue_flush(dev, qid);
if (!thread_is_pmd()) {
- ovs_mutex_unlock(&nonpmd_mempool_mutex);
+ rte_spinlock_unlock(&nonpmd_mempool_spinlock);
}
}
@@ -895,9 +1078,9 @@ netdev_dpdk_send__(struct netdev_dpdk *dev, int qid,
}
if (OVS_UNLIKELY(dropped)) {
- ovs_mutex_lock(&dev->mutex);
+ rte_spinlock_lock(&dev->spinlock);
dev->stats.tx_dropped += dropped;
- ovs_mutex_unlock(&dev->mutex);
+ rte_spinlock_unlock(&dev->spinlock);
}
}
}
@@ -918,12 +1101,12 @@ netdev_dpdk_set_etheraddr(struct netdev *netdev,
{
struct netdev_dpdk *dev = netdev_dpdk_cast(netdev);
- ovs_mutex_lock(&dev->mutex);
+ rte_spinlock_lock(&dev->spinlock);
if (!eth_addr_equals(dev->hwaddr, mac)) {
memcpy(dev->hwaddr, mac, ETH_ADDR_LEN);
netdev_change_seq_changed(netdev);
}
- ovs_mutex_unlock(&dev->mutex);
+ rte_spinlock_unlock(&dev->spinlock);
return 0;
}
@@ -934,9 +1117,9 @@ netdev_dpdk_get_etheraddr(const struct netdev *netdev,
{
struct netdev_dpdk *dev = netdev_dpdk_cast(netdev);
- ovs_mutex_lock(&dev->mutex);
+ rte_spinlock_lock(&dev->spinlock);
memcpy(mac, dev->hwaddr, ETH_ADDR_LEN);
- ovs_mutex_unlock(&dev->mutex);
+ rte_spinlock_unlock(&dev->spinlock);
return 0;
}
@@ -946,9 +1129,9 @@ netdev_dpdk_get_mtu(const struct netdev *netdev, int *mtup)
{
struct netdev_dpdk *dev = netdev_dpdk_cast(netdev);
- ovs_mutex_lock(&dev->mutex);
+ rte_spinlock_lock(&dev->spinlock);
*mtup = dev->mtu;
- ovs_mutex_unlock(&dev->mutex);
+ rte_spinlock_unlock(&dev->spinlock);
return 0;
}
@@ -961,8 +1144,8 @@ netdev_dpdk_set_mtu(const struct netdev *netdev, int mtu)
struct dpdk_mp *old_mp;
struct dpdk_mp *mp;
- ovs_mutex_lock(&dpdk_mutex);
- ovs_mutex_lock(&dev->mutex);
+ rte_spinlock_lock(&dpdk_spinlock);
+ rte_spinlock_lock(&dev->spinlock);
if (dev->mtu == mtu) {
err = 0;
goto out;
@@ -995,8 +1178,8 @@ netdev_dpdk_set_mtu(const struct netdev *netdev, int mtu)
dpdk_mp_put(old_mp);
netdev_change_seq_changed(netdev);
out:
- ovs_mutex_unlock(&dev->mutex);
- ovs_mutex_unlock(&dpdk_mutex);
+ rte_spinlock_unlock(&dev->spinlock);
+ rte_spinlock_unlock(&dpdk_spinlock);
return err;
}
@@ -1004,6 +1187,44 @@ static int
netdev_dpdk_get_carrier(const struct netdev *netdev_, bool *carrier);
static int
+netdev_dpdk_vhost_get_stats(const struct netdev *netdev,
+ struct netdev_stats *stats)
+{
+ struct netdev_dpdk *dev = netdev_dpdk_cast(netdev);
+
+ rte_spinlock_lock(&dev->spinlock);
+ memset(stats, 0, sizeof(*stats));
+ /* Unsupported Stats */
+ stats->rx_errors = UINT64_MAX;
+ stats->tx_errors = UINT64_MAX;
+ stats->multicast = UINT64_MAX;
+ stats->collisions = UINT64_MAX;
+ stats->rx_crc_errors = UINT64_MAX;
+ stats->rx_fifo_errors = UINT64_MAX;
+ stats->rx_frame_errors = UINT64_MAX;
+ stats->rx_length_errors = UINT64_MAX;
+ stats->rx_missed_errors= UINT64_MAX;
+ stats->rx_over_errors= UINT64_MAX;
+ stats->tx_aborted_errors= UINT64_MAX;
+ stats->tx_carrier_errors= UINT64_MAX;
+ stats->tx_errors= UINT64_MAX;
+ stats->tx_fifo_errors= UINT64_MAX;
+ stats->tx_heartbeat_errors= UINT64_MAX;
+ stats->tx_window_errors= UINT64_MAX;
+ stats->rx_bytes += UINT64_MAX;
+ stats->rx_dropped += UINT64_MAX;
+ stats->tx_bytes += UINT64_MAX;
+
+ /* Supported Stats */
+ stats->rx_packets += dev->stats.rx_packets;
+ stats->tx_packets += dev->stats.tx_packets;
+ stats->tx_dropped += dev->stats.tx_dropped;
+ rte_spinlock_unlock(&dev->spinlock);
+
+ return 0;
+}
+
+static int
netdev_dpdk_get_stats(const struct netdev *netdev, struct netdev_stats *stats)
{
struct netdev_dpdk *dev = netdev_dpdk_cast(netdev);
@@ -1011,7 +1232,7 @@ netdev_dpdk_get_stats(const struct netdev *netdev, struct netdev_stats *stats)
bool gg;
netdev_dpdk_get_carrier(netdev, &gg);
- ovs_mutex_lock(&dev->mutex);
+ rte_spinlock_lock(&dev->spinlock);
rte_eth_stats_get(dev->port_id, &rte_stats);
memset(stats, 0, sizeof(*stats));
@@ -1025,7 +1246,7 @@ netdev_dpdk_get_stats(const struct netdev *netdev, struct netdev_stats *stats)
stats->multicast = rte_stats.imcasts;
stats->tx_dropped = dev->stats.tx_dropped;
- ovs_mutex_unlock(&dev->mutex);
+ rte_spinlock_unlock(&dev->spinlock);
return 0;
}
@@ -1040,9 +1261,9 @@ netdev_dpdk_get_features(const struct netdev *netdev_,
struct netdev_dpdk *dev = netdev_dpdk_cast(netdev_);
struct rte_eth_link link;
- ovs_mutex_lock(&dev->mutex);
+ rte_spinlock_lock(&dev->spinlock);
link = dev->link;
- ovs_mutex_unlock(&dev->mutex);
+ rte_spinlock_unlock(&dev->spinlock);
if (link.link_duplex == ETH_LINK_AUTONEG_DUPLEX) {
if (link.link_speed == ETH_LINK_SPEED_AUTONEG) {
@@ -1082,9 +1303,9 @@ netdev_dpdk_get_ifindex(const struct netdev *netdev)
struct netdev_dpdk *dev = netdev_dpdk_cast(netdev);
int ifindex;
- ovs_mutex_lock(&dev->mutex);
+ rte_spinlock_lock(&dev->spinlock);
ifindex = dev->port_id;
- ovs_mutex_unlock(&dev->mutex);
+ rte_spinlock_unlock(&dev->spinlock);
return ifindex;
}
@@ -1094,10 +1315,31 @@ netdev_dpdk_get_carrier(const struct netdev *netdev_, bool *carrier)
{
struct netdev_dpdk *dev = netdev_dpdk_cast(netdev_);
- ovs_mutex_lock(&dev->mutex);
+ rte_spinlock_lock(&dev->spinlock);
+
check_link_status(dev);
*carrier = dev->link.link_status;
- ovs_mutex_unlock(&dev->mutex);
+
+ rte_spinlock_unlock(&dev->spinlock);
+
+ return 0;
+}
+
+static int
+netdev_dpdk_vhost_get_carrier(const struct netdev *netdev_, bool *carrier)
+{
+ struct netdev_dpdk *dev = netdev_dpdk_cast(netdev_);
+ struct virtio_net *virtio_dev = netdev_dpdk_get_virtio(dev);
+
+ rte_spinlock_lock(&dev->spinlock);
+
+ if (virtio_dev != NULL && (virtio_dev->flags & VIRTIO_DEV_RUNNING)) {
+ *carrier =1;
+ } else {
+ *carrier = 0 ;
+ }
+
+ rte_spinlock_unlock(&dev->spinlock);
return 0;
}
@@ -1108,9 +1350,9 @@ netdev_dpdk_get_carrier_resets(const struct netdev *netdev_)
struct netdev_dpdk *dev = netdev_dpdk_cast(netdev_);
long long int carrier_resets;
- ovs_mutex_lock(&dev->mutex);
+ rte_spinlock_lock(&dev->spinlock);
carrier_resets = dev->link_reset_cnt;
- ovs_mutex_unlock(&dev->mutex);
+ rte_spinlock_unlock(&dev->spinlock);
return carrier_resets;
}
@@ -1125,7 +1367,7 @@ netdev_dpdk_set_miimon(struct netdev *netdev_ OVS_UNUSED,
static int
netdev_dpdk_update_flags__(struct netdev_dpdk *dev,
enum netdev_flags off, enum netdev_flags on,
- enum netdev_flags *old_flagsp) OVS_REQUIRES(dev->mutex)
+ enum netdev_flags *old_flagsp) OVS_REQUIRES(dev->spinlock)
{
int err;
@@ -1141,18 +1383,20 @@ netdev_dpdk_update_flags__(struct netdev_dpdk *dev,
return 0;
}
- if (dev->flags & NETDEV_UP) {
- err = rte_eth_dev_start(dev->port_id);
- if (err)
- return -err;
- }
+ if (dev->type == DPDK) {
+ if (dev->flags & NETDEV_UP) {
+ err = rte_eth_dev_start(dev->port_id);
+ if (err)
+ return -err;
+ }
- if (dev->flags & NETDEV_PROMISC) {
- rte_eth_promiscuous_enable(dev->port_id);
- }
+ if (dev->flags & NETDEV_PROMISC) {
+ rte_eth_promiscuous_enable(dev->port_id);
+ }
- if (!(dev->flags & NETDEV_UP)) {
- rte_eth_dev_stop(dev->port_id);
+ if (!(dev->flags & NETDEV_UP)) {
+ rte_eth_dev_stop(dev->port_id);
+ }
}
return 0;
@@ -1166,9 +1410,9 @@ netdev_dpdk_update_flags(struct netdev *netdev_,
struct netdev_dpdk *netdev = netdev_dpdk_cast(netdev_);
int error;
- ovs_mutex_lock(&netdev->mutex);
+ rte_spinlock_lock(&netdev->spinlock);
error = netdev_dpdk_update_flags__(netdev, off, on, old_flagsp);
- ovs_mutex_unlock(&netdev->mutex);
+ rte_spinlock_unlock(&netdev->spinlock);
return error;
}
@@ -1182,9 +1426,9 @@ netdev_dpdk_get_status(const struct netdev *netdev_, struct smap *args)
if (dev->port_id < 0)
return ENODEV;
- ovs_mutex_lock(&dev->mutex);
+ rte_spinlock_lock(&dev->spinlock);
rte_eth_dev_info_get(dev->port_id, &dev_info);
- ovs_mutex_unlock(&dev->mutex);
+ rte_spinlock_unlock(&dev->spinlock);
smap_add_format(args, "driver_name", "%s", dev_info.driver_name);
@@ -1208,7 +1452,7 @@ netdev_dpdk_get_status(const struct netdev *netdev_, struct smap *args)
static void
netdev_dpdk_set_admin_state__(struct netdev_dpdk *dev, bool admin_state)
- OVS_REQUIRES(dev->mutex)
+ OVS_REQUIRES(dev->spinlock)
{
enum netdev_flags old_flags;
@@ -1239,9 +1483,9 @@ netdev_dpdk_set_admin_state(struct unixctl_conn *conn, int argc,
if (netdev && is_dpdk_class(netdev->netdev_class)) {
struct netdev_dpdk *dpdk_dev = netdev_dpdk_cast(netdev);
- ovs_mutex_lock(&dpdk_dev->mutex);
+ rte_spinlock_lock(&dpdk_dev->spinlock);
netdev_dpdk_set_admin_state__(dpdk_dev, up);
- ovs_mutex_unlock(&dpdk_dev->mutex);
+ rte_spinlock_unlock(&dpdk_dev->spinlock);
netdev_close(netdev);
} else {
@@ -1252,17 +1496,149 @@ netdev_dpdk_set_admin_state(struct unixctl_conn *conn, int argc,
} else {
struct netdev_dpdk *netdev;
- ovs_mutex_lock(&dpdk_mutex);
+ rte_spinlock_lock(&dpdk_spinlock);
LIST_FOR_EACH (netdev, list_node, &dpdk_list) {
- ovs_mutex_lock(&netdev->mutex);
+ rte_spinlock_lock(&netdev->spinlock);
netdev_dpdk_set_admin_state__(netdev, up);
- ovs_mutex_unlock(&netdev->mutex);
+ rte_spinlock_unlock(&netdev->spinlock);
}
- ovs_mutex_unlock(&dpdk_mutex);
+ rte_spinlock_unlock(&dpdk_spinlock);
}
unixctl_command_reply(conn, "OK");
}
+/*
+ * Set virtqueue flags so that we do not receive interrupts.
+ */
+static void
+set_irq_status(struct virtio_net *dev)
+{
+ dev->virtqueue[VIRTIO_RXQ]->used->flags = VRING_USED_F_NO_NOTIFY;
+ dev->virtqueue[VIRTIO_TXQ]->used->flags = VRING_USED_F_NO_NOTIFY;
+}
+
+/*
+ * A new virtio-net device is added to a vhost port.
+ */
+static int
+new_device(struct virtio_net *dev)
+{
+ struct netdev_dpdk *netdev;
+ bool exists = false;
+
+ /* Disable notifications. */
+ set_irq_status(dev);
+ dev->flags |= VIRTIO_DEV_RUNNING;
+
+ rte_spinlock_lock(&dpdk_spinlock);
+ /* Add device to the vhost port with the same name as that passed down. */
+ LIST_FOR_EACH(netdev, list_node, &dpdk_list) {
+ if (strncmp(dev->ifname, netdev->up.name, IFNAMSIZ) == 0) {
+ rte_spinlock_lock(&netdev->spinlock);
+ ovsrcu_set(&netdev->virtio_dev, dev);
+ rte_spinlock_unlock(&netdev->spinlock);
+ exists = true;
+ break;
+ }
+ }
+ rte_spinlock_unlock(&dpdk_spinlock);
+
+ if (!exists) {
+ VLOG_ERR("(%ld) VHOST Device can't be added - name not found \n",
+ dev->device_fh);
+ return -1;
+ }
+
+ VLOG_INFO("(%ld) VHOST Device has been added \n", dev->device_fh);
+
+ return 0;
+}
+
+/*
+ * Remove a virtio-net device from the specific vhost port. Use dev->remove
+ * flag to stop any more packets from being sent or received to/from a VM and
+ * ensure all currently queued packets have been sent/received before removing
+ * the device.
+ */
+static void
+destroy_device(volatile struct virtio_net *dev)
+{
+ struct netdev_dpdk *vhost_dev;
+ dev->flags &= ~VIRTIO_DEV_RUNNING;
+
+ rte_spinlock_lock(&dpdk_spinlock);
+ LIST_FOR_EACH (vhost_dev, list_node, &dpdk_list) {
+ if (netdev_dpdk_get_virtio(vhost_dev) == dev) {
+
+ /*
+ * Wait for other threads to quiesce before
+ * setting the virtio_dev to NULL.
+ */
+ ovsrcu_synchronize();
+
+ rte_spinlock_lock(&vhost_dev->spinlock);
+ ovsrcu_set(&vhost_dev->virtio_dev, NULL);
+ rte_spinlock_unlock(&vhost_dev->spinlock);
+
+ ovsrcu_quiesce_start();
+ }
+ }
+ rte_spinlock_unlock(&dpdk_spinlock);
+
+ VLOG_INFO("%ld Vhost Device has been removed\n", dev->device_fh);
+}
+
+struct virtio_net *
+netdev_dpdk_get_virtio(const struct netdev_dpdk *dev)
+{
+ return ovsrcu_get(struct virtio_net *, &dev->virtio_dev);
+}
+
+
+/*
+ * These callbacks allow virtio-net devices to be added to vhost ports when
+ * configuration has been fully complete.
+ */
+const struct virtio_net_device_ops virtio_net_device_ops =
+{
+ .new_device = new_device,
+ .destroy_device = destroy_device,
+};
+
+static int
+dpdk_vhost_class_init(void)
+{
+ int err = -1;
+
+ rte_vhost_driver_callback_register(&virtio_net_device_ops);
+
+ /* Register CUSE device to handle IOCTLs.
+ * Unless otherwise specified on the vswitchd command line, dev_basename
+ * is set to vhost-net.
+ */
+ err = rte_vhost_driver_register(dev_basename);
+
+ if (err != 0) {
+ VLOG_ERR("CUSE device setup failure.\n");
+ return -1;
+ }
+
+ ovs_thread_create("cuse_thread", start_cuse_session_loop, NULL);
+
+ return 0;
+}
+
+void *
+start_cuse_session_loop(void *dummy OVS_UNUSED)
+{
+ pthread_detach(pthread_self());
+
+ ovsrcu_quiesce_start();
+ rte_vhost_driver_session_start();
+
+ return NULL;
+}
+
static void
dpdk_common_init(void)
{
@@ -1332,7 +1708,7 @@ dpdk_ring_create(const char dev_name[], unsigned int port_no,
}
static int
-dpdk_ring_open(const char dev_name[], unsigned int *eth_port_id) OVS_REQUIRES(dpdk_mutex)
+dpdk_ring_open(const char dev_name[], unsigned int *eth_port_id) OVS_REQUIRES(dpdk_spinlock)
{
struct dpdk_ring *ivshmem;
unsigned int port_no;
@@ -1379,7 +1755,7 @@ netdev_dpdk_ring_construct(struct netdev *netdev)
return rte_eal_init_ret;
}
- ovs_mutex_lock(&dpdk_mutex);
+ rte_spinlock_lock(&dpdk_spinlock);
err = dpdk_ring_open(netdev->name, &port_no);
if (err) {
@@ -1389,11 +1765,12 @@ netdev_dpdk_ring_construct(struct netdev *netdev)
err = netdev_dpdk_init(netdev, port_no);
unlock_dpdk:
- ovs_mutex_unlock(&dpdk_mutex);
+ rte_spinlock_unlock(&dpdk_spinlock);
return err;
}
-#define NETDEV_DPDK_CLASS(NAME, INIT, CONSTRUCT, MULTIQ, SEND) \
+#define NETDEV_DPDK_CLASS(NAME, INIT, CONSTRUCT, DESTRUCT, MULTIQ, SEND, \
+ GET_CARRIER, GET_STATS, GET_FEATURES, GET_STATUS, RXQ_RECV) \
{ \
NAME, \
INIT, /* init */ \
@@ -1402,14 +1779,14 @@ unlock_dpdk:
\
netdev_dpdk_alloc, \
CONSTRUCT, \
- netdev_dpdk_destruct, \
+ DESTRUCT, \
netdev_dpdk_dealloc, \
netdev_dpdk_get_config, \
NULL, /* netdev_dpdk_set_config */ \
NULL, /* get_tunnel_config */ \
- NULL, /* build header */ \
- NULL, /* push header */ \
- NULL, /* pop header */ \
+ NULL, /* build header */ \
+ NULL, /* push header */ \
+ NULL, /* pop header */ \
netdev_dpdk_get_numa_id, /* get_numa_id */ \
MULTIQ, /* set_multiq */ \
\
@@ -1421,11 +1798,11 @@ unlock_dpdk:
netdev_dpdk_get_mtu, \
netdev_dpdk_set_mtu, \
netdev_dpdk_get_ifindex, \
- netdev_dpdk_get_carrier, \
+ GET_CARRIER, \
netdev_dpdk_get_carrier_resets, \
netdev_dpdk_set_miimon, \
- netdev_dpdk_get_stats, \
- netdev_dpdk_get_features, \
+ GET_STATS, \
+ GET_FEATURES, \
NULL, /* set_advertisements */ \
\
NULL, /* set_policing */ \
@@ -1447,7 +1824,7 @@ unlock_dpdk:
NULL, /* get_in6 */ \
NULL, /* add_router */ \
NULL, /* get_next_hop */ \
- netdev_dpdk_get_status, \
+ GET_STATUS, \
NULL, /* arp_lookup */ \
\
netdev_dpdk_update_flags, \
@@ -1456,7 +1833,7 @@ unlock_dpdk:
netdev_dpdk_rxq_construct, \
netdev_dpdk_rxq_destruct, \
netdev_dpdk_rxq_dealloc, \
- netdev_dpdk_rxq_recv, \
+ RXQ_RECV, \
NULL, /* rx_wait */ \
NULL, /* rxq_drain */ \
}
@@ -1465,16 +1842,43 @@ int
dpdk_init(int argc, char **argv)
{
int result;
+ int base = 0;
if (argc < 2 || strcmp(argv[1], "--dpdk"))
return 0;
- /* Make sure program name passed to rte_eal_init() is vswitchd. */
+ /* Ignore the --dpdk argument, but keep the program name argument as this
+ * is needed later for a call to rte_eal_init()
+ */
argv[1] = argv[0];
argc--;
argv++;
+ /* If the basename parameter has been provided, set 'dev_basename' to
+ * this string if it meets the correct criteria. Otherwise, set it to the
+ * default (vhost-net).
+ */
+
+ if (strcmp(argv[1], "--basename")==0 &&
+ (sizeof(argv[2]) <= MAX_BASENAME_SZ)) {
+ dev_basename = argv[2];
+
+ /* Remove the basename configuration parameters from the argument
+ * list, so that the correct elements are passed to the DPDK
+ * initialization function (including the program name as the first).
+ */
+ argv[2] = argv[0];
+ argc -= 2;
+ argv += 2; /* Increment by two to bypass the basename arguments */
+ base = 2;
+
+ VLOG_INFO("User-provided basename in use: /dev/%s\n", dev_basename);
+ } else {
+ dev_basename = "vhost-net";
+ VLOG_INFO("No basename provided - defaulting to /dev/vhost-net\n");
+ }
+
/* Make sure things are initialized ... */
result = rte_eal_init(argc, argv);
if (result < 0) {
@@ -1491,7 +1895,7 @@ dpdk_init(int argc, char **argv)
/* We are called from the main thread here */
thread_set_nonpmd();
- return result + 1;
+ return result + 1 + base;
}
const struct netdev_class dpdk_class =
@@ -1499,16 +1903,42 @@ const struct netdev_class dpdk_class =
"dpdk",
NULL,
netdev_dpdk_construct,
+ netdev_dpdk_destruct,
netdev_dpdk_set_multiq,
- netdev_dpdk_eth_send);
+ netdev_dpdk_eth_send,
+ netdev_dpdk_get_carrier,
+ netdev_dpdk_get_stats,
+ netdev_dpdk_get_features,
+ netdev_dpdk_get_status,
+ netdev_dpdk_rxq_recv);
const struct netdev_class dpdk_ring_class =
NETDEV_DPDK_CLASS(
"dpdkr",
NULL,
netdev_dpdk_ring_construct,
+ netdev_dpdk_destruct,
+ NULL,
+ netdev_dpdk_ring_send,
+ netdev_dpdk_get_carrier,
+ netdev_dpdk_get_stats,
+ netdev_dpdk_get_features,
+ netdev_dpdk_get_status,
+ netdev_dpdk_rxq_recv);
+
+const struct netdev_class dpdk_vhost_class =
+ NETDEV_DPDK_CLASS(
+ "dpdkvhost",
+ dpdk_vhost_class_init,
+ netdev_dpdk_vhost_construct,
+ netdev_dpdk_vhost_destruct,
+ netdev_dpdk_vhost_set_multiq,
+ netdev_dpdk_vhost_send,
+ netdev_dpdk_vhost_get_carrier,
+ netdev_dpdk_vhost_get_stats,
+ NULL,
NULL,
- netdev_dpdk_ring_send);
+ netdev_dpdk_vhost_rxq_recv);
void
netdev_dpdk_register(void)
@@ -1523,6 +1953,7 @@ netdev_dpdk_register(void)
dpdk_common_init();
netdev_register_provider(&dpdk_class);
netdev_register_provider(&dpdk_ring_class);
+ netdev_register_provider(&dpdk_vhost_class);
ovsthread_once_done(&once);
}
}
diff --git a/lib/netdev.c b/lib/netdev.c
index 2bda77f..48d77bd 100644
--- a/lib/netdev.c
+++ b/lib/netdev.c
@@ -108,7 +108,8 @@ bool
netdev_is_pmd(const struct netdev *netdev)
{
return (!strcmp(netdev->netdev_class->type, "dpdk") ||
- !strcmp(netdev->netdev_class->type, "dpdkr"));
+ !strcmp(netdev->netdev_class->type, "dpdkr") ||
+ !strcmp(netdev->netdev_class->type, "dpdkvhost"));
}
static void
diff --git a/utilities/automake.mk b/utilities/automake.mk
index 834be58..2d4e0cc 100644
--- a/utilities/automake.mk
+++ b/utilities/automake.mk
@@ -41,7 +41,8 @@ EXTRA_DIST += \
utilities/ovs-save \
utilities/ovs-tcpundump.in \
utilities/ovs-test.in \
- utilities/ovs-vlan-test.in
+ utilities/ovs-vlan-test.in \
+ utilities/qemu-wrap.py
MAN_ROOTS += \
utilities/ovs-appctl.8.in \
utilities/ovs-benchmark.1.in \
diff --git a/utilities/qemu-wrap.py b/utilities/qemu-wrap.py
new file mode 100755
index 0000000..5cee849
--- /dev/null
+++ b/utilities/qemu-wrap.py
@@ -0,0 +1,389 @@
+#!/usr/bin/python
+#
+# BSD LICENSE
+#
+# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+#
+
+#####################################################################
+# This script is designed to modify the call to the QEMU emulator
+# to support userspace vhost when starting a guest machine through
+# libvirt with vhost enabled. The steps to enable this are as follows
+# and should be run as root:
+#
+# 1. Place this script in a libvirtd's binary search PATH ($PATH)
+# A good location would be in the same directory that the QEMU
+# binary is located
+#
+# 2. Ensure that the script has the same owner/group and file
+# permissions as the QEMU binary
+#
+# 3. Update the VM xml file using "virsh edit VM.xml"
+#
+# 3.a) Set the VM to use the launch script
+#
+# Set the emulator path contained in the
+# <emulator><emulator/> tags
+#
+# e.g replace <emulator>/usr/bin/qemu-kvm<emulator/>
+# with <emulator>/usr/bin/qemu-wrap.py<emulator/>
+#
+# 3.b) Set the VM's device's to use vhost-net offload
+#
+# <interface type="network">
+# <model type="virtio"/>
+# <driver name="vhost"/>
+# <interface/>
+#
+# 4. Enable libvirt to access our userpace device file by adding it to
+# controllers cgroup for libvirtd using the following steps
+#
+# 4.a) In /etc/libvirt/qemu.conf add/edit the following lines:
+# 1) cgroup_controllers = [ ... "devices", ... ]
+# 2) clear_emulator_capabilities = 0
+# 3) user = "root"
+# 4) group = "root"
+# 5) cgroup_device_acl = [
+# "/dev/null", "/dev/full", "/dev/zero",
+# "/dev/random", "/dev/urandom",
+# "/dev/ptmx", "/dev/kvm", "/dev/kqemu",
+# "/dev/rtc", "/dev/hpet", "/dev/net/tun",
+# "/dev/<devbase-name>-<index>",
+# "/dev/hugepages"
+# ]
+#
+# 4.b) Disable SELinux or set to permissive mode
+#
+# 4.c) Mount cgroup device controller
+# "mkdir /dev/cgroup"
+# "mount -t cgroup none /dev/cgroup -o devices"
+#
+# 4.d) Set hugetlbfs_mount variable - ( Optional )
+# VMs using userspace vhost must use hugepage backed
+# memory. This can be enabled in the libvirt XML
+# config by adding a memory backing section to the
+# XML config e.g.
+# <memoryBacking>
+# <hugepages/>
+# </memoryBacking>
+# This memory backing section should be added after the
+# <memory> and <currentMemory> sections. This will add
+# flags "-mem-prealloc -mem-path <path>" to the QEMU
+# command line. The hugetlbfs_mount variable can be used
+# to override the default <path> passed through by libvirt.
+#
+# if "-mem-prealloc" or "-mem-path <path>" are not passed
+# through and a vhost device is detected then these options will
+# be automatically added by this script. This script will detect
+# the system hugetlbfs mount point to be used for <path>. The
+# default <path> for this script can be overidden by the
+# hugetlbfs_dir variable in the configuration section of this script.
+#
+#
+# 4.e) Restart the libvirtd system process
+# e.g. on Fedora "systemctl restart libvirtd.service"
+#
+#
+# 4.f) Edit the Configuration Parameters section of this script
+# to point to the correct emulator location and set any
+# addition options
+#
+# The script modifies the libvirtd Qemu call by modifying/adding
+# options based on the configuration parameters below.
+# NOTE:
+# emul_path and us_vhost_path must be set
+# All other parameters are optional
+#####################################################################
+
+
+#############################################
+# Configuration Parameters
+#############################################
+#Path to QEMU binary
+emul_path = "/usr/local/bin/qemu-system-x86_64"
+
+#Path to userspace vhost device file
+# This filename should match the --dev-basename --dev-index parameters of
+# the command used to launch the userspace vhost sample application e.g.
+# if the sample app lauch command is:
+# ./build/vhost-switch ..... --dev-basename usvhost --dev-index 1
+# then this variable should be set to:
+# us_vhost_path = "/dev/usvhost-1"
+us_vhost_path = "/dev/usvhost-1"
+
+#List of additional user defined emulation options. These options will
+#be added to all Qemu calls
+emul_opts_user = []
+
+#List of additional user defined emulation options for vhost only.
+#These options will only be added to vhost enabled guests
+emul_opts_user_vhost = []
+
+#For all VHOST enabled VMs, the VM memory is preallocated from hugetlbfs
+# Set this variable to one to enable this option for all VMs
+use_huge_all = 0
+
+#Instead of autodetecting, override the hugetlbfs directory by setting
+#this variable
+hugetlbfs_dir = ""
+
+#############################################
+
+
+#############################################
+# ****** Do Not Modify Below this Line ******
+#############################################
+
+import sys, os, subprocess
+import time
+import signal
+
+
+#List of open userspace vhost file descriptors
+fd_list = []
+
+#additional virtio device flags when using userspace vhost
+vhost_flags = [ "csum=off",
+ "gso=off",
+ "guest_tso4=off",
+ "guest_tso6=off",
+ "guest_ecn=off"
+ ]
+
+#String of the path to the Qemu process pid
+qemu_pid = "/tmp/%d-qemu.pid" % os.getpid()
+
+#############################################
+# Signal haldler to kill Qemu subprocess
+#############################################
+def kill_qemu_process(signum, stack):
+ pidfile = open(qemu_pid, 'r')
+ pid = int(pidfile.read())
+ os.killpg(pid, signal.SIGTERM)
+ pidfile.close()
+
+
+#############################################
+# Find the system hugefile mount point.
+# Note:
+# if multiple hugetlbfs mount points exist
+# then the first one found will be used
+#############################################
+def find_huge_mount():
+
+ if (len(hugetlbfs_dir)):
+ return hugetlbfs_dir
+
+ huge_mount = ""
+
+ if (os.access("/proc/mounts", os.F_OK)):
+ f = open("/proc/mounts", "r")
+ line = f.readline()
+ while line:
+ line_split = line.split(" ")
+ if line_split[2] == 'hugetlbfs':
+ huge_mount = line_split[1]
+ break
+ line = f.readline()
+ else:
+ print "/proc/mounts not found"
+ exit (1)
+
+ f.close
+ if len(huge_mount) == 0:
+ print "Failed to find hugetlbfs mount point"
+ exit (1)
+
+ return huge_mount
+
+
+#############################################
+# Get a userspace Vhost file descriptor
+#############################################
+def get_vhost_fd():
+
+ if (os.access(us_vhost_path, os.F_OK)):
+ fd = os.open( us_vhost_path, os.O_RDWR)
+ else:
+ print ("US-Vhost file %s not found" %us_vhost_path)
+ exit (1)
+
+ return fd
+
+
+#############################################
+# Check for vhostfd. if found then replace
+# with our own vhost fd and append any vhost
+# flags onto the end
+#############################################
+def modify_netdev_arg(arg):
+
+ global fd_list
+ vhost_in_use = 0
+ s = ''
+ new_opts = []
+ netdev_opts = arg.split(",")
+
+ for opt in netdev_opts:
+ #check if vhost is used
+ if "vhost" == opt[:5]:
+ vhost_in_use = 1
+ else:
+ new_opts.append(opt)
+
+ #if using vhost append vhost options
+ if vhost_in_use == 1:
+ #append vhost on option
+ new_opts.append('vhost=on')
+ #append vhostfd ption
+ new_fd = get_vhost_fd()
+ new_opts.append('vhostfd=' + str(new_fd))
+ fd_list.append(new_fd)
+
+ #concatenate all options
+ for opt in new_opts:
+ if len(s) > 0:
+ s+=','
+
+ s+=opt
+
+ return s
+
+
+#############################################
+# Main
+#############################################
+def main():
+
+ global fd_list
+ global vhost_in_use
+ new_args = []
+ num_cmd_args = len(sys.argv)
+ emul_call = ''
+ mem_prealloc_set = 0
+ mem_path_set = 0
+ num = 0;
+
+ #parse the parameters
+ while (num < num_cmd_args):
+ arg = sys.argv[num]
+
+ #Check netdev +1 parameter for vhostfd
+ if arg == '-netdev':
+ num_vhost_devs = len(fd_list)
+ new_args.append(arg)
+
+ num+=1
+ arg = sys.argv[num]
+ mod_arg = modify_netdev_arg(arg)
+ new_args.append(mod_arg)
+
+ #append vhost flags if this is a vhost device
+ # and -device is the next arg
+ # i.e -device -opt1,-opt2,...,-opt3,%vhost
+ if (num_vhost_devs < len(fd_list)):
+ num+=1
+ arg = sys.argv[num]
+ if arg == '-device':
+ new_args.append(arg)
+ num+=1
+ new_arg = sys.argv[num]
+ for flag in vhost_flags:
+ new_arg = ''.join([new_arg,',',flag])
+ new_args.append(new_arg)
+ else:
+ new_args.append(arg)
+ elif arg == '-mem-prealloc':
+ mem_prealloc_set = 1
+ new_args.append(arg)
+ elif arg == '-mem-path':
+ mem_path_set = 1
+ new_args.append(arg)
+
+ else:
+ new_args.append(arg)
+
+ num+=1
+
+ #Set Qemu binary location
+ emul_call+=emul_path
+ emul_call+=" "
+
+ #Add prealloc mem options if using vhost and not already added
+ if ((len(fd_list) > 0) and (mem_prealloc_set == 0)):
+ emul_call += "-mem-prealloc "
+
+ #Add mempath mem options if using vhost and not already added
+ if ((len(fd_list) > 0) and (mem_path_set == 0)):
+ #Detect and add hugetlbfs mount point
+ mp = find_huge_mount()
+ mp = "".join(["-mem-path ", mp])
+ emul_call += mp
+ emul_call += " "
+
+ #add user options
+ for opt in emul_opts_user:
+ emul_call += opt
+ emul_call += " "
+
+ #Add add user vhost only options
+ if len(fd_list) > 0:
+ for opt in emul_opts_user_vhost:
+ emul_call += opt
+ emul_call += " "
+
+ #Add updated libvirt options
+ iter_args = iter(new_args)
+ #skip 1st arg i.e. call to this script
+ next(iter_args)
+ for arg in iter_args:
+ emul_call+=str(arg)
+ emul_call+= " "
+
+ emul_call += "-pidfile %s " % qemu_pid
+ #Call QEMU
+ process = subprocess.Popen(emul_call, shell=True, preexec_fn=os.setsid)
+
+ for sig in [signal.SIGTERM, signal.SIGINT, signal.SIGHUP, signal.SIGQUIT]:
+ signal.signal(sig, kill_qemu_process)
+
+ process.wait()
+
+ #Close usvhost files
+ for fd in fd_list:
+ os.close(fd)
+ #Cleanup temporary files
+ if os.access(qemu_pid, os.F_OK):
+ os.remove(qemu_pid)
+
+
+
+if __name__ == "__main__":
+ main()
diff --git a/vswitchd/ovs-vswitchd.c b/vswitchd/ovs-vswitchd.c
index 812c00b..822d53e 100644
--- a/vswitchd/ovs-vswitchd.c
+++ b/vswitchd/ovs-vswitchd.c
@@ -252,7 +252,9 @@ usage(void)
daemon_usage();
vlog_usage();
printf("\nDPDK options:\n"
- " --dpdk options Initialize DPDK datapath.\n");
+ " --dpdk options Initialize DPDK datapath.\n"
+ " --basename BASENAME override default character device name\n"
+ " for use with userspace vHost.\n");
printf("\nOther options:\n"
" --unixctl=SOCKET override default control socket name\n"
" -h, --help display this help message\n"
--
1.7.4.1
More information about the dev
mailing list