[ovs-dev] [PATCH v5 1/3] netdev-dpdk: Remove dpdkvhostcuse ports

Daniele Di Proietto diproiettod at ovn.org
Tue Aug 16 00:36:09 UTC 2016


Given that the branching is imminent I pushed the patch to master (after
fixing some clang warnings), to get it into 2.6, as I'm happy with the
interface.

Sean, if we settle on a better interface to suit your needs, I'm fine with
changing it in the next few days.

Thanks for the series,

Daniele

2016-08-15 8:11 GMT-07:00 Ciara Loftus <ciara.loftus at intel.com>:

> This commit removes the 'dpdkvhostcuse' port type from the userspace
> datapath. vhost-cuse ports are quickly becoming obsolete as the
> vhost-user port type begins to support a greater feature-set thanks to
> the addition of things like vhost-user multiqueue and potential
> upcoming features like vhost-user client-mode and vhost-user reconnect.
> The feature is also expected to be removed from DPDK soon.
>
> One potential drawback of the removal of this support is that a
> userspace vHost port type is not available in OVS for use with older
> versions of QEMU (pre v2.2). Considering v2.2 is nearly two years old
> this should however be a low impact change.
>
> Signed-off-by: Ciara Loftus <ciara.loftus at intel.com>
> Acked-by: Flavio Leitner <fbl at sysclose.org>
> Acked-by: Daniele Di Proietto <diproiettod at vmware.com>
> Acked-by: Ilya Maximets <i.maximets at samsung.com>
> ---
>  INSTALL.DPDK-ADVANCED.md                    | 242 -----------------
>  NEWS                                        |   1 +
>  acinclude.m4                                |  12 -
>  lib/netdev-dpdk.c                           | 108 +-------
>  rhel/README.RHEL                            |   2 -
>  rhel/etc_sysconfig_network-scripts_ifup-ovs |   7 -
>  utilities/automake.mk                       |   1 -
>  utilities/qemu-wrap.py                      | 389
> ----------------------------
>  vswitchd/vswitch.xml                        |  12 -
>  9 files changed, 5 insertions(+), 769 deletions(-)
>  delete mode 100755 utilities/qemu-wrap.py
>
> diff --git a/INSTALL.DPDK-ADVANCED.md b/INSTALL.DPDK-ADVANCED.md
> index dd36ae4..5d19f2b 100755
> --- a/INSTALL.DPDK-ADVANCED.md
> +++ b/INSTALL.DPDK-ADVANCED.md
> @@ -461,13 +461,6 @@ For users wanting to do packet forwarding using
> kernel stack below are the steps
>       ```
>
>  ## <a name="vhost"></a> 6. Vhost Walkthrough
> -
> -DPDK 16.07 supports two types of vhost:
> -
> -1. vhost-user - enabled default
> -
> -2. vhost-cuse - Legacy, disabled by default
> -
>  ### 6.1 vhost-user
>
>    - Prerequisites:
> @@ -621,241 +614,6 @@ DPDK 16.07 supports two types of vhost:
>
>        Note: For information on libvirt and further tuning refer [libvirt].
>
> -### 6.2 vhost-cuse
> -
> -  - Prerequisites:
> -
> -    QEMU version >= 2.2
> -
> -  - Enable vhost-cuse support
> -
> -    1. Enable vhost cuse support in DPDK
> -
> -       Set `CONFIG_RTE_LIBRTE_VHOST_USER=n` in config/common_linuxapp
> and follow the
> -       steps in 2.2 section of INSTALL.DPDK guide to build DPDK with cuse
> support.
> -       OVS will detect that DPDK has vhost-cuse libraries compiled and in
> turn will enable
> -       support for it in the switch and disable vhost-user support.
> -
> -    2. Insert the Cuse module
> -
> -       `modprobe cuse`
> -
> -    3. Build and insert the `eventfd_link` module
> -
> -       ```
> -       cd $DPDK_DIR/lib/librte_vhost/eventfd_link/
> -       make
> -       insmod $DPDK_DIR/lib/librte_vhost/eventfd_link.ko
> -       ```
> -
> -  - Adding vhost-cuse ports to Switch
> -
> -    Unlike DPDK ring ports, DPDK vhost-cuse ports can have arbitrary
> names.
> -    For vhost-cuse, the name of the port type is `dpdkvhostcuse`
> -
> -    ```
> -    ovs-vsctl add-port br0 vhost-cuse-1 -- set Interface vhost-cuse-1
> -    type=dpdkvhostcuse
> -    ```
> -
> -    When attaching vhost-cuse ports to QEMU, the name provided during the
> -    add-port operation must match the ifname parameter on the QEMU cmd
> line.
> -
> -  - Adding vhost-cuse ports to VM
> -
> -    vhost-cuse ports use a Linux* character device to communicate with
> QEMU.
> -    By default it is set to `/dev/vhost-net`. It is possible to reuse this
> -    standard device for DPDK vhost, which makes setup a little simpler
> but it
> -    is better practice to specify an alternative character device in
> order to
> -    avoid any conflicts if kernel vhost is to be used in parallel.
> -
> -    1. This step is only needed if using an alternative character device.
> -
> -       ```
> -       ./utilities/ovs-vsctl --no-wait set Open_vSwitch . \
> -            other_config:cuse-dev-name=my-vhost-net
> -       ```
> -
> -       In the example above, the character device to be used will be
> -       `/dev/my-vhost-net`.
> -
> -    2. In case of reusing kernel vhost character device, there would be
> conflict
> -       user should remove it.
> -
> -       `rm -rf /dev/vhost-net`
> -
> -    3. Configure virtio-net adapters
> -
> -       The following parameters must be passed to the QEMU binary, repeat
> -       the below parameters for multiple devices.
> -
> -       ```
> -       -netdev tap,id=<id>,script=no,downscript=no,ifname=<name>,vhost=on
> -       -device virtio-net-pci,netdev=net1,mac=<mac>
> -       ```
> -
> -       The DPDK vhost library will negotiate its own features, so they
> -       need not be passed in as command line params. Note that as offloads
> -       are disabled this is the equivalent of setting
> -
> -       `csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off`
> -
> -       When using an alternative character device, it must be explicitly
> -       passed to QEMU using the `vhostfd` argument
> -
> -       ```
> -       -netdev tap,id=<id>,script=no,downscript=no,ifname=<name>,
> vhost=on,
> -       vhostfd=<open_fd> -device virtio-net-pci,netdev=net1,mac=<mac>
> -       ```
> -
> -       The open file descriptor must be passed to QEMU running as a child
> -       process. This could be done with a simple python script.
> -
> -       ```
> -       #!/usr/bin/python
> -       fd = os.open("/dev/usvhost", os.O_RDWR)
> -       subprocess.call("qemu-system-x86_64 .... -netdev
> tap,id=vhostnet0,\
> -                       vhost=on,vhostfd=" + fd +"...", shell=True)
> -       ```
> -
> -    4. Configure huge pages
> -
> -       QEMU must allocate the VM's memory on hugetlbfs. Vhost ports
> access a
> -       virtio-net device's virtual rings and packet buffers mapping the
> VM's
> -       physical memory on hugetlbfs. To enable vhost-ports to map the VM's
> -       memory into their process address space, pass the following
> parameters
> -       to QEMU
> -
> -       `-object memory-backend-file,id=mem,size=4096M,mem-path=/dev/
> hugepages,
> -       share=on -numa node,memdev=mem -mem-prealloc`
> -
> -  - VM Configuration with QEMU wrapper
> -
> -    The QEMU wrapper script automatically detects and calls QEMU with the
> -    necessary parameters. It performs the following actions:
> -
> -    * Automatically detects the location of the hugetlbfs and inserts this
> -    into the command line parameters.
> -    * Automatically open file descriptors for each virtio-net device and
> -    inserts this into the command line parameters.
> -    * Calls QEMU passing both the command line parameters passed to the
> -    script itself and those it has auto-detected.
> -
> -    Before use, you **must** edit the configuration parameters section of
> the
> -    script to point to the correct emulator location and set additional
> -    settings. Of these settings, `emul_path` and `us_vhost_path` **must**
> be
> -    set. All other settings are optional.
> -
> -    To use directly from the command line simply pass the wrapper some of
> the
> -    QEMU parameters: it will configure the rest. For example:
> -
> -    ```
> -    qemu-wrap.py -cpu host -boot c -hda <disk image> -m 4096 -smp 4
> -    --enable-kvm -nographic -vnc none -net none -netdev tap,id=net1,
> -    script=no,downscript=no,ifname=if1,vhost=on -device virtio-net-pci,
> -    netdev=net1,mac=00:00:00:00:00:01
> -    ```
> -
> -  - VM Configuration with libvirt
> -
> -    If you are using libvirt, you must enable libvirt to access the
> character
> -    device by adding it to controllers cgroup for libvirtd using the
> following
> -    steps.
> -
> -    1. In `/etc/libvirt/qemu.conf` add/edit the following lines:
> -
> -       ```
> -       clear_emulator_capabilities = 0
> -       user = "root"
> -       group = "root"
> -       cgroup_device_acl = [
> -             "/dev/null", "/dev/full", "/dev/zero",
> -             "/dev/random", "/dev/urandom",
> -             "/dev/ptmx", "/dev/kvm", "/dev/kqemu",
> -             "/dev/rtc", "/dev/hpet", "/dev/net/tun",
> -             "/dev/<my-vhost-device>",
> -             "/dev/hugepages"]
> -       ```
> -
> -       <my-vhost-device> refers to "vhost-net" if using the
> `/dev/vhost-net`
> -       device. If you have specificed a different name in the database
> -       using the "other_config:cuse-dev-name" parameter, please specify
> that
> -       filename instead.
> -
> -    2. Disable SELinux or set to permissive mode
> -
> -    3. Restart the libvirtd process
> -       For example, on Fedora:
> -
> -       `systemctl restart libvirtd.service`
> -
> -    After successfully editing the configuration, you may launch your
> -    vhost-enabled VM. The XML describing the VM can be configured like so
> -    within the <qemu:commandline> section:
> -
> -    1. Set up shared hugepages:
> -
> -       ```
> -       <qemu:arg value='-object'/>
> -       <qemu:arg value='memory-backend-file,id=
> mem,size=4096M,mem-path=/dev/hugepages,share=on'/>
> -       <qemu:arg value='-numa'/>
> -       <qemu:arg value='node,memdev=mem'/>
> -       <qemu:arg value='-mem-prealloc'/>
> -       ```
> -
> -    2. Set up your tap devices:
> -
> -       ```
> -       <qemu:arg value='-netdev'/>
> -       <qemu:arg value='type=tap,id=net1,script=no,downscript=no,
> ifname=vhost0,vhost=on'/>
> -       <qemu:arg value='-device'/>
> -       <qemu:arg value='virtio-net-pci,netdev=
> net1,mac=00:00:00:00:00:01'/>
> -       ```
> -
> -    Repeat for as many devices as are desired, modifying the id, ifname
> -    and mac as necessary.
> -
> -    Again, if you are using an alternative character device (other than
> -    `/dev/vhost-net`), please specify the file descriptor like so:
> -
> -    `<qemu:arg value='type=tap,id=net3,script=no,downscript=no,
> ifname=vhost0,vhost=on,vhostfd=<open_fd>'/>`
> -
> -    Where <open_fd> refers to the open file descriptor of the character
> device.
> -    Instructions of how to retrieve the file descriptor can be found in
> the
> -    "DPDK vhost VM configuration" section.
> -    Alternatively, the process is automated with the qemu-wrap.py script,
> -    detailed in the next section.
> -
> -    Now you may launch your VM using virt-manager, or like so:
> -
> -   `virsh create my_vhost_vm.xml`
> -
> -  - VM Configuration with libvirt & QEMU wrapper
> -
> -    To use the qemu-wrapper script in conjuntion with libvirt, follow the
> -    steps in the previous section before proceeding with the following
> steps:
> -
> -    1. Place `qemu-wrap.py` in libvirtd binary search PATH ($PATH)
> -       Ideally in the same directory that the QEMU binary is located.
> -
> -    2. Ensure that the script has the same owner/group and file
> permissions
> -       as the QEMU binary.
> -
> -    3. Update the VM xml file using "virsh edit VM.xml"
> -
> -       Set the VM to use the launch script.
> -       Set the emulator path contained in the `<emulator><emulator/>`
> tags.
> -       For example, replace `<emulator>/usr/bin/qemu-kvm<emulator/>` with
> -          `<emulator>/usr/bin/qemu-wrap.py<emulator/>`
> -
> -    4. Edit the Configuration Parameters section of the script to point to
> -       the correct emulator location and set any additional options. If
> you are
> -       using a alternative character device name, please set
> "us_vhost_path" to the
> -       location of that device. The script will automatically detect and
> insert
> -       the correct "vhostfd" value in the QEMU command line arguments.
> -
> -    5. Use virt-manager to launch the VM
> -
>  ### 6.3 DPDK backend inside VM
>
>    Please note that additional configuration is required if you want to run
> diff --git a/NEWS b/NEWS
> index 2fd3958..5dbcd1d 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -71,6 +71,7 @@ Post-v2.5.0
>       * Support for DPDK 16.07
>       * Optional support for DPDK pdump enabled.
>       * Jumbo frame support
> +     * Remove dpdkvhostcuse port type.
>     - Increase number of registers to 16.
>     - ovs-benchmark: This utility has been removed due to lack of use and
>       bitrot.
> diff --git a/acinclude.m4 b/acinclude.m4
> index aa57b47..5a6dca7 100644
> --- a/acinclude.m4
> +++ b/acinclude.m4
> @@ -201,18 +201,6 @@ AC_DEFUN([OVS_CHECK_DPDK], [
>        AC_LANG_PROGRAM(
>          [
>            #include <rte_config.h>
> -#if !RTE_LIBRTE_VHOST_USER
> -#error
> -#endif
> -        ], [])
> -      ], [],
> -      [AC_DEFINE([VHOST_CUSE], [1], [DPDK vhost-cuse support enabled,
> vhost-user disabled.])
> -       DPDK_EXTRA_LIB="-lfuse"])
> -
> -    AC_COMPILE_IFELSE([
> -      AC_LANG_PROGRAM(
> -        [
> -          #include <rte_config.h>
>  #if RTE_LIBRTE_VHOST_NUMA
>  #error
>  #endif
> diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
> index c767fd4..6998452 100644
> --- a/lib/netdev-dpdk.c
> +++ b/lib/netdev-dpdk.c
> @@ -140,9 +140,6 @@ BUILD_ASSERT_DECL((MAX_NB_MBUF /
> ROUND_DOWN_POW2(MAX_NB_MBUF/MIN_NB_MBUF))
>  #define OVS_VHOST_QUEUE_DISABLED    (-2) /* Queue was disabled by guest
> and not
>                                            * yet mapped to another queue.
> */
>
> -#ifdef VHOST_CUSE
> -static char *cuse_dev_name = NULL;    /* Character device cuse_dev_name.
> */
> -#endif
>  static char *vhost_sock_dir = NULL;   /* Location of vhost-user sockets */
>
>  #define VHOST_ENQ_RETRY_NUM 8
> @@ -878,34 +875,6 @@ dpdk_dev_parse_name(const char dev_name[], const char
> prefix[],
>  }
>
>  static int
> -vhost_construct_helper(struct netdev *netdev) OVS_REQUIRES(dpdk_mutex)
> -{
> -    if (rte_eal_init_ret) {
> -        return rte_eal_init_ret;
> -    }
> -
> -    return netdev_dpdk_init(netdev, -1, DPDK_DEV_VHOST);
> -}
> -
> -static int
> -netdev_dpdk_vhost_cuse_construct(struct netdev *netdev)
> -{
> -    struct netdev_dpdk *dev = netdev_dpdk_cast(netdev);
> -    int err;
> -
> -    if (rte_eal_init_ret) {
> -        return rte_eal_init_ret;
> -    }
> -
> -    ovs_mutex_lock(&dpdk_mutex);
> -    strncpy(CONST_CAST(char *, dev->vhost_id), netdev->name,
> -            sizeof dev->vhost_id);
> -    err = vhost_construct_helper(netdev);
> -    ovs_mutex_unlock(&dpdk_mutex);
> -    return err;
> -}
> -
> -static int
>  netdev_dpdk_vhost_user_construct(struct netdev *netdev)
>  {
>      struct netdev_dpdk *dev = netdev_dpdk_cast(netdev);
> @@ -942,7 +911,7 @@ netdev_dpdk_vhost_user_construct(struct netdev
> *netdev)
>          fatal_signal_add_file_to_unlink(dev->vhost_id);
>          VLOG_INFO("Socket %s created for vhost-user port %s\n",
>                    dev->vhost_id, name);
> -        err = vhost_construct_helper(netdev);
> +        err = netdev_dpdk_init(netdev, -1, DPDK_DEV_VHOST);
>      }
>
>      ovs_mutex_unlock(&dpdk_mutex);
> @@ -2499,7 +2468,7 @@ static void *
>  start_vhost_loop(void *dummy OVS_UNUSED)
>  {
>       pthread_detach(pthread_self());
> -     /* Put the cuse thread into quiescent state. */
> +     /* Put the vhost thread into quiescent state. */
>       ovsrcu_quiesce_start();
>       rte_vhost_driver_session_start();
>       return NULL;
> @@ -2518,12 +2487,6 @@ dpdk_vhost_class_init(void)
>  }
>
>  static int
> -dpdk_vhost_cuse_class_init(void)
> -{
> -    return 0;
> -}
> -
> -static int
>  dpdk_vhost_user_class_init(void)
>  {
>      return 0;
> @@ -2985,29 +2948,6 @@ netdev_dpdk_vhost_user_reconfigure(struct netdev
> *netdev)
>      return 0;
>  }
>
> -static int
> -netdev_dpdk_vhost_cuse_reconfigure(struct netdev *netdev)
> -{
> -    struct netdev_dpdk *dev = netdev_dpdk_cast(netdev);
> -
> -    ovs_mutex_lock(&dpdk_mutex);
> -    ovs_mutex_lock(&dev->mutex);
> -
> -    netdev->n_txq = dev->requested_n_txq;
> -    netdev->n_rxq = 1;
> -
> -    if (dev->requested_mtu != dev->mtu) {
> -        if (!netdev_dpdk_mempool_configure(dev)) {
> -            netdev_change_seq_changed(netdev);
> -        }
> -    }
> -
> -    ovs_mutex_unlock(&dev->mutex);
> -    ovs_mutex_unlock(&dpdk_mutex);
> -
> -    return 0;
> -}
> -
>  #define NETDEV_DPDK_CLASS(NAME, INIT, CONSTRUCT, DESTRUCT,    \
>                            SET_CONFIG, SET_TX_MULTIQ, SEND,    \
>                            GET_CARRIER, GET_STATS,             \
> @@ -3091,8 +3031,8 @@ process_vhost_flags(char *flag, char *default_val,
> int size,
>
>      val = smap_get(ovs_other_config, flag);
>
> -    /* Depending on which version of vhost is in use, process the
> vhost-specific
> -     * flag if it is provided, otherwise resort to default value.
> +    /* Process the vhost-sock-dir flag if it is provided, otherwise
> resort to
> +     * default value.
>       */
>      if (val && (strlen(val) <= size)) {
>          changed = 1;
> @@ -3316,9 +3256,7 @@ dpdk_init__(const struct smap *ovs_other_config)
>      bool auto_determine = true;
>      int err = 0;
>      cpu_set_t cpuset;
> -#ifndef VHOST_CUSE
>      char *sock_dir_subcomponent;
> -#endif
>
>      if (!smap_get_bool(ovs_other_config, "dpdk-init", false)) {
>          VLOG_INFO("DPDK Disabled - to change this requires a restart.\n");
> @@ -3326,11 +3264,6 @@ dpdk_init__(const struct smap *ovs_other_config)
>      }
>
>      VLOG_INFO("DPDK Enabled, initializing");
> -
> -#ifdef VHOST_CUSE
> -    if (process_vhost_flags("cuse-dev-name", xstrdup("vhost-net"),
> -                            PATH_MAX, ovs_other_config, &cuse_dev_name)) {
> -#else
>      if (process_vhost_flags("vhost-sock-dir", xstrdup(ovs_rundir()),
>                              NAME_MAX, ovs_other_config,
>                              &sock_dir_subcomponent)) {
> @@ -3353,7 +3286,6 @@ dpdk_init__(const struct smap *ovs_other_config)
>          free(sock_dir_subcomponent);
>      } else {
>          vhost_sock_dir = sock_dir_subcomponent;
> -#endif
>      }
>
>      argv = grow_argv(&argv, 0, 1);
> @@ -3445,18 +3377,6 @@ dpdk_init__(const struct smap *ovs_other_config)
>
>      ovs_thread_create("dpdk_watchdog", dpdk_watchdog, NULL);
>
> -#ifdef VHOST_CUSE
> -    /* Register CUSE device to handle IOCTLs.
> -     * Unless otherwise specified, cuse_dev_name is set to vhost-net.
> -     */
> -    err = rte_vhost_driver_register(cuse_dev_name, 0);
> -
> -    if (err != 0) {
> -        VLOG_ERR("CUSE device setup failure.");
> -        return;
> -    }
> -#endif
> -
>      dpdk_vhost_class_init();
>
>  #ifdef DPDK_PDUMP
> @@ -3522,22 +3442,6 @@ static const struct netdev_class dpdk_ring_class =
>          netdev_dpdk_reconfigure,
>          netdev_dpdk_rxq_recv);
>
> -static const struct netdev_class OVS_UNUSED dpdk_vhost_cuse_class =
> -    NETDEV_DPDK_CLASS(
> -        "dpdkvhostcuse",
> -        dpdk_vhost_cuse_class_init,
> -        netdev_dpdk_vhost_cuse_construct,
> -        netdev_dpdk_vhost_destruct,
> -        NULL,
> -        NULL,
> -        netdev_dpdk_vhost_send,
> -        netdev_dpdk_vhost_get_carrier,
> -        netdev_dpdk_vhost_get_stats,
> -        NULL,
> -        NULL,
> -        netdev_dpdk_vhost_cuse_reconfigure,
> -        netdev_dpdk_vhost_rxq_recv);
> -
>  static const struct netdev_class OVS_UNUSED dpdk_vhost_user_class =
>      NETDEV_DPDK_CLASS(
>          "dpdkvhostuser",
> @@ -3560,11 +3464,7 @@ netdev_dpdk_register(void)
>      dpdk_common_init();
>      netdev_register_provider(&dpdk_class);
>      netdev_register_provider(&dpdk_ring_class);
> -#ifdef VHOST_CUSE
> -    netdev_register_provider(&dpdk_vhost_cuse_class);
> -#else
>      netdev_register_provider(&dpdk_vhost_user_class);
> -#endif
>  }
>
>  void
> diff --git a/rhel/README.RHEL b/rhel/README.RHEL
> index 21b50cc..fec9c75 100644
> --- a/rhel/README.RHEL
> +++ b/rhel/README.RHEL
> @@ -36,8 +36,6 @@ assignments.  The following OVS-specific variable names
> are supported:
>          * "OVSDPDKRPort", if <name> is a DPDK ring port (name must
>            start with dpdkr and end with portid, eg "dpdkr0")
>
> -        * "OVSDPDKVhostPort" if <name> is a DPDK vhost-cuse port
> -
>          * "OVSDPDKVhostUserPort" if <name> is a DPDK vhost-user port
>
>          * "OVSDPDKBond" if <name> is an OVS DPDK bond.
> diff --git a/rhel/etc_sysconfig_network-scripts_ifup-ovs
> b/rhel/etc_sysconfig_network-scripts_ifup-ovs
> index be0f2dd..e49e6fe 100755
> --- a/rhel/etc_sysconfig_network-scripts_ifup-ovs
> +++ b/rhel/etc_sysconfig_network-scripts_ifup-ovs
> @@ -179,13 +179,6 @@ case "$TYPE" in
>                         -- add-port "$OVS_BRIDGE" "$DEVICE" $OVS_OPTIONS \
>                         -- set Interface "$DEVICE" type=dpdkr
> ${OVS_EXTRA+-- $OVS_EXTRA}
>                 ;;
> -       OVSDPDVhostPort)
> -               ifup_ovs_bridge
> -               ovs-vsctl -t ${TIMEOUT} \
> -                       -- --if-exists del-port "$OVS_BRIDGE" "$DEVICE" \
> -                       -- add-port "$OVS_BRIDGE" "$DEVICE" $OVS_OPTIONS \
> -                       -- set Interface "$DEVICE" type=dpdkvhost
> ${OVS_EXTRA+-- $OVS_EXTRA}
> -               ;;
>         OVSDPDKVhostUserPort)
>                 ifup_ovs_bridge
>                 ovs-vsctl -t ${TIMEOUT} \
> diff --git a/utilities/automake.mk b/utilities/automake.mk
> index 9d5b425..380418a 100644
> --- a/utilities/automake.mk
> +++ b/utilities/automake.mk
> @@ -58,7 +58,6 @@ EXTRA_DIST += \
>         utilities/ovs-test.in \
>         utilities/ovs-vlan-test.in \
>         utilities/ovs-vsctl-bashcomp.bash \
> -       utilities/qemu-wrap.py \
>         utilities/checkpatch.py
>  MAN_ROOTS += \
>         utilities/ovs-appctl.8.in \
> diff --git a/utilities/qemu-wrap.py b/utilities/qemu-wrap.py
> deleted file mode 100755
> index 7847c8c..0000000
> --- a/utilities/qemu-wrap.py
> +++ /dev/null
> @@ -1,389 +0,0 @@
> -#! /usr/bin/env python
> -#
> -#   BSD LICENSE
> -#
> -#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
> -#   All rights reserved.
> -#
> -#   Redistribution and use in source and binary forms, with or without
> -#   modification, are permitted provided that the following conditions
> -#   are met:
> -#
> -#     * Redistributions of source code must retain the above copyright
> -#       notice, this list of conditions and the following disclaimer.
> -#     * Redistributions in binary form must reproduce the above copyright
> -#       notice, this list of conditions and the following disclaimer in
> -#       the documentation and/or other materials provided with the
> -#       distribution.
> -#     * Neither the name of Intel Corporation nor the names of its
> -#       contributors may be used to endorse or promote products derived
> -#       from this software without specific prior written permission.
> -#
> -#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> -#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> -#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> -#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> -#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> -#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> -#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> -#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> -#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> -#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> -#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> -#
> -
> -#####################################################################
> -# This script is designed to modify the call to the QEMU emulator
> -# to support userspace vhost when starting a guest machine through
> -# libvirt with vhost enabled. The steps to enable this are as follows
> -# and should be run as root:
> -#
> -# 1. Place this script in a libvirtd's binary search PATH ($PATH)
> -#    A good location would be in the same directory that the QEMU
> -#    binary is located
> -#
> -# 2. Ensure that the script has the same owner/group and file
> -#    permissions as the QEMU binary
> -#
> -# 3. Update the VM xml file using "virsh edit VM.xml"
> -#
> -#    3.a) Set the VM to use the launch script
> -#
> -#        Set the emulator path contained in the
> -#        <emulator><emulator/> tags
> -#
> -#        e.g replace <emulator>/usr/bin/qemu-kvm<emulator/>
> -#        with    <emulator>/usr/bin/qemu-wrap.py<emulator/>
> -#
> -#     3.b) Set the VM's device's to use vhost-net offload
> -#
> -#        <interface type="network">
> -#           <model type="virtio"/>
> -#           <driver name="vhost"/>
> -#        <interface/>
> -#
> -# 4. Enable libvirt to access our userpace device file by adding it to
> -#    controllers cgroup for libvirtd using the following steps
> -#
> -#   4.a) In /etc/libvirt/qemu.conf add/edit the following lines:
> -#         1) cgroup_controllers = [ ... "devices", ... ]
> -#         2) clear_emulator_capabilities = 0
> -#         3) user = "root"
> -#         4) group = "root"
> -#         5) cgroup_device_acl = [
> -#                "/dev/null", "/dev/full", "/dev/zero",
> -#                "/dev/random", "/dev/urandom",
> -#                "/dev/ptmx", "/dev/kvm", "/dev/kqemu",
> -#                "/dev/rtc", "/dev/hpet", "/dev/net/tun",
> -#                "/dev/<devbase-name>-<index>",
> -#                "/dev/hugepages"
> -#            ]
> -#
> -#   4.b) Disable SELinux or set to permissive mode
> -#
> -#   4.c) Mount cgroup device controller
> -#        "mkdir /dev/cgroup"
> -#        "mount -t cgroup none /dev/cgroup -o devices"
> -#
> -#   4.d) Set hugetlbfs_mount variable - ( Optional )
> -#        VMs using userspace vhost must use hugepage backed
> -#        memory. This can be enabled in the libvirt XML
> -#        config by adding a memory backing section to the
> -#        XML config e.g.
> -#             <memoryBacking>
> -#             <hugepages/>
> -#             </memoryBacking>
> -#        This memory backing section should be added after the
> -#        <memory> and <currentMemory> sections. This will add
> -#        flags "-mem-prealloc -mem-path <path>" to the QEMU
> -#        command line. The hugetlbfs_mount variable can be used
> -#        to override the default <path> passed through by libvirt.
> -#
> -#        if "-mem-prealloc" or "-mem-path <path>" are not passed
> -#        through and a vhost device is detected then these options will
> -#        be automatically added by this script. This script will detect
> -#        the system hugetlbfs mount point to be used for <path>. The
> -#        default <path> for this script can be overidden by the
> -#        hugetlbfs_dir variable in the configuration section of this
> script.
> -#
> -#
> -#   4.e) Restart the libvirtd system process
> -#        e.g. on Fedora "systemctl restart libvirtd.service"
> -#
> -#
> -#   4.f) Edit the Configuration Parameters section of this script
> -#        to point to the correct emulator location and set any
> -#        addition options
> -#
> -# The script modifies the libvirtd Qemu call by modifying/adding
> -# options based on the configuration parameters below.
> -# NOTE:
> -#     emul_path and us_vhost_path must be set
> -#     All other parameters are optional
> -#####################################################################
> -
> -
> -#############################################
> -# Configuration Parameters
> -#############################################
> -#Path to QEMU binary
> -emul_path = "/usr/local/bin/qemu-system-x86_64"
> -
> -#Path to userspace vhost device file
> -# This filename should match the --dev-basename --dev-index parameters of
> -# the command used to launch the userspace vhost sample application e.g.
> -# if the sample app lauch command is:
> -#    ./build/vhost-switch ..... --dev-basename usvhost --dev-index 1
> -# then this variable should be set to:
> -#   us_vhost_path = "/dev/usvhost-1"
> -us_vhost_path = "/dev/usvhost-1"
> -
> -#List of additional user defined emulation options. These options will
> -#be added to all Qemu calls
> -emul_opts_user = []
> -
> -#List of additional user defined emulation options for vhost only.
> -#These options will only be added to vhost enabled guests
> -emul_opts_user_vhost = []
> -
> -#For all VHOST enabled VMs, the VM memory is preallocated from hugetlbfs
> -# Set this variable to one to enable this option for all VMs
> -use_huge_all = 0
> -
> -#Instead of autodetecting, override the hugetlbfs directory by setting
> -#this variable
> -hugetlbfs_dir = ""
> -
> -#############################################
> -
> -
> -#############################################
> -# ****** Do Not Modify Below this Line ******
> -#############################################
> -
> -import sys, os, subprocess
> -import time
> -import signal
> -
> -
> -#List of open userspace vhost file descriptors
> -fd_list = []
> -
> -#additional virtio device flags when using userspace vhost
> -vhost_flags = [ "csum=off",
> -                "gso=off",
> -                "guest_tso4=off",
> -                "guest_tso6=off",
> -                "guest_ecn=off"
> -              ]
> -
> -#String of the path to the Qemu process pid
> -qemu_pid = "/tmp/%d-qemu.pid" % os.getpid()
> -
> -#############################################
> -# Signal haldler to kill Qemu subprocess
> -#############################################
> -def kill_qemu_process(signum, stack):
> -    pidfile = open(qemu_pid, 'r')
> -    pid = int(pidfile.read())
> -    os.killpg(pid, signal.SIGTERM)
> -    pidfile.close()
> -
> -
> -#############################################
> -# Find the system hugefile mount point.
> -# Note:
> -# if multiple hugetlbfs mount points exist
> -# then the first one found will be used
> -#############################################
> -def find_huge_mount():
> -
> -    if (len(hugetlbfs_dir)):
> -        return hugetlbfs_dir
> -
> -    huge_mount = ""
> -
> -    if (os.access("/proc/mounts", os.F_OK)):
> -        f = open("/proc/mounts", "r")
> -        line = f.readline()
> -        while line:
> -            line_split = line.split(" ")
> -            if line_split[2] == 'hugetlbfs':
> -                huge_mount = line_split[1]
> -                break
> -            line = f.readline()
> -    else:
> -        print "/proc/mounts not found"
> -        exit (1)
> -
> -    f.close
> -    if len(huge_mount) == 0:
> -        print "Failed to find hugetlbfs mount point"
> -        exit (1)
> -
> -    return huge_mount
> -
> -
> -#############################################
> -# Get a userspace Vhost file descriptor
> -#############################################
> -def get_vhost_fd():
> -
> -    if (os.access(us_vhost_path, os.F_OK)):
> -        fd = os.open( us_vhost_path, os.O_RDWR)
> -    else:
> -        print ("US-Vhost file %s not found" %us_vhost_path)
> -        exit (1)
> -
> -    return fd
> -
> -
> -#############################################
> -# Check for vhostfd. if found then replace
> -# with our own vhost fd and append any vhost
> -# flags onto the end
> -#############################################
> -def modify_netdev_arg(arg):
> -
> -    global fd_list
> -    vhost_in_use = 0
> -    s = ''
> -    new_opts = []
> -    netdev_opts = arg.split(",")
> -
> -    for opt in netdev_opts:
> -        #check if vhost is used
> -        if "vhost" == opt[:5]:
> -            vhost_in_use = 1
> -        else:
> -            new_opts.append(opt)
> -
> -    #if using vhost append vhost options
> -    if vhost_in_use == 1:
> -        #append vhost on option
> -        new_opts.append('vhost=on')
> -        #append vhostfd ption
> -        new_fd = get_vhost_fd()
> -        new_opts.append('vhostfd=' + str(new_fd))
> -        fd_list.append(new_fd)
> -
> -    #concatenate all options
> -    for opt in new_opts:
> -        if len(s) > 0:
> -            s+=','
> -
> -        s+=opt
> -
> -    return s
> -
> -
> -#############################################
> -# Main
> -#############################################
> -def main():
> -
> -    global fd_list
> -    global vhost_in_use
> -    new_args = []
> -    num_cmd_args = len(sys.argv)
> -    emul_call = ''
> -    mem_prealloc_set = 0
> -    mem_path_set = 0
> -    num = 0;
> -
> -    #parse the parameters
> -    while (num < num_cmd_args):
> -        arg = sys.argv[num]
> -
> -        #Check netdev +1 parameter for vhostfd
> -        if arg == '-netdev':
> -            num_vhost_devs = len(fd_list)
> -            new_args.append(arg)
> -
> -            num+=1
> -            arg = sys.argv[num]
> -            mod_arg = modify_netdev_arg(arg)
> -            new_args.append(mod_arg)
> -
> -            #append vhost flags if this is a vhost device
> -            # and -device is the next arg
> -            # i.e -device -opt1,-opt2,...,-opt3,%vhost
> -            if (num_vhost_devs < len(fd_list)):
> -                num+=1
> -                arg = sys.argv[num]
> -                if arg == '-device':
> -                    new_args.append(arg)
> -                    num+=1
> -                    new_arg = sys.argv[num]
> -                    for flag in vhost_flags:
> -                        new_arg = ''.join([new_arg,',',flag])
> -                    new_args.append(new_arg)
> -                else:
> -                    new_args.append(arg)
> -        elif arg == '-mem-prealloc':
> -            mem_prealloc_set = 1
> -            new_args.append(arg)
> -        elif arg == '-mem-path':
> -            mem_path_set = 1
> -            new_args.append(arg)
> -
> -        else:
> -            new_args.append(arg)
> -
> -        num+=1
> -
> -    #Set Qemu binary location
> -    emul_call+=emul_path
> -    emul_call+=" "
> -
> -    #Add prealloc mem options if using vhost and not already added
> -    if ((len(fd_list) > 0) and (mem_prealloc_set == 0)):
> -        emul_call += "-mem-prealloc "
> -
> -    #Add mempath mem options if using vhost and not already added
> -    if ((len(fd_list) > 0) and (mem_path_set == 0)):
> -        #Detect and add hugetlbfs mount point
> -        mp = find_huge_mount()
> -        mp = "".join(["-mem-path ", mp])
> -        emul_call += mp
> -        emul_call += " "
> -
> -    #add user options
> -    for opt in emul_opts_user:
> -        emul_call += opt
> -        emul_call += " "
> -
> -    #Add add user vhost only options
> -    if len(fd_list) > 0:
> -        for opt in emul_opts_user_vhost:
> -            emul_call += opt
> -            emul_call += " "
> -
> -    #Add updated libvirt options
> -    iter_args = iter(new_args)
> -    #skip 1st arg i.e. call to this script
> -    next(iter_args)
> -    for arg in iter_args:
> -        emul_call+=str(arg)
> -        emul_call+= " "
> -
> -    emul_call += "-pidfile %s " % qemu_pid
> -    #Call QEMU
> -    process = subprocess.Popen(emul_call, shell=True,
> preexec_fn=os.setsid)
> -
> -    for sig in [signal.SIGTERM, signal.SIGINT, signal.SIGHUP,
> signal.SIGQUIT]:
> -        signal.signal(sig, kill_qemu_process)
> -
> -    process.wait()
> -
> -    #Close usvhost files
> -    for fd in fd_list:
> -        os.close(fd)
> -    #Cleanup temporary files
> -    if os.access(qemu_pid, os.F_OK):
> -        os.remove(qemu_pid)
> -
> -
> -
> -if __name__ == "__main__":
> -    main()
> diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml
> index 780bd2d..8331b49 100644
> --- a/vswitchd/vswitch.xml
> +++ b/vswitchd/vswitch.xml
> @@ -286,18 +286,6 @@
>          </p>
>        </column>
>
> -      <column name="other_config" key="cuse-dev-name"
> -              type='{"type": "string"}'>
> -        <p>
> -          Specifies the name of the vhost-cuse character device to open
> for
> -          vhost-cuse support.
> -        </p>
> -        <p>
> -          The default is vhost-net. Changing this value requires
> restarting
> -          the daemon.
> -        </p>
> -      </column>
> -
>        <column name="other_config" key="vhost-sock-dir"
>                type='{"type": "string"}'>
>          <p>
> --
> 2.4.3
>
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> http://openvswitch.org/mailman/listinfo/dev
>



More information about the dev mailing list