[ovs-dev] [PATCH v3 2/3] netdev-dpdk: Convert initialization from cmdline to db

[Panu Matilainen]

On 01/13/2016 11:17 PM, Aaron Conole wrote:
> Existing DPDK integration is provided by use of command line options which
> must be split out and passed to librte in a special manner. However, this
> forces any configuration to be passed by way of a special DPDK flag, and
> interferes with ovs+dpdk packaging solutions.
>
> This commit delays dpdk initialization until after the OVS database
> connection is established, at which point ovs initializes librte. It
> pulls all of the config data from the OVS database, and assembles a
> new argv/argc pair to be passed along.
>
> Signed-off-by: Aaron Conole <aconole at redhat.com>
> ---
> v2:
> * Removed trailing whitespace
> * Followed for() loop brace coding style
> * Automatically enable DPDK when adding a DPDK enabled port
> * Fixed an issue on startup when DPDK enabled ports are present
> * Updated the documentation (including vswitch.xml) and documented all
>    new parameters
> * Dropped the premature initialization test
>
> v3:
> * Improved description language in INSTALL.DPDK.md
> * Fixed the ovs-vsctl examples for DPDK
> * Returned to the global dpdk-init (bullet 3 from v2)
> * Fixed a build error when compiling without dpdk support enabled
> * converted to xstrdup, for consistency after rebasing
>
>   INSTALL.DPDK.md         |  81 +++++++++++++++-----
>   lib/netdev-dpdk.c       | 191 ++++++++++++++++++++++++++++++++++--------------
>   lib/netdev-dpdk.h       |  22 ++++--
>   vswitchd/bridge.c       |   3 +
>   vswitchd/ovs-vswitchd.c |  25 +------
>   vswitchd/vswitch.xml    | 118 +++++++++++++++++++++++++++++-
>   6 files changed, 336 insertions(+), 104 deletions(-)
>
> diff --git a/INSTALL.DPDK.md b/INSTALL.DPDK.md
> index 96b686c..46bd1a8 100644
> --- a/INSTALL.DPDK.md
> +++ b/INSTALL.DPDK.md
> @@ -143,22 +143,64 @@ Using the DPDK with ovs-vswitchd:
>
>   5. Start vswitchd:
>
> -   DPDK configuration arguments can be passed to vswitchd via `--dpdk`
> -   argument. This needs to be first argument passed to vswitchd process.
> -   dpdk arg -c is ignored by ovs-dpdk, but it is a required parameter
> -   for dpdk initialization.
> +   DPDK configuration arguments can be passed to vswitchd via Open_vSwitch
> +   other_config database. The recognized configuration options are listed.
> +   Defaults will be provided for all values not explicitly set.
> +
> +   * dpdk-init
> +   Specifies whether OVS should initialize and support DPDK ports. This is
> +   a boolean, and defaults to false.
> +
> +   * dpdk-lcore-mask
> +   Specifies the CPU cores on which dpdk lcore threads should be spawned.
> +   The DPDK lcore threads are used for DPDK library tasks, such as
> +   library internal message processing, logging, etc. Value should be in
> +   the form of a hex string (so '0x123') similar to the 'taskset' mask
> +   input.
> +   If not specified, the value will be determined by choosing the lowest
> +   CPU core from initial cpu affinity list. Otherwise, the value will be
> +   passed directly to the DPDK library.
> +   For performance reasons, it is best to set this to a single core on
> +   the system, rather than allow lcore threads to float.
> +
> +   * dpdk-mem-channels
> +   This sets the number of memory spread channels per CPU socket. It is purely
> +   an optimization flag.
> +   * dpdk-alloc-mem
> +   This sets the total memory to preallocate from hugepages regardless of
> +   processor socket. It is recommended to use dpdk-socket-mem instead.

Since dpdk-socket-mem is the recommended option, does it actually make 
sense to support this at all? In case of DPDK itself I'd hazard a guess 
the -m option simply predates --socket-mem so its just a historical 
leftover (but again, this is just a guess)

> +   * dpdk-socket-mem
> +   Comma separated list of memory to pre-allocate from hugepages on specific
> +   sockets.
> +
> +   * dpdk-hugepage-dir
> +   Directory where hugetlbfs is mounted
> +
> +   * cuse-dev-name
> +   Option to set the vhost_cuse character device name.
> +
> +   * vhost-sock-dir
> +   Option to set the path to the vhost_user unix socket files.
> +

Hmm, this is quite a limited set of options considering that previously 
you could pass arbitrary DPDK options to vswitchd (unless I'm being 
blind again, wouldn't be the first time :)

I dont remember all DPDK options off-hand, much less know how many of 
them are actually used in the wild, but what I suspect people would 
sooner than later miss are:
- support for adding virtual devices (ie --vdev equivalent)
- blacklist/whitelist of devices (especially with virtio devices)

Adding virtual devices via "raw" DPDK args is horribly klunky of course, 
it could use some nicer way, ditto for blacklist/whitelist.

Anyway, I think it'd be important to have *some* way of passing 
arbitrary DPDK flags to OVS, klunky or not. That way people wont get 
stuck if they happen to need some esoteric option in their setup, and 
also allows testing new features before proper integration is 
implemented. For example libvirt has all sorts of higher level options 
in its config, but also a special syntax to pass arbitrary options to qemu:
https://libvirt.org/drvqemu.html#qemucommand

	- Panu -