[ovs-dev] [PATCH v2] ovsdb: Expose vhost-user socket directory in ovsdb

[Aaron Conole]

"Mooney, Sean K" <sean.k.mooney at intel.com> writes:

> "Wojciechowicz, RobertX" <robertx.wojciechowicz at intel.com> writes:
>
>> Hi Ben,
>>
>>
>>> -----Original Message-----
>>> From: Ben Pfaff [mailto:blp at ovn.org]
>>> Sent: Tuesday, July 5, 2016 5:07 PM
>>> To: Wojciechowicz, RobertX <robertx.wojciechowicz at intel.com>
>>> Cc: dev at openvswitch.org
>>> Subject: Re: [ovs-dev] [PATCH v2] ovsdb: Expose vhost-user socket directory
>>> in ovsdb
>>>
>>> On Mon, Jul 04, 2016 at 07:22:40AM +0000, Wojciechowicz, RobertX wrote:
>>> > Hi,
>>> >
>>> > > -----Original Message-----
>>> > > From: Ben Pfaff [mailto:blp at ovn.org]
>>> > > Sent: Saturday, July 2, 2016 2:49 AM
>>> > > To: Wojciechowicz, RobertX <robertx.wojciechowicz at intel.com>
>>> > > Cc: dev at openvswitch.org
>>> > > Subject: Re: [ovs-dev] [PATCH v2] ovsdb: Expose vhost-user socket
>>> directory
>>> > > in ovsdb
>>> > >
>>> > > On Mon, Jun 20, 2016 at 10:16:51AM +0000, Wojciechowicz, RobertX
>>> wrote:
>>> > > > Hi,
>>> > > >
>>> > > > > -----Original Message-----
>>> > > > > From: Ben Pfaff [mailto:blp at ovn.org]
>>> > > > > Sent: Wednesday, June 8, 2016 10:41 PM
>>> > > > > To: Wojciechowicz, RobertX <robertx.wojciechowicz at intel.com>
>>> > > > > Cc: dev at openvswitch.org
>>> > > > > Subject: Re: [ovs-dev] [PATCH v2] ovsdb: Expose vhost-user socket
>>> > > directory
>>> > > > > in ovsdb
>>> > > > >
>>> > > > > On Thu, Jun 02, 2016 at 11:25:56AM +0100, Robert Wojciechowicz
>>> wrote:
>>> > > > > > In order to correctly interoperate with Openstack and ODL,
>>> > > > > > the vhost-user socket directory must be exposed from OVS via
>>> OVSDB.
>>> > > > > > Different distros may package OVS in different ways,
>>> > > > > > so the locations of these sockets may vary depending on how
>>> > > > > > ovs-vswitchd has been started. Some clients need information
>>> where
>>> > > > > > the sockets are located when instantiating Qemu virtual machines.
>>> > > > > > The full vhost-user socket directory is constructed from current
>>> > > > > > OVS working directory and optionally from specified subdirectory.
>>> > > > > > This patch exposes vhost-user socket directory in Open_vSwitch
>>> > > > > > table in other_config column in two following keys:
>>> > > > > > 1. ovs-run-dir    - OVS working directory
>>> > > > > > 2. vhost-sock-dir - subdirectory of ovs-run-dir (might be empty)
>>> > > > > >
>>> > > > > > Signed-off-by: Robert Wojciechowicz
>>> > > <robertx.wojciechowicz at intel.com>
>>> > > > > >
>>> > > > > > v1->v2
>>> > > > > > - moving vswitch-idl.h dependency inside #ifdef block
>>> > > > > > - sock_dir_subcomponent initialization with ""
>>> > > > >
>>> > > > > Same comment as v1: architecturally, ovs-vswitchd only reads
>>> > > > > other-config columns, it never writes to them.  Please fix.
>>> > > >
>>> > > > If ovs-vswitchd cannot writes to other-config then the only place
>>> > > > for writing default values to this column I can think of is vswitch
>>> > > > startup script ovs-ctl.
>>> > > > Basically I tested in my environment the below solution
>>> > > > and it seems to solve our issue.
>>> > > > Is it acceptable approach?
>>> > >
>>> > > It looks like you're trying to use other-config to report something,
>>> > > instead of to configure something.  That's not what it's for.
>>> >
>>> > Actually I'm trying to add missing information to the OVSDB.
>>> > By default ovs-vswitchd is already configured that vhost-user
>>> > sockects are created in the rundir, but this information
>>> > is not available in the OVSDB. Third-party scripts, which need
>>> > this information are forced to take some guesses about this.
>>> > Basically this approach is very similar to storing hostname
>>> > in this patch:
>>> > http://openvswitch.org/pipermail/dev/2016-March/068511.html
>>>
>>> There is a difference between external-ids and other-config.
>>> other-config is to configure the switch.  That patch uses external-ids.
>>
>> [RW] Yes, of course, but my point is that the configuration
>> currently looks as follows:
>> 1. start ovsdb
>> 2. vhost-sock-dir is not configured
>> 3. start ovs-vswitchd
>> 4. ovs-vswitchd in the function dpdk_init__ configures vhost-sock-dir
>> from ovs_rundir() and sock_dir_subcomponent
>> 5. vhost-sock-dir is now configured, but still there is no information
>> in the ovsdb
>
> I don't understand this flow.  Can you tell me what you mean by
> vhost-sock-dir is configured but not configured?
>
> [sean-k-mooney]
> @aaron Following your change to move the dpdk and vhost-user socket dir paramtater
> To the db if a non default value for vhost-sockt-dir is used it will
> be present in the ovsdb.
> This is good as it can be discovered remotely by odl and reported to
> openstack so that i
> can tell libvirt what the relative path is from the ovs run dir.
> That is where we have a problem. The ovs run dir is both a compile time constant
> That is set via configure flags and can be overridden on the
> commandline when starting ovs.
> Because this value is never stored in the ovsdb odl cannot discover it
> remotely.

Okay.  That sounds like it may be desirable to have the runtime directory
information exposed through some "queryable" interface.

> As different disto have different conventions for where the run
> directory should be loacated
> /var/run/openvswitch vs /usr/var/run/openvswitch we cannot support multiple distros
> in the same deployment with odl as a controller.
>
> In an openstack deployment when you use odl or ovn for that matter
> openstack does not have
> a neutron agent running the system with ovs so no command line tools
> such as app-ctl or other
> methods such as greping ps or reading service files can be used to
> discover the ovs run dir to be
> able to generate the absolute path required by libvirt/qemu to booth the vm.

Okay.  So is there no method accessible to neutron to execute anything
on the ovs system?  Just want to make sure I understand completely.
Things like executing ovs-*ctl, and others are out.

> To answer your initial question though what I belive Robert was trying
> to convey with the workflow is as
> Follows.  When the dpdk init is run in the ovs-vswitchd the root path
> for the vhost-user sockets is caluated
> By concatenating the ovs run dir with the vhos-user-socket-dir. This
> value is stored in memory in the ovs-vswitchd
> Or recaulated when need by ovs but it is never made externally
> accessible. The direct result of this
> Is that since we do not have the information to calculate this path
> while allowing the user/distro to choose
> An arbitrary ovs run dir, the openstack support for vhost-user with
> odl requires that ovs use
> /var/run/openvswtich as its run dir regardless of the distro.
>
> Roberts proposal is trying to remove this restriction complies with
> the debian/Ubuntu packaging scheme
> But conflits with the centos/rhel/fedora packaging guidelines.

Okay, I'll think about how to get this information out.

There are, actually, two pieces of information that I think need to be
exposed.  Please correct me if I'm wrong or feel free to add any other:

1. ovs current runtime directory
2. current dpdk status (active/inactive/unavailable, where active means
dpdk-init=true at startup, inactive means it was not true at startup,
and unavailable means no compile time support is enabled).

Are there any other bits of information?
What interfaces do you have available for use (nw sockets, db, etc.)?

>> Now the question is how the third-party scripts can find out where actually
>> vhost-user sockets are located?
>>
>> Br,
>> Robert