[ovs-dev] [PATCH v2] ovsdb: Expose vhost-user socket directory in ovsdb

Wed Aug 10 13:06:24 UTC 2016

"Wojciechowicz, RobertX" <robertx.wojciechowicz at intel.com> writes:

> Hi Aaron,
>
> did you have time by any chance to get back to this topic?

I haven't, sorry.  Was busy on PTO, welcoming a future Free Software
user (and possible developer) into the world ;)

I have some things to finish up and then I'll get back to it.

> If I correctly understood you have some idea what's the best way to expose
> this vhost-user/runtime directory.
>
> Br,
> Robert
>
>
>> -----Original Message-----
>> From: dev [mailto:dev-bounces at openvswitch.org] On Behalf Of Mooney,
>> Sean K
>> Sent: Wednesday, July 20, 2016 2:37 PM
>> To: Aaron Conole <aconole at redhat.com>
>> Cc: dev at openvswitch.org; Flavio Leitner <fbl at sysclose.org>
>> Subject: Re: [ovs-dev] [PATCH v2] ovsdb: Expose vhost-user socket directory
>> in ovsdb
>> 
>> Hi sorry for the delay
>> Replies inline.
>> 
>> > -----Original Message-----
>> > From: Aaron Conole [mailto:aconole at redhat.com]
>> > Sent: Monday, July 11, 2016 6:44 PM
>> > To: Mooney, Sean K <sean.k.mooney at intel.com>
>> > Cc: Flavio Leitner <fbl at sysclose.org>; dev at openvswitch.org
>> > Subject: Re: [ovs-dev] [PATCH v2] ovsdb: Expose vhost-user socket
>> directory in
>> > ovsdb
>> >
>> > "Mooney, Sean K" <sean.k.mooney at intel.com> writes:
>> >
>> > > "Wojciechowicz, RobertX" <robertx.wojciechowicz at intel.com> writes:
>> > >
>> > >> Hi Ben,
>> > >>
>> > >>
>> > >>> -----Original Message-----
>> > >>> From: Ben Pfaff [mailto:blp at ovn.org]
>> > >>> Sent: Tuesday, July 5, 2016 5:07 PM
>> > >>> To: Wojciechowicz, RobertX <robertx.wojciechowicz at intel.com>
>> > >>> Cc: dev at openvswitch.org
>> > >>> Subject: Re: [ovs-dev] [PATCH v2] ovsdb: Expose vhost-user socket
>> > >>> directory in ovsdb
>> > >>>
>> > >>> On Mon, Jul 04, 2016 at 07:22:40AM +0000, Wojciechowicz, RobertX
>> wrote:
>> > >>> > Hi,
>> > >>> >
>> > >>> > > -----Original Message-----
>> > >>> > > From: Ben Pfaff [mailto:blp at ovn.org]
>> > >>> > > Sent: Saturday, July 2, 2016 2:49 AM
>> > >>> > > To: Wojciechowicz, RobertX <robertx.wojciechowicz at intel.com>
>> > >>> > > Cc: dev at openvswitch.org
>> > >>> > > Subject: Re: [ovs-dev] [PATCH v2] ovsdb: Expose vhost-user
>> > >>> > > socket
>> > >>> directory
>> > >>> > > in ovsdb
>> > >>> > >
>> > >>> > > On Mon, Jun 20, 2016 at 10:16:51AM +0000, Wojciechowicz,
>> RobertX
>> > >>> wrote:
>> > >>> > > > Hi,
>> > >>> > > >
>> > >>> > > > > -----Original Message-----
>> > >>> > > > > From: Ben Pfaff [mailto:blp at ovn.org]
>> > >>> > > > > Sent: Wednesday, June 8, 2016 10:41 PM
>> > >>> > > > > To: Wojciechowicz, RobertX <robertx.wojciechowicz at
>> > >>> > > > > intel.com>
>> > >>> > > > > Cc: dev at openvswitch.org
>> > >>> > > > > Subject: Re: [ovs-dev] [PATCH v2] ovsdb: Expose vhost-user
>> > >>> > > > > socket
>> > >>> > > directory
>> > >>> > > > > in ovsdb
>> > >>> > > > >
>> > >>> > > > > On Thu, Jun 02, 2016 at 11:25:56AM +0100, Robert
>> > >>> > > > > Wojciechowicz
>> > >>> wrote:
>> > >>> > > > > > In order to correctly interoperate with Openstack and ODL,
>> > >>> > > > > > the vhost-user socket directory must be exposed from OVS
>> > >>> > > > > > via
>> > >>> OVSDB.
>> > >>> > > > > > Different distros may package OVS in different ways, so
>> > >>> > > > > > the locations of these sockets may vary depending on how
>> > >>> > > > > > ovs-vswitchd has been started. Some clients need
>> > >>> > > > > > information
>> > >>> where
>> > >>> > > > > > the sockets are located when instantiating Qemu virtual
>> machines.
>> > >>> > > > > > The full vhost-user socket directory is constructed from
>> > >>> > > > > > current OVS working directory and optionally from specified
>> > subdirectory.
>> > >>> > > > > > This patch exposes vhost-user socket directory in
>> > >>> > > > > > Open_vSwitch table in other_config column in two following
>> keys:
>> > >>> > > > > > 1. ovs-run-dir    - OVS working directory
>> > >>> > > > > > 2. vhost-sock-dir - subdirectory of ovs-run-dir (might be
>> > >>> > > > > > empty)
>> > >>> > > > > >
>> > >>> > > > > > Signed-off-by: Robert Wojciechowicz
>> > >>> > > <robertx.wojciechowicz at intel.com>
>> > >>> > > > > >
>> > >>> > > > > > v1->v2
>> > >>> > > > > > - moving vswitch-idl.h dependency inside #ifdef block
>> > >>> > > > > > - sock_dir_subcomponent initialization with ""
>> > >>> > > > >
>> > >>> > > > > Same comment as v1: architecturally, ovs-vswitchd only reads
>> > >>> > > > > other-config columns, it never writes to them.  Please fix.
>> > >>> > > >
>> > >>> > > > If ovs-vswitchd cannot writes to other-config then the only
>> > >>> > > > place for writing default values to this column I can think of
>> > >>> > > > is vswitch startup script ovs-ctl.
>> > >>> > > > Basically I tested in my environment the below solution and it
>> > >>> > > > seems to solve our issue.
>> > >>> > > > Is it acceptable approach?
>> > >>> > >
>> > >>> > > It looks like you're trying to use other-config to report
>> > >>> > > something, instead of to configure something.  That's not what it's
>> for.
>> > >>> >
>> > >>> > Actually I'm trying to add missing information to the OVSDB.
>> > >>> > By default ovs-vswitchd is already configured that vhost-user
>> > >>> > sockects are created in the rundir, but this information is not
>> > >>> > available in the OVSDB. Third-party scripts, which need this
>> > >>> > information are forced to take some guesses about this.
>> > >>> > Basically this approach is very similar to storing hostname in
>> > >>> > this patch:
>> > >>> > http://openvswitch.org/pipermail/dev/2016-March/068511.html
>> > >>>
>> > >>> There is a difference between external-ids and other-config.
>> > >>> other-config is to configure the switch.  That patch uses external-ids.
>> > >>
>> > >> [RW] Yes, of course, but my point is that the configuration currently
>> > >> looks as follows:
>> > >> 1. start ovsdb
>> > >> 2. vhost-sock-dir is not configured
>> > >> 3. start ovs-vswitchd
>> > >> 4. ovs-vswitchd in the function dpdk_init__ configures vhost-sock-dir
>> > >> from ovs_rundir() and sock_dir_subcomponent 5. vhost-sock-dir is now
>> > >> configured, but still there is no information in the ovsdb
>> > >
>> > > I don't understand this flow.  Can you tell me what you mean by
>> > > vhost-sock-dir is configured but not configured?
>> > >
>> > > [sean-k-mooney]
>> > > @aaron Following your change to move the dpdk and vhost-user socket
>> > > dir paramtater To the db if a non default value for vhost-sockt-dir is
>> > > used it will be present in the ovsdb.
>> > > This is good as it can be discovered remotely by odl and reported to
>> > > openstack so that i can tell libvirt what the relative path is from
>> > > the ovs run dir.
>> > > That is where we have a problem. The ovs run dir is both a compile
>> > > time constant That is set via configure flags and can be overridden on
>> > > the commandline when starting ovs.
>> > > Because this value is never stored in the ovsdb odl cannot discover it
>> > > remotely.
>> >
>> > Okay.  That sounds like it may be desirable to have the runtime directory
>> > information exposed through some "queryable" interface.
>> >
>> > > As different disto have different conventions for where the run
>> > > directory should be loacated /var/run/openvswitch vs
>> > > /usr/var/run/openvswitch we cannot support multiple distros in the
>> > > same deployment with odl as a controller.
>> > >
>> > > In an openstack deployment when you use odl or ovn for that matter
>> > > openstack does not have a neutron agent running the system with ovs so
>> > > no command line tools such as app-ctl or other methods such as greping
>> > > ps or reading service files can be used to discover the ovs run dir to
>> > > be able to generate the absolute path required by libvirt/qemu to
>> > > booth the vm.
>> >
>> > Okay.  So is there no method accessible to neutron to execute anything on
>> the
>> > ovs system?  Just want to make sure I understand completely.
>> > Things like executing ovs-*ctl, and others are out.
>> >
>> 
>> [Mooney, Sean K]  correct, when neutron is configured to use an external
>> controller such as odl
>> It cannot execut any commands itself on the server running ovs. With ovn
>> ovn runs an agent on the platform
>> In the form of the local controller but for odl and may other controller such a
>> floodlight etc  they are agentless
>> By design so the external controller is also not able to execute commands on
>> the system running ovs directly.
>> As such ovs-*ctl tools cannot be used to retrieve this information.
>> 
>> > > To answer your initial question though what I belive Robert was trying
>> > > to convey with the workflow is as Follows.  When the dpdk init is run
>> > > in the ovs-vswitchd the root path for the vhost-user sockets is
>> > > caluated By concatenating the ovs run dir with the
>> > > vhos-user-socket-dir. This value is stored in memory in the
>> > > ovs-vswitchd Or recaulated when need by ovs but it is never made
>> > > externally accessible. The direct result of this Is that since we do
>> > > not have the information to calculate this path while allowing the
>> > > user/distro to choose An arbitrary ovs run dir, the openstack support
>> > > for vhost-user with odl requires that ovs use /var/run/openvswtich as
>> > > its run dir regardless of the distro.
>> > >
>> > > Roberts proposal is trying to remove this restriction complies with
>> > > the debian/Ubuntu packaging scheme But conflits with the
>> > > centos/rhel/fedora packaging guidelines.
>> >
>> > Okay, I'll think about how to get this information out.
>> >
>> > There are, actually, two pieces of information that I think need to be
>> exposed.
>> > Please correct me if I'm wrong or feel free to add any other:
>> >
>> > 1. ovs current runtime directory
>> [Mooney, Sean K] yes this is needed
>> > 2. current dpdk status (active/inactive/unavailable, where active means
>> dpdk-
>> > init=true at startup, inactive means it was not true at startup, and
>> unavailable
>> > means no compile time support is enabled).
>> 
>> 
>> [Mooney, Sean K] this is not needed as it is reported already. In this context
>> we actually don't
>> Care if dpdk is available or not but rather if vhost-user is available. The
>> Open_vSwitch table contains
>> An iface_types field which report the available interface types. This is
>> updated by the ovs-vswitchd when
>> It starts. If the vswitch was compiled with dpdk but started with
>> dpdk_init=false then the dpdk interfaces will not
>> Be present in the list.  If the dpdk_init=true but dpdk was compiled with
>> vhost-cuse the the dpdkvhostcuse
>> interfaces type will be listed but the dpdkvhsotuser interface type will be
>> not.  Since our dependency is
>> on the presence of vhost-user support and not dpdk we use the iface_types
>> field in combination with checking
>> the ovs bridge datapath_type to determine if we should use vhost-user or
>> not.
>> 
>> >
>> > Are there any other bits of information?
>> > What interfaces do you have available for use (nw sockets, db, etc.)?
>> >
>> 
>> [Mooney, Sean K] odl has two interfaces with which it can communicate with
>> ovs.
>> the ovsdb protocol over tcp and openflow which is also over tcp I think. I do
>> not believe openflow
>> Can be used to query this information remotely however I may be wrong.
>> the ovsdb protocol can, provided the information is stored in the ovsdb.
>> 
>> > >> Now the question is how the third-party scripts can find out where
>> > >> actually vhost-user sockets are located?
>> > >>
>> > >> Br,
>> > >> Robert
>> _______________________________________________
>> dev mailing list
>> dev at openvswitch.org
>> http://openvswitch.org/mailman/listinfo/dev