[ovs-discuss] Handling conf.db ownership on OVS_USER_ID changes

Numan Siddique nusiddiq at redhat.com
Tue May 7 16:46:11 UTC 2019


On Mon, May 6, 2019 at 6:04 PM Aaron Conole <aconole at redhat.com> wrote:

> Jaime Caamaño Ruiz <jcaamano at suse.de> writes:
>
> >> Agree. I will try to address this issue. I think we can have a
> >> separate
> >> run time/log directory for OVN. ovn-controller needs to talk to the
> >> local ovsdb-server
> >> and br-int.mgmt and other related socket interfaces, so it needs
> >> access
> >> to
> >> the /var/run/openvswitch/ folder. I think we can solve this.
> >
> > I have a (yet to be submitted patch) on my side that changes OVN to run
> > as the same user as OVS. Thought it as less disruptive than also
> > changing the log directory.
>
>
Great. Looking forward for the patch.

Thanks


> That sounds interesting.
>
> > openvswitch-ipsec still logs as root though and I dont see a way around
> > that so it probably needs to log to a different directory.
> >
> > Jaime.
> >
> > -----Original Message-----
> > From: Numan Siddique <nusiddiq at redhat.com>
> > To: Aaron Conole <aconole at redhat.com>
> > Cc: Jaime Caamaño Ruiz <jcaamano at suse.de>, ovs-discuss <ovs-discuss at ope
> > nvswitch.org>
> > Subject: Re: [ovs-discuss] Handling conf.db ownership on OVS_USER_ID
> > changes
> > Date: Thu, 2 May 2019 19:20:28 +0530
> >
> >
> >
> >
> > On Mon, Apr 29, 2019, 9:36 PM Aaron Conole <aconole at redhat.com> wrote:
> >> Jaime Caamaño Ruiz <jcaamano at suse.de> writes:
> >>
> >> >> As a "security concern" you mean something among the lines where
> >> one
> >> >> of ovs-* processes running under openvswitch user would go ahead
> >> and
> >> >> create a file with its owner that later one of ovn processes would
> >> >> blindly reuse without checking that it actually belongs to
> >> root:root?
> >> >
> >> > One example is that the OVS user could create link as any OVN log
> >> file
> >> > to other file owned by root and then OVN would write to that file.
> >>
> >> There's a long-standing issue of OVN running as root.  I don't see
> >> any
> >> reason it should.
> >
> > Agree. I will try to address this issue. I think we can have a separate
> > run time/log directory for OVN. ovn-controller needs to talk to the
> > local ovsdb-server
> > and br-int.mgmt and other related socket interfaces, so it needs access
> > to
> > the /var/run/openvswitch/ folder. I think we can solve this.
> >
> > Thanks
> > Numan
> >
> >> >> Can you give a more concrete example? I believe logrotate is
> >> running
> >> >> under root and should be able to rotate everything?
> >> >
> >> > The logrotate configuration for openvswitch logs has a 'su'
> >> directive
> >> > to run under the openvswitch user, precisely to prevent something
> >> > similar to the above. So it wont be able to rotate root owned logs
> >> in
> >> > the openvswitch directory. How it fails precisely depends on global
> >> > logrotate configuration. For example it will fail to create the new
> >> log
> >> > file after rotation if the 'create' directive is used. Or it may
> >> fail
> >> > to compress the rotated log file as it wont be able to read it.
> >> >
> >> > BR
> >> > Jaime
> >> >
> >> > -----Original Message-----
> >> > From: Ansis Atteka <ansisatteka at gmail.com>
> >> > To: jcaamano at suse.de
> >> > Cc: ovs-discuss at openvswitch.org, Ben Pfaff <blp at ovn.org>
> >> > Subject: Re: [ovs-discuss] Handling conf.db ownership on
> >> OVS_USER_ID
> >> > changes
> >> > Date: Wed, 24 Apr 2019 19:07:24 -0700
> >> >
> >> > On Tue, 23 Apr 2019 at 09:30, Jaime Caamaño Ruiz <jcaamano at suse.de>
> >> > wrote:
> >> >>
> >> >> Hello
> >> >>
> >> >> So the non root owned log directory (and run directory) is shared
> >> >> between non root OVS processes and root OVN processes. Doesn't
> >> this
> >> >> raise some security concerns?
> >> >
> >> > As a "security concern" you mean something among the lines where
> >> one
> >> > of ovs-* processes running under openvswitch user would go ahead
> >> and
> >> > create a file with its owner that later one of ovn processes would
> >> > blindly reuse without checking that it actually belongs to
> >> root:root?
> >> >
> >> >
> >> >>
> >> >> Also, since logrotate rotates the logs with the OVS user, it may
> >> fail
> >> >> to some extent to rotate the root owned OVN log files.
> >> Consequences
> >> >> vary but in it's current state some logging might be lost. This
> >> could
> >> >> be improved but I would guess that the approprioate thing to do is
> >> to
> >> >> etiher use a different log directory for OVN or make its processes
> >> >> run
> >> >> with the OVS user also. Has any of this been considered?
> >> >
> >> > Can you give a more concrete example? I believe logrotate is
> >> running
> >> > under root and should be able to rotate everything?
> >> >
> >> >
> >> >>
> >> >> BR
> >> >> Jaime.
> >> >>
> >> >>
> >> >> -----Original Message-----
> >> >> From: Jaime Caamaño Ruiz <jcaamano at suse.de>
> >> >> Reply-to: jcaamano at suse.com
> >> >> To: Ansis Atteka <ansisatteka at gmail.com>
> >> >> Cc: ovs-discuss at openvswitch.org, Ben Pfaff <blp at ovn.org>
> >> >> Subject: Re: [ovs-discuss] Handling conf.db ownership on
> >> OVS_USER_ID
> >> >> changes
> >> >> Date: Wed, 17 Apr 2019 12:52:32 +0200
> >> >>
> >> >> > You also need to chown /var/log/openvswitch.*.log files.
> >> >>
> >> >> OVS seems to be already handling this. I dont know the details but
> >> I
> >> >> guess that before dropping capabilities, OVS chowns these by
> >> itself.
> >> >>
> >> >> > However,
> >> >> > what about other daemons, like ovn? Do they share run time dir?
> >> >> > Haven't looked into this in a while and don't have a setup to
> >> check
> >> >> > myself.
> >> >>
> >> >> The permisions of the openvswitch run directory are already being
> >> >> handled by the service unit file. There is read access to files
> >> >> created
> >> >> there by OVS and AFAIK what it ultimately makes it a no problem
> >> for
> >> >> OVN
> >> >> is that it still runs as root with the provided unit files. If
> >> anyone
> >> >> changes this to a user different than the openvswitch user, then
> >> that
> >> >> is an already existing issue. Am I missing something here?
> >> >>
> >> >> Agree with you other considerations.
> >> >>
> >> >> BR
> >> >> Jaime.
> >> >>
> >> >> -----Original Message-----
> >> >> From: Ansis Atteka <ansisatteka at gmail.com>
> >> >> To: jcaamano at suse.de
> >> >> Cc: ovs-discuss at openvswitch.org, Ben Pfaff <blp at ovn.org>
> >> >> Subject: Re: [ovs-discuss] Handling conf.db ownership on
> >> OVS_USER_ID
> >> >> changes
> >> >> Date: Tue, 16 Apr 2019 18:21:48 -0700
> >> >>
> >> >> On Tue, 16 Apr 2019 at 17:20, Jaime Caamaño Ruiz <jcaamano at suse.de
> >> >
> >> >> wrote:
> >> >> >
> >> >> > My intention was doing it at systemd unit (prestart) for file
> >> >> > conf.db
> >> >> > (and .conf.db.~lock~ that stays around) only.
> >> >>
> >> >> You also need to chown /var/log/openvswitch.*.log files.
> >> >> >
> >> >> > I was not thinking on absolutely fool proof mechanism, among
> >> other
> >> >> > things because the admin might have customized the location for
> >> the
> >> >> > database. Also, updating and restarting the service are things
> >> that
> >> >> > would usually be monitored. So best effort.
> >> >>
> >> >> make sure that best effort solution informs admin in case of error
> >> so
> >> >> that he does not have to go file by file to find the offending
> >> one.
> >> >>
> >> >> >
> >> >> > At systemd unit prestart the service is stopped and OVS nor
> >> nobody
> >> >> > should really be messing with the database files owned by OVS.
> >> If
> >> >> > the
> >> >> > spec file is changing things back to root unappropriately then
> >> that
> >> >> > should be changed too.
> >> >>
> >> >> Yes.
> >> >>
> >> >> >
> >> >> > The runtime /run/openvswitch directory is also controlled by
> >> >> > systemd
> >> >> > and at least on my system is removed if the service is not
> >> active,
> >> >> > either by graceful or ungraceful exit.
> >> >>
> >> >> I somewhat agree with this, because back in those days we were
> >> using
> >> >> SystemV where the init.d managed lifecycle of run directory..
> >> >> However,
> >> >> what about other daemons, like ovn? Do they share run time dir?
> >> >> Haven't looked into this in a while and don't have a setup to
> >> check
> >> >> myself.
> >> >>
> >> >> >
> >> >> > I guess we will need to look for specific issues. Any hint to
> >> find
> >> >> > that
> >> >> > patch?
> >> >>
> >> >> 1. test upgrade to your patched OVS and also from your patched OVS
> >> to
> >> >> another future version
> >> >> 2. if you change existing spec files that are shared with on
> >> Fedora
> >> >> or
> >> >> RHEL/CentOS make sure you don't break anything for them.
> >> >> 3. prefer consistency w.r.t. default behavior across deb and rpm
> >> >> packages (if you change user by default consider to do that for
> >> all
> >> >> platforms)
> >> >> 4. besides database you will also need to set right access bits to
> >> >> log
> >> >> files. On my Ubuntu it is "-rw-r----- 1 root adm " which means:
> >> >> 4.1. you have to chown it; OR
> >> >> 4.2. add openvswitch user to adm group and chmod that file
> >> >> 5. test logrotation. Maybe you can use a one forced logrotation
> >> >> invocation to change the ownrer.
> >> >> 6. check that other stuff like ovs-monitor-ipsec is not affected.
> >> It
> >> >> stil need to talk with StrongSwan or libreswan that best to my
> >> >> knowledge still runs as root.
> >> >> 7. if you use USER= then:
> >> >> 7.1. For Linux datapath make sure you retain CAP_NET_ADMIN Linux
> >> >> Capabilities
> >> >> 7.2. I don't know if there are special privileges you need to
> >> retain
> >> >> for DPDK. And in future possibly AF_XDP.
> >> >> 8. I haven't looked in a while but check if ovn (that has its own
> >> >> Systemd service file) shares run dir with OVS
> >> >>
> >> >> >
> >> >> > BR
> >> >> > Jaime.
> >> >> >
> >> >> > -----Original Message-----
> >> >> > From: Ansis Atteka <ansisatteka at gmail.com>
> >> >> > To: jcaamano at suse.de
> >> >> > Cc: ovs-discuss at openvswitch.org, Ben Pfaff <blp at ovn.org>
> >> >> > Subject: Re: [ovs-discuss] Handling conf.db ownership on
> >> >> > OVS_USER_ID
> >> >> > changes
> >> >> > Date: Tue, 16 Apr 2019 16:11:44 -0700
> >> >> >
> >> >> > On Tue, 16 Apr 2019 at 15:36, Jaime Caamaño Ruiz <jcaamano at suse.
> >> de>
> >> >> > wrote:
> >> >> > >
> >> >> > > > On any given system, I would expect ovsdb-server to run as
> >> the
> >> >> > > > same
> >> >> > > > user
> >> >> > > > every time.  Is there a reason to sometimes use a different
> >> >> > > > user?
> >> >> > >
> >> >> > > Well, changing from root which was the only supported option
> >> in
> >> >> > > the
> >> >> > > past and current default to a different user (like the
> >> suggested
> >> >> > > openvswitch), this one time, might be somewhat common. And the
> >> >> > > scenario
> >> >> > > that I am looking at is suporting a pianless upgrade through
> >> it.
> >> >> >
> >> >> > Actually debian packages still by default use "root" user. Not
> >> >> > "openvswitch".
> >> >> >
> >> >> > Long time ago there was an attempt to change the default user
> >> >> > across
> >> >> > all different flavor packages to openvswitch (you can lookup the
> >> >> > patch
> >> >> > on mailing list). However, as you noticed upgrades are tricky.
> >> >> > Hence
> >> >> > packages built from our debian/rules and
> >> rhel/openvswitch.spec.in
> >> >> > files still use "root" as default user. The packages built with
> >> >> > rhel/openvswitch-fedora.spec.in are an exception where the user
> >> >> > indeed
> >> >> > is "openvswitch". Unless you passed --without libcapng flag to
> >> >> > rpmbuild invocation. Then the user would still be "root".
> >> >> >
> >> >> > Here are the difficulties with automating the change of file
> >> >> > ownership:
> >> >> > 1. from which context to change ownership? As chown must be
> >> invoked
> >> >> > from something that runs under root then there is not much
> >> choice -
> >> >> > either package installation time (%post scriptlet) or daemon
> >> >> > startup
> >> >> > time (assuming you are not using Systemd's USER= feature in spec
> >> >> > file)
> >> >> > or something that admin explicitly has to invoke from bash as
> >> >> > "root".
> >> >> > 1.1. if it is package installation time then you can't update
> >> user
> >> >> > gracefully at daemon init time. It will be possible only at
> >> >> > installation time.
> >> >> > 1.2. if it is installation time then you can run into weird race
> >> >> > conditions where some code that was executed under root (as
> >> package
> >> >> > %post scriptlets) or previous instance of OVS that was still
> >> >> > running
> >> >> > as root could change ownership back to root for some files and
> >> >> > cause
> >> >> > unexpected "permission denied" issues (the patch I mentioned was
> >> >> > affected with these rare race conditions)
> >> >> > 2. for which files to change ownership? What happens if Unix
> >> domain
> >> >> > socket file remained on fileystem with "root" permissions when
> >> OVS
> >> >> > was
> >> >> > abruptly killed? What if someone else put a file under ovs
> >> >> > directories
> >> >> > with non-root user - should you blindly chown that too? What if
> >> >> > %post
> >> >> > scriptlet that runs under root non-atomically created a file and
> >> in
> >> >> > next line chown() it back - is there a window for race? Should
> >> you
> >> >> > follow directories/symlinks recursively? Changing user for
> >> conf.db
> >> >> > file is not enough...
> >> >> >
> >> >> > I am not saying it is impossible to change user gracefully on
> >> >> > upgrades. I am saying that code may have to be reorganized quite
> >> a
> >> >> > bit
> >> >> > to eliminate the issues I raised above.
> >> >> >
> >> >> > Alternatively, for Centos, RHEL and Fedora one can use SElinux
> >> to
> >> >> > confine processes and leave them running as root (instead of
> >> >> > running
> >> >> > them under limited privilege user). I believe on SUSE (and
> >> Debian)
> >> >> > once could use AppArmor to achieve the same results. At least
> >> for
> >> >> > SElinux it was a little bit easier to make upgrade seamless than
> >> >> > with
> >> >> > Linux user confinement approach.
> >> >> >
> >> >> > >
> >> >> > > So if that makes sense to you, I will go ahead with the patch.
> >> >> > >
> >> >> > > BR
> >> >> > > Jaime.
> >> >> > >
> >> >> > > -----Original Message-----
> >> >> > > From: Ben Pfaff <blp at ovn.org>
> >> >> > > To: jcaamano at suse.de
> >> >> > > Cc: ovs-discuss at openvswitch.org
> >> >> > > Subject: Re: [ovs-discuss] Handling conf.db ownership on
> >> >> > > OVS_USER_ID
> >> >> > > changes
> >> >> > > Date: Tue, 16 Apr 2019 15:25:24 -0700
> >> >> > >
> >> >> > > On Tue, Apr 16, 2019 at 11:32:21PM +0200, Jaime Caamaño Ruiz
> >> >> > > wrote:
> >> >> > > > When sysconfig OVS_USER_ID is changed to a different user,
> >> it
> >> >> > > > requires
> >> >> > > > a manual ownership change of the OVS conf.db database if
> >> >> > > > existing
> >> >> > > > or
> >> >> > > > otherwise ovsdb-server will fail to (re)start. I was
> >> wondering
> >> >> > > > if
> >> >> > > > I
> >> >> > > > am
> >> >> > > > missing any particular reason why this is change of
> >> ownership
> >> >> > > > is
> >> >> > > > not
> >> >> > > > automatically being handled through the service unit file as
> >> it
> >> >> > > > is
> >> >> > > > being done with other items.
> >> >> > >
> >> >> > > On any given system, I would expect ovsdb-server to run as the
> >> >> > > same
> >> >> > > user
> >> >> > > every time.  Is there a reason to sometimes use a different
> >> user?
> >> >> > >
> >> >> > > Or, perhaps you are saying that, just before invoking ovsdb-
> >> >> > > server,
> >> >> > > the
> >> >> > > service unit should chown (or whatever) the database file to
> >> the
> >> >> > > user
> >> >> > > that it is going to use to invoke ovsdb-server.  That might be
> >> >> > > reasonable, but no one has thought to do it yet.  Do you want
> >> to
> >> >> > > submit
> >> >> > > a patch?
> >> >> > >
> >> >> > > Thanks,
> >> >> > >
> >> >> > > Ben.
> >> >> > >
> >> >> > > _______________________________________________
> >> >> > > discuss mailing list
> >> >> > > discuss at openvswitch.org
> >> >> > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
> >> >> >
> >> >> >
> >> >>
> >> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > discuss mailing list
> >> > discuss at openvswitch.org
> >> > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
> >> _______________________________________________
> >> discuss mailing list
> >> discuss at openvswitch.org
> >> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20190507/a111e1e6/attachment-0001.html>


More information about the discuss mailing list