[ovs-dev] [PATCH] rhel: update udev rules to allow vfio access

[Aaron Conole]

Alex Williamson <alex.williamson at redhat.com> writes:

> On Thu, 18 Apr 2019 13:23:54 -0300
> Flavio Leitner <fbl at sysclose.org> wrote:
>
>> On Thu, Apr 18, 2019 at 11:05:28AM -0400, Aaron Conole wrote:
>> > On some systems, it's possible that the initialization of the misc chardev
>> > associated with /dev/vfio/vfio is delayed.  This happens on machines with
>> > large numbers of cores (at least 88+).  If this delay exceeds the time
>> > required for ovs-vswitchd to call the dpdk initialization routine, the
>> > permissions won't be updated and the open call will return EACCES.
>> > 
>> > To fix this, we explicitly allow global open.  This means any user may
>> > open() the vfio device before the vfio modules are initialized, thus
>> > triggering a module load.  The applications (including ovs-vswitchd)
>> > would be pended while the module loads.  This should be safe, as any user
>> > may open the vfio device once the module is loaded anyway, since the.
>> > module rewrites the permissions as 0666.
>> > 
>> > Signed-off-by: Aaron Conole <aconole at redhat.com>
>> > ---
>> >  rhel/usr_lib_udev_rules.d_91-vfio.rules | 1 +
>> >  1 file changed, 1 insertion(+)
>> > 
>> > diff --git a/rhel/usr_lib_udev_rules.d_91-vfio.rules
>> > b/rhel/usr_lib_udev_rules.d_91-vfio.rules
>> > index 8e34b2a2b..c0504ab5a 100644
>> > --- a/rhel/usr_lib_udev_rules.d_91-vfio.rules
>> > +++ b/rhel/usr_lib_udev_rules.d_91-vfio.rules
>> > @@ -1 +1,2 @@
>> >  ACTION=="add", SUBSYSTEM=="vfio*", GROUP="hugetlbfs", MODE="0660"
>> > +ACTION=="add", SUBSYSTEM=="misc", KERNEL=="vfio", MODE="0666"  
>> 
>> Wouldn't be better to fix this at the module to fix the permissions
>> before whatever is taking long to initialize?

It's a bit of chicken and egg.  The module needs to be ready before we can
open it, and we open it before it's ready because of the way DPDK is
designed.

There might be a better way of rewriting the DPDK initialization; it's a
longer path, and I'm not sure it's really worth it but maybe it's better?

> The stronger security policy is to not allow unprivileged users to
> trigger an operation which loads a module, so no, upper level system
> policy should relax that as necessary, but not the module.  VFIO is
> unavailable until enabled through privileged entities, so there's some
> degree that ovs is "jumping the gun" in making use of it, afaict.

Thanks.  I agree, system policy is the right place to relax the
requirement.  I also agree, the way DPDK gets initialized it would be
prematurely asking for vfio (since the module isn't even finished
loading).

I guess another approach could be to patch where the open() call
happens, to retry on EACCES for a specific amount of time (delaying
startup but allowing the module load to complete).  It's a different
sort of hack.  Maybe it's an acceptable compromise?

> Thanks,
>
> Alex