[ovs-discuss] Inserting and using the compiled OVS kernel modules

Levente Csikor csikor at tmit.bme.hu
Sat Jan 12 13:08:47 UTC 2019


Alrighty then ;)

Thanks for all your help!

BR,
Levi 

On Fri, 2019-01-11 at 10:22 -0800, Gregory Rose wrote:
> On 1/11/2019 1:08 AM, Levente Csikor wrote:
> > I have installed the same OVS on another server, but with the same
> > properties (e.g., Ubunt 18.04, linux kernel 4.15.0-32-generic).
> > 
> > The make modules_install command now has been processed properly.
> > In particular,
> >   - the new module is still in the .../extra folder
> >   - but when I want to load the kernel module via modprobe this new
> > one
> > will be loaded by default
> >   - I still don't have /etc/modprobe.d/openvswitch.conf!
> > 
> > As I did not do the tricky 'renaming' steps this time, how can I
> > load
> > the old kernel module with modprobe - Should I again rely on the
> > currently non-existing /etc/modprobe.d/openvswitch.conf?
> > 
> > Thanks
> 
> You can't load the old kernel module with modprobe once you've
> installed 
> the out of tree OVS kernel
> modules.  There is no method for switching - you can delete the 
> ../extra/openvswitch kernel modules and
> then run depmod -a again to get modprobe to load the original, built-
> in 
> openvswitch kernel module.
> But then, obviously, you'll have to run 'make modules_install' again
> in 
> order to run the out of tree
> kernel modules.
> 
> If your configuration runs fine without the openvswitch.conf config
> file 
> then that's OK, it is only provided
> because sometimes it's required.
> 
> - Greg
> 
> > 
> > 
> > 
> > On Fri, 2019-01-11 at 09:14 +0100, Levente Csikor wrote:
> > > On Thu, 2019-01-10 at 09:34 -0800, Gregory Rose wrote:
> > > > On 1/10/2019 3:37 AM, Levente Csikor wrote:
> > > > > Issue has just been solved, sorry for the avalanche of
> > > > > questions
> > > > > :)
> > > > > 
> > > > > The solution was the following:
> > > > > I have created a directory for the new module:
> > > > > # mkdir /lib/modules/$(uname -r)/kernel/net/openvswitch2.10.0
> > > > > 
> > > > > Copy the compiled module(s) of the new openvswitch found in
> > > > > /lib/modules/$(uname -r)/extra
> > > > > # cp /lib/modules/$(uname -r)/extra/* /lib/modules/$(uname
> > > > > -r)/kernel/net/openvswitch2.10.0
> > > > > 
> > > > > I tried depmod -a (not sure if it was necessary
> > > > > # depmod -a
> > > > > it returned no error!
> > > > 
> > > > depmod -a is always necessary and will be executed when running
> > > > 'make
> > > > modules_install'.
> > > 
> > > OK, then it was indeed necessary.
> > > 
> > > > > Then, I have realized that the new kernel module's name is
> > > > > not
> > > > > openvswitch2.10.0, but openvswitch2.
> > > > > 
> > > > > So, now modinfo and modprobe sees it properly, and it can be
> > > > > loaded.
> > > > 
> > > > I'm glad you got it working but what you had to do seems very
> > > > strange.
> > > > Were you running 'make
> > > > modules_install' as root?  I didn't see any 'sudo' in front of
> > > > the
> > > > commands you were running.
> > > 
> > > Yes, I have done everything with sudo and/or as root.
> > > The reason why I needed to carry out these steps is because I did
> > > not
> > > want to remove the old openvswitch kernel module shipped with the
> > > kernel.
> > > And if I did not do this trick, the command modprobe openvswitch
> > > loaded
> > > the default openvswitch module instead of the newly installed one
> > > in
> > > the .../extra folder.
> > > 
> > > > What are the contents of your /etc/modprobe.d/openvswitch.conf?
> > > > 
> > > > Should look something like this:
> > > > 
> > > > $ cat openvswitch.conf
> > > > search extra
> > > > override openvswitch * extra
> > > > override openvswitch * weak-updates
> > > > override vport-geneve * extra
> > > > override vport-geneve * weak-updates
> > > > override vport-gre * extra
> > > > override vport-gre * weak-updates
> > > > override vport-lisp * extra
> > > > override vport-lisp * weak-updates
> > > > override vport-stt * extra
> > > > override vport-stt * weak-updates
> > > > override vport-vxlan * extra
> > > > override vport-vxlan * weak-updates
> > > 
> > > Maybe this is an issue!
> > > I do not have any openvswitch.conf in /etc/modprobe.d :S
> > > Should I have done it manually, or it should have been created
> > > during
> > > installing openvswitch (for the record, I have not issued the
> > > command
> > > make install, as I did not want to overwrite the already
> > > installed
> > > binaries either)?
> > > 
> > > 
> > > levi
> > > 
> > > 
> > > > - Greg
> > > > 
> > > > > Thanks,
> > > > > Levi
> > > > > 
> > > > > On Thu, 2019-01-10 at 11:19 +0100, Levente Csikor wrote:
> > > > > > Signing issue has been resolved by creating signing keys
> > > > > > and
> > > > > > name
> > > > > > them
> > > > > > as openvswitch wants to have them
> > > > > > 
> > > > > > Creating a key:
> > > > > > $ sudo openssl req -new -x509 -sha512 -newkey rsa:4096
> > > > > > -nodes
> > > > > > -keyout
> > > > > > key.pem -days 36500 -out certificate.pem
> > > > > > $ sudo cp key.pem /usr/src/linux-headers-$(uname
> > > > > > -r)/certs/signing_key.pem
> > > > > > $ sudo cp certificate.pem /usr/src/linux-headers-$(uname
> > > > > > -r)/certs/signing_key.x509
> > > > > > 
> > > > > > Now, sudo make modules_install has no issues:
> > > > > > cd datapath/linux && make modules_install
> > > > > > make[1]: Entering directory '/home/csikor/openvswitch-
> > > > > > 2.10.0/datapath/linux'
> > > > > > make -C /lib/modules/4.15.0-32-generic/build
> > > > > > M=/home/csikor/openvswitch-2.10.0/datapath/linux
> > > > > > modules_install
> > > > > > make[2]: Entering directory '/usr/src/linux-headers-4.15.0-
> > > > > > 32-
> > > > > > generic'
> > > > > >     INSTALL /home/csikor/openvswitch-
> > > > > > 2.10.0/datapath/linux/openvswitch.ko
> > > > > >     INSTALL /home/csikor/openvswitch-
> > > > > > 2.10.0/datapath/linux/vport-
> > > > > > geneve.ko
> > > > > >     INSTALL /home/csikor/openvswitch-
> > > > > > 2.10.0/datapath/linux/vport-
> > > > > > gre.ko
> > > > > >     INSTALL /home/csikor/openvswitch-
> > > > > > 2.10.0/datapath/linux/vport-
> > > > > > lisp.ko
> > > > > >     INSTALL /home/csikor/openvswitch-
> > > > > > 2.10.0/datapath/linux/vport-
> > > > > > stt.ko
> > > > > >     INSTALL /home/csikor/openvswitch-
> > > > > > 2.10.0/datapath/linux/vport-
> > > > > > vxlan.ko
> > > > > >     DEPMOD  4.15.0-32-generic
> > > > > > make[2]: Leaving directory '/usr/src/linux-headers-4.15.0-
> > > > > > 32-
> > > > > > generic'
> > > > > > depmod `sed -n 's/#define UTS_RELEASE "\([^"]*\)"/\1/p'
> > > > > > /lib/modules/4.15.0-32-
> > > > > > generic/build/include/generated/utsrelease.h`
> > > > > > make[1]: Leaving directory '/home/csikor/openvswitch-
> > > > > > 2.10.0/datapath/linux'
> > > > > > 
> > > > > > 
> > > > > > Other issues mentioned before still apply :(
> > > > > > 
> > > > > > cheers,
> > > > > > levi
> > > > > > 
> > > > > > On Thu, 2019-01-10 at 11:06 +0100, Levente Csikor wrote:
> > > > > > > Hi,
> > > > > > > I have updated the compiler, and tried gcc 7.3 and even
> > > > > > > 8.2
> > > > > > > (lates
> > > > > > > gcc-
> > > > > > > 7 and gcc-8 packages in Ubuntu 16.04).
> > > > > > > I have just gotten to know that gcc 7.3 is with retpoline
> > > > > > > so
> > > > > > > it
> > > > > > > should
> > > > > > > not have any issue.
> > > > > > > However, after the compilation is finished with no
> > > > > > > errors,
> > > > > > > insmod
> > > > > > > or
> > > > > > > more precisely dmesh still says the same:
> > > > > > > [ 6092.123271] Spectre V2 : System may be vulnerable to
> > > > > > > spectre
> > > > > > > v2
> > > > > > > [ 6092.123280] openvswitch: loading module not compiled
> > > > > > > with
> > > > > > > retpoline
> > > > > > > compiler.
> > > > > > > [ 6092.128378] openvswitch: Open vSwitch switching
> > > > > > > datapath
> > > > > > > 2.10.0
> > > > > > > 
> > > > > > > And lsmod|grep openvswitch still gives nothing back.
> > > > > > > This also implies that there is nothing in the
> > > > > > > /sys/modules/
> > > > > > > named
> > > > > > > as
> > > > > > > openvswitch :(
> > > > > > > 
> > > > > > > When I call sudo make modules_install in openvswitch
> > > > > > > source,
> > > > > > > it
> > > > > > > seems
> > > > > > > it succeeds but there are sigining issues:
> > > > > > > cd datapath/linux && make modules_install
> > > > > > > make[1]: Entering directory '/home/csikor/openvswitch-
> > > > > > > 2.10.0/datapath/linux'
> > > > > > > make -C /lib/modules/4.15.0-32-generic/build
> > > > > > > M=/home/csikor/openvswitch-2.10.0/datapath/linux
> > > > > > > modules_install
> > > > > > > make[2]: Entering directory '/usr/src/linux-headers-
> > > > > > > 4.15.0-
> > > > > > > 32-
> > > > > > > generic'
> > > > > > >     INSTALL /home/csikor/openvswitch-
> > > > > > > 2.10.0/datapath/linux/openvswitch.ko
> > > > > > > At main.c:160:
> > > > > > > - SSL error:02001002:system library:fopen:No such file or
> > > > > > > directory:
> > > > > > > ../crypto/bio/bss_file.c:74
> > > > > > > - SSL error:2006D080:BIO routines:BIO_new_file:no such
> > > > > > > file:
> > > > > > > ../crypto/bio/bss_file.c:81
> > > > > > > sign-file: certs/signing_key.pem: No such file or
> > > > > > > directory
> > > > > > >     INSTALL /home/csikor/openvswitch-
> > > > > > > 2.10.0/datapath/linux/vport-
> > > > > > > geneve.ko
> > > > > > > At main.c:160:
> > > > > > > - SSL error:02001002:system library:fopen:No such file or
> > > > > > > directory:
> > > > > > > ../crypto/bio/bss_file.c:74
> > > > > > > - SSL error:2006D080:BIO routines:BIO_new_file:no such
> > > > > > > file:
> > > > > > > ../crypto/bio/bss_file.c:81
> > > > > > > sign-file: certs/signing_key.pem: No such file or
> > > > > > > directory
> > > > > > >     INSTALL /home/csikor/openvswitch-
> > > > > > > 2.10.0/datapath/linux/vport-
> > > > > > > gre.ko
> > > > > > > At main.c:160:
> > > > > > > - SSL error:02001002:system library:fopen:No such file or
> > > > > > > directory:
> > > > > > > ../crypto/bio/bss_file.c:74
> > > > > > > - SSL error:2006D080:BIO routines:BIO_new_file:no such
> > > > > > > file:
> > > > > > > ../crypto/bio/bss_file.c:81
> > > > > > > sign-file: certs/signing_key.pem: No such file or
> > > > > > > directory
> > > > > > >     INSTALL /home/csikor/openvswitch-
> > > > > > > 2.10.0/datapath/linux/vport-
> > > > > > > lisp.ko
> > > > > > > At main.c:160:
> > > > > > > - SSL error:02001002:system library:fopen:No such file or
> > > > > > > directory:
> > > > > > > ../crypto/bio/bss_file.c:74
> > > > > > > - SSL error:2006D080:BIO routines:BIO_new_file:no such
> > > > > > > file:
> > > > > > > ../crypto/bio/bss_file.c:81
> > > > > > > sign-file: certs/signing_key.pem: No such file or
> > > > > > > directory
> > > > > > >     INSTALL /home/csikor/openvswitch-
> > > > > > > 2.10.0/datapath/linux/vport-
> > > > > > > stt.ko
> > > > > > > At main.c:160:
> > > > > > > - SSL error:02001002:system library:fopen:No such file or
> > > > > > > directory:
> > > > > > > ../crypto/bio/bss_file.c:74
> > > > > > > - SSL error:2006D080:BIO routines:BIO_new_file:no such
> > > > > > > file:
> > > > > > > ../crypto/bio/bss_file.c:81
> > > > > > > sign-file: certs/signing_key.pem: No such file or
> > > > > > > directory
> > > > > > >     INSTALL /home/csikor/openvswitch-
> > > > > > > 2.10.0/datapath/linux/vport-
> > > > > > > vxlan.ko
> > > > > > > At main.c:160:
> > > > > > > - SSL error:02001002:system library:fopen:No such file or
> > > > > > > directory:
> > > > > > > ../crypto/bio/bss_file.c:74
> > > > > > > - SSL error:2006D080:BIO routines:BIO_new_file:no such
> > > > > > > file:
> > > > > > > ../crypto/bio/bss_file.c:81
> > > > > > > sign-file: certs/signing_key.pem: No such file or
> > > > > > > directory
> > > > > > >     DEPMOD  4.15.0-32-generic
> > > > > > > make[2]: Leaving directory '/usr/src/linux-headers-
> > > > > > > 4.15.0-32-
> > > > > > > generic'
> > > > > > > depmod `sed -n 's/#define UTS_RELEASE "\([^"]*\)"/\1/p'
> > > > > > > /lib/modules/4.15.0-32-
> > > > > > > generic/build/include/generated/utsrelease.h`
> > > > > > > make[1]: Leaving directory '/home/csikor/openvswitch-
> > > > > > > 2.10.0/datapath/linux'
> > > > > > > 
> > > > > > > Should i somehow resolve this?
> > > > > > > If yes, how can I resolve this? Should I generate a
> > > > > > > signing_key.pem
> > > > > > > and
> > > > > > >    put it into somewhere (e.g., /usr/src/linux-headers-
> > > > > > > $(uname
> > > > > > > -r)/certs)?
> > > > > > > 
> > > > > > > On the other hand, an openvswitch.ko module now can be
> > > > > > > found
> > > > > > > at:
> > > > > > > lib/modules/4.15.0-32-generic/extra/openvswitch.ko
> > > > > > > 
> > > > > > > which has a 'last accessed timestamp' for today, so it
> > > > > > > has
> > > > > > > just
> > > > > > > been
> > > > > > > made by the make modules_install command.
> > > > > > > However, modprobe openvswitch still loads the original,
> > > > > > > standard
> > > > > > > linux
> > > > > > > kernel-provided module.
> > > > > > > How can I force modprobe to use the one in the .../extra/
> > > > > > > folder?
> > > > > > > I did not want to do any symlink tricks for the first
> > > > > > > sight,
> > > > > > > but
> > > > > > > maybe
> > > > > > > I will need to do that (?)
> > > > > > > 
> > > > > > > Thanks,
> > > > > > > levi
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > On Sat, 2019-01-05 at 08:34 -0800, Gregory Rose wrote:
> > > > > > > > On 1/5/2019 2:34 AM, Levente Csikor wrote:
> > > > > > > > > Yes, I thought so, however I don't see anything:
> > > > > > > > > 
> > > > > > > > > # modprobe openvswitch
> > > > > > > > > # dmesg |tail -n 8
> > > > > > > > > [250423.894258] PKCS#7 signature not signed with a
> > > > > > > > > trusted
> > > > > > > > > key
> > > > > > > > > [250423.894271] Spectre V2 : System may be vulnerable
> > > > > > > > > to
> > > > > > > > > spectre
> > > > > > > > > v2
> > > > > > > > > [250423.894273] openvswitch: loading module not
> > > > > > > > > compiled
> > > > > > > > > with
> > > > > > > > > retpoline
> > > > > > > > > compiler.
> > > > > > > > 
> > > > > > > > You need to upgrade your compiler.
> > > > > > > > 
> > > > > > > > > [250423.896970] openvswitch: Open vSwitch switching
> > > > > > > > > datapath
> > > > > > > > > 2.10.0
> > > > > > > > > [250423.897064] openvswitch: LISP tunneling driver
> > > > > > > > > [250423.897064] GRE over IPv4 demultiplexor driver
> > > > > > > > > [250423.898141] openvswitch: GRE over IPv4 tunneling
> > > > > > > > > driver
> > > > > > > > > [250423.898488] openvswitch: GRE over IPv6 tunneling
> > > > > > > > > driver
> > > > > > > > > # lsmod |grep openv
> > > > > > > > > #
> > > > > > > > 
> > > > > > > > That's truly odd, I've never seen it start to load like
> > > > > > > > that
> > > > > > > > and
> > > > > > > > then
> > > > > > > > just disappear.  However, the compiler
> > > > > > > > issue should be addressed.  Upgrade your compiler to
> > > > > > > > one
> > > > > > > > with
> > > > > > > > retpoline
> > > > > > > > support and then see if
> > > > > > > > that fixes the problem.
> > > > > > > > 
> > > > > > > > - Greg
> > > > > > > > 
> > > > > > > > > I have never encountered such a thing. When I used
> > > > > > > > > insmod,
> > > > > > > > > at
> > > > > > > > > least
> > > > > > > > > the
> > > > > > > > > module itself was definitely loaded.
> > > > > > > > > Any other thoughts on tracing what is happening and
> > > > > > > > > why
> > > > > > > > > the
> > > > > > > > > module
> > > > > > > > > is
> > > > > > > > > not loaded?
> > > > > > > > > 
> > > > > > > > > Cheers,
> > > > > > > > > levi
> > > > > > > > > 
> > > > > > > > > On Fri, 2019-01-04 at 09:25 -0800, Gregory Rose
> > > > > > > > > wrote:
> > > > > > > > > > On 1/3/2019 11:48 PM, Levente Csikor wrote:
> > > > > > > > > > > I do not have any openvswitch in /sys/module.
> > > > > > > > > > > I guess I need to have the module inserted
> > > > > > > > > > > correctly
> > > > > > > > > > > to
> > > > > > > > > > > have
> > > > > > > > > > > it,
> > > > > > > > > > > don't
> > > > > > > > > > > I?
> > > > > > > > > > 
> > > > > > > > > > If there is no openvswitch directory under
> > > > > > > > > > /sys/modules
> > > > > > > > > > then
> > > > > > > > > > the
> > > > > > > > > > module
> > > > > > > > > > isn't loaded.
> > > > > > > > > > 
> > > > > > > > > > > On the other hand, I have made the
> > > > > > > > > > > modules_install as
> > > > > > > > > > > well,
> > > > > > > > > > > and
> > > > > > > > > > > now
> > > > > > > > > > > when I say modprobe openvswitch, dmesg says the
> > > > > > > > > > > version
> > > > > > > > > > > 2.10.0
> > > > > > > > > > > has
> > > > > > > > > > > been
> > > > > > > > > > > loaded and no errors have been raised during
> > > > > > > > > > > insertion.
> > > > > > > > > > > However, lsmod|grep openvswitch returns
> > > > > > > > > > > nothing...strange.
> > > > > > > > > > > I guess the latter is the main cause of not
> > > > > > > > > > > having
> > > > > > > > > > > openvswitch
> > > > > > > > > > > in
> > > > > > > > > > > /sys/module/ :S
> > > > > > > > > > 
> > > > > > > > > > Correct.
> > > > > > > > > > 
> > > > > > > > > > > Any thoughts?
> > > > > > > > > > 
> > > > > > > > > > You should see something like this in your dmesg
> > > > > > > > > > output:
> > > > > > > > > > 
> > > > > > > > > > [167126.796728] openvswitch: Open vSwitch switching
> > > > > > > > > > datapath
> > > > > > > > > > 2.10.90
> > > > > > > > > > [167126.796869] openvswitch: LISP tunneling driver
> > > > > > > > > > [167126.796870] GRE over IPv4 demultiplexor driver
> > > > > > > > > > [167126.797172] openvswitch: GRE over IPv4
> > > > > > > > > > tunneling
> > > > > > > > > > driver
> > > > > > > > > > [167126.797406] openvswitch: GRE over IPv6
> > > > > > > > > > tunneling
> > > > > > > > > > driver
> > > > > > > > > > [167126.797526] openvswitch: Geneve tunneling
> > > > > > > > > > driver
> > > > > > > > > > [167126.797528] openvswitch: VxLAN tunneling driver
> > > > > > > > > > [167126.797529] openvswitch: STT tunneling driver
> > > > > > > > > > 
> > > > > > > > > > - Greg
> > > > > > > > > > > Cheers
> > > > > > > > > > > 
> > > > > > > > > > > On Thu, 2019-01-03 at 08:47 -0800, Gregory Rose
> > > > > > > > > > > wrote:
> > > > > > > > > > > > Run 'cat /sys/module/openvswitch/version' and
> > > > > > > > > > > > that
> > > > > > > > > > > > should
> > > > > > > > > > > > give
> > > > > > > > > > > > you
> > > > > > > > > > > > output like this:
> > > > > > > > > > > > 
> > > > > > > > > > > > # cat /sys/module/openvswitch/version
> > > > > > > > > > > > 2.10.1
> > > > > > > 
> > > > > > > _______________________________________________
> > > > > > > discuss mailing list
> > > > > > > discuss at openvswitch.org
> > > > > > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
> > > > > > 
> > > > > > _______________________________________________
> > > > > > discuss mailing list
> > > > > > discuss at openvswitch.org
> > > > > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
> 
> 


More information about the discuss mailing list