[ovs-discuss] Inserting and using the compiled OVS kernel modules

Gregory Rose gvrose8192 at gmail.com
Fri Jan 11 18:22:38 UTC 2019 

On 1/11/2019 1:08 AM, Levente Csikor wrote:
> I have installed the same OVS on another server, but with the same
> properties (e.g., Ubunt 18.04, linux kernel 4.15.0-32-generic).
>
> The make modules_install command now has been processed properly.
> In particular,
>   - the new module is still in the .../extra folder
>   - but when I want to load the kernel module via modprobe this new one
> will be loaded by default
>   - I still don't have /etc/modprobe.d/openvswitch.conf!
>
> As I did not do the tricky 'renaming' steps this time, how can I load
> the old kernel module with modprobe - Should I again rely on the
> currently non-existing /etc/modprobe.d/openvswitch.conf?
>
> Thanks

You can't load the old kernel module with modprobe once you've installed 
the out of tree OVS kernel
modules.  There is no method for switching - you can delete the 
../extra/openvswitch kernel modules and
then run depmod -a again to get modprobe to load the original, built-in 
openvswitch kernel module.
But then, obviously, you'll have to run 'make modules_install' again in 
order to run the out of tree
kernel modules.

If your configuration runs fine without the openvswitch.conf config file 
then that's OK, it is only provided
because sometimes it's required.

- Greg

>
>
>
> On Fri, 2019-01-11 at 09:14 +0100, Levente Csikor wrote:
>> On Thu, 2019-01-10 at 09:34 -0800, Gregory Rose wrote:
>>> On 1/10/2019 3:37 AM, Levente Csikor wrote:
>>>> Issue has just been solved, sorry for the avalanche of questions
>>>> :)
>>>>
>>>> The solution was the following:
>>>> I have created a directory for the new module:
>>>> # mkdir /lib/modules/$(uname -r)/kernel/net/openvswitch2.10.0
>>>>
>>>> Copy the compiled module(s) of the new openvswitch found in
>>>> /lib/modules/$(uname -r)/extra
>>>> # cp /lib/modules/$(uname -r)/extra/* /lib/modules/$(uname
>>>> -r)/kernel/net/openvswitch2.10.0
>>>>
>>>> I tried depmod -a (not sure if it was necessary
>>>> # depmod -a
>>>> it returned no error!
>>> depmod -a is always necessary and will be executed when running
>>> 'make
>>> modules_install'.
>> OK, then it was indeed necessary.
>>
>>>> Then, I have realized that the new kernel module's name is not
>>>> openvswitch2.10.0, but openvswitch2.
>>>>
>>>> So, now modinfo and modprobe sees it properly, and it can be
>>>> loaded.
>>> I'm glad you got it working but what you had to do seems very
>>> strange.
>>> Were you running 'make
>>> modules_install' as root?  I didn't see any 'sudo' in front of the
>>> commands you were running.
>> Yes, I have done everything with sudo and/or as root.
>> The reason why I needed to carry out these steps is because I did not
>> want to remove the old openvswitch kernel module shipped with the
>> kernel.
>> And if I did not do this trick, the command modprobe openvswitch
>> loaded
>> the default openvswitch module instead of the newly installed one in
>> the .../extra folder.
>>
>>> What are the contents of your /etc/modprobe.d/openvswitch.conf?
>>>
>>> Should look something like this:
>>>
>>> $ cat openvswitch.conf
>>> search extra
>>> override openvswitch * extra
>>> override openvswitch * weak-updates
>>> override vport-geneve * extra
>>> override vport-geneve * weak-updates
>>> override vport-gre * extra
>>> override vport-gre * weak-updates
>>> override vport-lisp * extra
>>> override vport-lisp * weak-updates
>>> override vport-stt * extra
>>> override vport-stt * weak-updates
>>> override vport-vxlan * extra
>>> override vport-vxlan * weak-updates
>> Maybe this is an issue!
>> I do not have any openvswitch.conf in /etc/modprobe.d :S
>> Should I have done it manually, or it should have been created during
>> installing openvswitch (for the record, I have not issued the command
>> make install, as I did not want to overwrite the already installed
>> binaries either)?
>>
>>
>> levi
>>
>>
>>> - Greg
>>>
>>>> Thanks,
>>>> Levi
>>>>
>>>> On Thu, 2019-01-10 at 11:19 +0100, Levente Csikor wrote:
>>>>> Signing issue has been resolved by creating signing keys and
>>>>> name
>>>>> them
>>>>> as openvswitch wants to have them
>>>>>
>>>>> Creating a key:
>>>>> $ sudo openssl req -new -x509 -sha512 -newkey rsa:4096 -nodes
>>>>> -keyout
>>>>> key.pem -days 36500 -out certificate.pem
>>>>> $ sudo cp key.pem /usr/src/linux-headers-$(uname
>>>>> -r)/certs/signing_key.pem
>>>>> $ sudo cp certificate.pem /usr/src/linux-headers-$(uname
>>>>> -r)/certs/signing_key.x509
>>>>>
>>>>> Now, sudo make modules_install has no issues:
>>>>> cd datapath/linux && make modules_install
>>>>> make[1]: Entering directory '/home/csikor/openvswitch-
>>>>> 2.10.0/datapath/linux'
>>>>> make -C /lib/modules/4.15.0-32-generic/build
>>>>> M=/home/csikor/openvswitch-2.10.0/datapath/linux
>>>>> modules_install
>>>>> make[2]: Entering directory '/usr/src/linux-headers-4.15.0-32-
>>>>> generic'
>>>>>     INSTALL /home/csikor/openvswitch-
>>>>> 2.10.0/datapath/linux/openvswitch.ko
>>>>>     INSTALL /home/csikor/openvswitch-
>>>>> 2.10.0/datapath/linux/vport-
>>>>> geneve.ko
>>>>>     INSTALL /home/csikor/openvswitch-
>>>>> 2.10.0/datapath/linux/vport-
>>>>> gre.ko
>>>>>     INSTALL /home/csikor/openvswitch-
>>>>> 2.10.0/datapath/linux/vport-
>>>>> lisp.ko
>>>>>     INSTALL /home/csikor/openvswitch-
>>>>> 2.10.0/datapath/linux/vport-
>>>>> stt.ko
>>>>>     INSTALL /home/csikor/openvswitch-
>>>>> 2.10.0/datapath/linux/vport-
>>>>> vxlan.ko
>>>>>     DEPMOD  4.15.0-32-generic
>>>>> make[2]: Leaving directory '/usr/src/linux-headers-4.15.0-32-
>>>>> generic'
>>>>> depmod `sed -n 's/#define UTS_RELEASE "\([^"]*\)"/\1/p'
>>>>> /lib/modules/4.15.0-32-
>>>>> generic/build/include/generated/utsrelease.h`
>>>>> make[1]: Leaving directory '/home/csikor/openvswitch-
>>>>> 2.10.0/datapath/linux'
>>>>>
>>>>>
>>>>> Other issues mentioned before still apply :(
>>>>>
>>>>> cheers,
>>>>> levi
>>>>>
>>>>> On Thu, 2019-01-10 at 11:06 +0100, Levente Csikor wrote:
>>>>>> Hi,
>>>>>> I have updated the compiler, and tried gcc 7.3 and even 8.2
>>>>>> (lates
>>>>>> gcc-
>>>>>> 7 and gcc-8 packages in Ubuntu 16.04).
>>>>>> I have just gotten to know that gcc 7.3 is with retpoline so
>>>>>> it
>>>>>> should
>>>>>> not have any issue.
>>>>>> However, after the compilation is finished with no errors,
>>>>>> insmod
>>>>>> or
>>>>>> more precisely dmesh still says the same:
>>>>>> [ 6092.123271] Spectre V2 : System may be vulnerable to
>>>>>> spectre
>>>>>> v2
>>>>>> [ 6092.123280] openvswitch: loading module not compiled with
>>>>>> retpoline
>>>>>> compiler.
>>>>>> [ 6092.128378] openvswitch: Open vSwitch switching datapath
>>>>>> 2.10.0
>>>>>>
>>>>>> And lsmod|grep openvswitch still gives nothing back.
>>>>>> This also implies that there is nothing in the /sys/modules/
>>>>>> named
>>>>>> as
>>>>>> openvswitch :(
>>>>>>
>>>>>> When I call sudo make modules_install in openvswitch source,
>>>>>> it
>>>>>> seems
>>>>>> it succeeds but there are sigining issues:
>>>>>> cd datapath/linux && make modules_install
>>>>>> make[1]: Entering directory '/home/csikor/openvswitch-
>>>>>> 2.10.0/datapath/linux'
>>>>>> make -C /lib/modules/4.15.0-32-generic/build
>>>>>> M=/home/csikor/openvswitch-2.10.0/datapath/linux
>>>>>> modules_install
>>>>>> make[2]: Entering directory '/usr/src/linux-headers-4.15.0-
>>>>>> 32-
>>>>>> generic'
>>>>>>     INSTALL /home/csikor/openvswitch-
>>>>>> 2.10.0/datapath/linux/openvswitch.ko
>>>>>> At main.c:160:
>>>>>> - SSL error:02001002:system library:fopen:No such file or
>>>>>> directory:
>>>>>> ../crypto/bio/bss_file.c:74
>>>>>> - SSL error:2006D080:BIO routines:BIO_new_file:no such file:
>>>>>> ../crypto/bio/bss_file.c:81
>>>>>> sign-file: certs/signing_key.pem: No such file or directory
>>>>>>     INSTALL /home/csikor/openvswitch-
>>>>>> 2.10.0/datapath/linux/vport-
>>>>>> geneve.ko
>>>>>> At main.c:160:
>>>>>> - SSL error:02001002:system library:fopen:No such file or
>>>>>> directory:
>>>>>> ../crypto/bio/bss_file.c:74
>>>>>> - SSL error:2006D080:BIO routines:BIO_new_file:no such file:
>>>>>> ../crypto/bio/bss_file.c:81
>>>>>> sign-file: certs/signing_key.pem: No such file or directory
>>>>>>     INSTALL /home/csikor/openvswitch-
>>>>>> 2.10.0/datapath/linux/vport-
>>>>>> gre.ko
>>>>>> At main.c:160:
>>>>>> - SSL error:02001002:system library:fopen:No such file or
>>>>>> directory:
>>>>>> ../crypto/bio/bss_file.c:74
>>>>>> - SSL error:2006D080:BIO routines:BIO_new_file:no such file:
>>>>>> ../crypto/bio/bss_file.c:81
>>>>>> sign-file: certs/signing_key.pem: No such file or directory
>>>>>>     INSTALL /home/csikor/openvswitch-
>>>>>> 2.10.0/datapath/linux/vport-
>>>>>> lisp.ko
>>>>>> At main.c:160:
>>>>>> - SSL error:02001002:system library:fopen:No such file or
>>>>>> directory:
>>>>>> ../crypto/bio/bss_file.c:74
>>>>>> - SSL error:2006D080:BIO routines:BIO_new_file:no such file:
>>>>>> ../crypto/bio/bss_file.c:81
>>>>>> sign-file: certs/signing_key.pem: No such file or directory
>>>>>>     INSTALL /home/csikor/openvswitch-
>>>>>> 2.10.0/datapath/linux/vport-
>>>>>> stt.ko
>>>>>> At main.c:160:
>>>>>> - SSL error:02001002:system library:fopen:No such file or
>>>>>> directory:
>>>>>> ../crypto/bio/bss_file.c:74
>>>>>> - SSL error:2006D080:BIO routines:BIO_new_file:no such file:
>>>>>> ../crypto/bio/bss_file.c:81
>>>>>> sign-file: certs/signing_key.pem: No such file or directory
>>>>>>     INSTALL /home/csikor/openvswitch-
>>>>>> 2.10.0/datapath/linux/vport-
>>>>>> vxlan.ko
>>>>>> At main.c:160:
>>>>>> - SSL error:02001002:system library:fopen:No such file or
>>>>>> directory:
>>>>>> ../crypto/bio/bss_file.c:74
>>>>>> - SSL error:2006D080:BIO routines:BIO_new_file:no such file:
>>>>>> ../crypto/bio/bss_file.c:81
>>>>>> sign-file: certs/signing_key.pem: No such file or directory
>>>>>>     DEPMOD  4.15.0-32-generic
>>>>>> make[2]: Leaving directory '/usr/src/linux-headers-4.15.0-32-
>>>>>> generic'
>>>>>> depmod `sed -n 's/#define UTS_RELEASE "\([^"]*\)"/\1/p'
>>>>>> /lib/modules/4.15.0-32-
>>>>>> generic/build/include/generated/utsrelease.h`
>>>>>> make[1]: Leaving directory '/home/csikor/openvswitch-
>>>>>> 2.10.0/datapath/linux'
>>>>>>
>>>>>> Should i somehow resolve this?
>>>>>> If yes, how can I resolve this? Should I generate a
>>>>>> signing_key.pem
>>>>>> and
>>>>>>    put it into somewhere (e.g., /usr/src/linux-headers-$(uname
>>>>>> -r)/certs)?
>>>>>>
>>>>>> On the other hand, an openvswitch.ko module now can be found
>>>>>> at:
>>>>>> lib/modules/4.15.0-32-generic/extra/openvswitch.ko
>>>>>>
>>>>>> which has a 'last accessed timestamp' for today, so it has
>>>>>> just
>>>>>> been
>>>>>> made by the make modules_install command.
>>>>>> However, modprobe openvswitch still loads the original,
>>>>>> standard
>>>>>> linux
>>>>>> kernel-provided module.
>>>>>> How can I force modprobe to use the one in the .../extra/
>>>>>> folder?
>>>>>> I did not want to do any symlink tricks for the first sight,
>>>>>> but
>>>>>> maybe
>>>>>> I will need to do that (?)
>>>>>>
>>>>>> Thanks,
>>>>>> levi
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sat, 2019-01-05 at 08:34 -0800, Gregory Rose wrote:
>>>>>>> On 1/5/2019 2:34 AM, Levente Csikor wrote:
>>>>>>>> Yes, I thought so, however I don't see anything:
>>>>>>>>
>>>>>>>> # modprobe openvswitch
>>>>>>>> # dmesg |tail -n 8
>>>>>>>> [250423.894258] PKCS#7 signature not signed with a
>>>>>>>> trusted
>>>>>>>> key
>>>>>>>> [250423.894271] Spectre V2 : System may be vulnerable to
>>>>>>>> spectre
>>>>>>>> v2
>>>>>>>> [250423.894273] openvswitch: loading module not compiled
>>>>>>>> with
>>>>>>>> retpoline
>>>>>>>> compiler.
>>>>>>> You need to upgrade your compiler.
>>>>>>>
>>>>>>>> [250423.896970] openvswitch: Open vSwitch switching
>>>>>>>> datapath
>>>>>>>> 2.10.0
>>>>>>>> [250423.897064] openvswitch: LISP tunneling driver
>>>>>>>> [250423.897064] GRE over IPv4 demultiplexor driver
>>>>>>>> [250423.898141] openvswitch: GRE over IPv4 tunneling
>>>>>>>> driver
>>>>>>>> [250423.898488] openvswitch: GRE over IPv6 tunneling
>>>>>>>> driver
>>>>>>>> # lsmod |grep openv
>>>>>>>> #
>>>>>>> That's truly odd, I've never seen it start to load like
>>>>>>> that
>>>>>>> and
>>>>>>> then
>>>>>>> just disappear.  However, the compiler
>>>>>>> issue should be addressed.  Upgrade your compiler to one
>>>>>>> with
>>>>>>> retpoline
>>>>>>> support and then see if
>>>>>>> that fixes the problem.
>>>>>>>
>>>>>>> - Greg
>>>>>>>
>>>>>>>> I have never encountered such a thing. When I used
>>>>>>>> insmod,
>>>>>>>> at
>>>>>>>> least
>>>>>>>> the
>>>>>>>> module itself was definitely loaded.
>>>>>>>> Any other thoughts on tracing what is happening and why
>>>>>>>> the
>>>>>>>> module
>>>>>>>> is
>>>>>>>> not loaded?
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> levi
>>>>>>>>
>>>>>>>> On Fri, 2019-01-04 at 09:25 -0800, Gregory Rose wrote:
>>>>>>>>> On 1/3/2019 11:48 PM, Levente Csikor wrote:
>>>>>>>>>> I do not have any openvswitch in /sys/module.
>>>>>>>>>> I guess I need to have the module inserted correctly
>>>>>>>>>> to
>>>>>>>>>> have
>>>>>>>>>> it,
>>>>>>>>>> don't
>>>>>>>>>> I?
>>>>>>>>> If there is no openvswitch directory under /sys/modules
>>>>>>>>> then
>>>>>>>>> the
>>>>>>>>> module
>>>>>>>>> isn't loaded.
>>>>>>>>>
>>>>>>>>>> On the other hand, I have made the modules_install as
>>>>>>>>>> well,
>>>>>>>>>> and
>>>>>>>>>> now
>>>>>>>>>> when I say modprobe openvswitch, dmesg says the
>>>>>>>>>> version
>>>>>>>>>> 2.10.0
>>>>>>>>>> has
>>>>>>>>>> been
>>>>>>>>>> loaded and no errors have been raised during
>>>>>>>>>> insertion.
>>>>>>>>>> However, lsmod|grep openvswitch returns
>>>>>>>>>> nothing...strange.
>>>>>>>>>> I guess the latter is the main cause of not having
>>>>>>>>>> openvswitch
>>>>>>>>>> in
>>>>>>>>>> /sys/module/ :S
>>>>>>>>> Correct.
>>>>>>>>>
>>>>>>>>>> Any thoughts?
>>>>>>>>> You should see something like this in your dmesg
>>>>>>>>> output:
>>>>>>>>>
>>>>>>>>> [167126.796728] openvswitch: Open vSwitch switching
>>>>>>>>> datapath
>>>>>>>>> 2.10.90
>>>>>>>>> [167126.796869] openvswitch: LISP tunneling driver
>>>>>>>>> [167126.796870] GRE over IPv4 demultiplexor driver
>>>>>>>>> [167126.797172] openvswitch: GRE over IPv4 tunneling
>>>>>>>>> driver
>>>>>>>>> [167126.797406] openvswitch: GRE over IPv6 tunneling
>>>>>>>>> driver
>>>>>>>>> [167126.797526] openvswitch: Geneve tunneling driver
>>>>>>>>> [167126.797528] openvswitch: VxLAN tunneling driver
>>>>>>>>> [167126.797529] openvswitch: STT tunneling driver
>>>>>>>>>
>>>>>>>>> - Greg
>>>>>>>>>> Cheers
>>>>>>>>>>
>>>>>>>>>> On Thu, 2019-01-03 at 08:47 -0800, Gregory Rose
>>>>>>>>>> wrote:
>>>>>>>>>>> Run 'cat /sys/module/openvswitch/version' and that
>>>>>>>>>>> should
>>>>>>>>>>> give
>>>>>>>>>>> you
>>>>>>>>>>> output like this:
>>>>>>>>>>>
>>>>>>>>>>> # cat /sys/module/openvswitch/version
>>>>>>>>>>> 2.10.1
>>>>>> _______________________________________________
>>>>>> discuss mailing list
>>>>>> discuss at openvswitch.org
>>>>>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>>>>> _______________________________________________
>>>>> discuss mailing list
>>>>> discuss at openvswitch.org
>>>>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

More information about the discuss mailing list