[ovs-discuss] rmmod: ERROR: Module openvswitch is in use -- when ovn config is added to conf.db

[Jesse Gross]

I think this isn't actually related to tunnels - ovn-controller
deletes the tunnel ports that it creates when it exits gracefully.
Plus, it seems like vport-geneve was unloaded successfully.

When an OVS vport is created, a reference is taken on the
corresponding vport-*.ko module. However, in the case of internal
ports created by the datapath, the internal device vport is actually
part of the main OVS module and so we are taking a reference on
ourselves. As a result, the only way to unload the module is to delete
all of the datapaths. It seems like this is what is happening here and
is what force-reload-kmod is working around.

Obviously, there's no benefit in taking a reference on ourselves, so
we could just check for this and not take a reference. Combined with
the fact that lightweight tunnels eliminates the need for vport
modules that are not part of OVS (on new enough kernels), then
unloading the module would be enough to gracefully cleanup everything.

On Thu, Sep 22, 2016 at 7:59 AM, Guru Shetty <guru at ovn.org> wrote:
> I think it usually happens because the geneve tunnel exists in the kernel
> datapath. The right way to do this is via '/etc/init.d/openvswitch-switch
> force-reload-kmod'
>
> On 22 September 2016 at 01:46, Flavio Fernandes <flavio at flaviof.com> wrote:
>>
>> [cc: Jesse]
>>
>> Hi folks,
>>
>> While playing with branch-2.6, I've noticed that something may not be
>> cleaned properly
>> in the kernel with openvswitch module; [only] when ovn was configured.
>>
>> What I observe [below] is that as soon as I configure info used by
>> ovn-controller to
>> connect to sb db, I am unable to unload the ovs kernel module.
>>
>> Any ideas on what commands I could use to further determine what could be
>> causing
>> this? I did not try it on master, but suspect the issue exists there as
>> well.
>>
>> Thanks,
>>
>> -- flaviof
>>
>> ==
>>
>> $ lsb_release -sc
>> trusty
>>
>> cd ~/ovs.git
>>
>> ./boot.sh && \
>> ./configure --prefix=/usr --localstatedir=/var  --sysconfdir=/etc \
>>                   --enable-ssl --with-linux=/lib/modules/$(uname -r)/build
>> && \
>> make -j$(nproc) && \
>> sudo make install ; echo $?
>>
>> for x in libcrc32c nf_conntrack nf_nat nf_nat_ipv6 nf_nat_ipv4
>> nf_defrag_ipv4 gre ; do \
>>    sudo modprobe $x ; \
>> done
>>
>> sudo insmod ./datapath/linux/openvswitch.ko && \
>> sudo insmod ./datapath/linux/vport-geneve.ko && \
>> echo ok
>>
>> sudo cp ./debian/openvswitch-switch.init /etc/init.d/openvswitch-switch
>>
>> sudo /etc/init.d/openvswitch-switch start
>> sudo /usr/share/openvswitch/scripts/ovn-ctl restart_northd
>> sudo /usr/share/openvswitch/scripts/ovn-ctl restart_controller
>>
>> #### insert commands that configure conf.db with OVN related commands
>> #### to make issue happen :/
>>
>> sudo /usr/share/openvswitch/scripts/ovn-ctl stop_controller
>> sudo /usr/share/openvswitch/scripts/ovn-ctl stop_northd
>> sudo /etc/init.d/openvswitch-switch stop
>>
>> # as you can see there is nothing related to ovs running...
>> $ ps aux | grep "ov[sn]"
>> $
>>
>> $ sudo rmmod vport_geneve
>> $
>>
>> # Happy case...
>> $ # this works fine if commands that configure conf.db with ovn are not
>> used...
>> $ sudo rmmod openvswitch
>> $ sudo lsmod | grep openv
>> $
>>
>> ==
>>
>> # However, not as nice if conf.db is configured with encaps info:
>> $ sudo rmmod openvswitch
>> rmmod: ERROR: Module openvswitch is in use
>>
>> $ sudo lsmod | grep openv
>> openvswitch           253098  2
>> gre                    13796  1 openvswitch
>> nf_defrag_ipv4         12758  2 openvswitch,nf_conntrack_ipv4
>> nf_nat_ipv4            13263  1 openvswitch
>> nf_defrag_ipv6         34768  2 openvswitch,nf_conntrack_ipv6
>> nf_nat_ipv6            13279  1 openvswitch
>> nf_nat                 21841  3 openvswitch,nf_nat_ipv4,nf_nat_ipv6
>> nf_conntrack           97201  6
>> openvswitch,nf_nat,nf_nat_ipv4,nf_nat_ipv6,nf_conntrack_ipv4,nf_conntrack_ipv6
>> libcrc32c              12644  1 openvswitch
>>
>> $ sudo modprobe --show-depends openvswitch
>> insmod /lib/modules/3.13.0-95-generic/kernel/lib/libcrc32c.ko
>> insmod /lib/modules/3.13.0-95-generic/kernel/net/ipv4/gre.ko
>> insmod /lib/modules/3.13.0-95-generic/kernel/net/ipv4/ip_tunnel.ko
>> insmod /lib/modules/3.13.0-95-generic/kernel/drivers/net/vxlan.ko
>> insmod
>> /lib/modules/3.13.0-95-generic/kernel/net/openvswitch/openvswitch.ko
>>
>>
>> #### commands that configure conf.db with OVN related commands to make
>> rmmod fail
>>
>> HOST_IP=$(ip -4 addr show eth0 | grep -oP "(?<=inet ).*(?=/)")
>> OVN_DB_IP=$HOST_IP
>> OVN_SB_DB_PORT=${2-6642}
>> OVN_SB_REMOTE="tcp:${OVN_DB_IP}:${OVN_SB_DB_PORT}"
>> sudo ovs-vsctl --no-wait set open_vswitch .
>> external-ids:ovn-bridge="br-int"
>> sudo ovs-vsctl --no-wait set open_vswitch .
>> external-ids:ovn-encap-ip="$HOST_IP"
>> sudo ovs-vsctl --no-wait set open_vswitch .
>> external-ids:ovn-encap-type="geneve"
>> sudo ovs-vsctl --no-wait set open_vswitch .
>> external-ids:ovn-remote="$OVN_SB_REMOTE"
>>
>>
>>
>> _______________________________________________
>> discuss mailing list
>> discuss at openvswitch.org
>> http://openvswitch.org/mailman/listinfo/discuss
>>
>
>
> _______________________________________________
> discuss mailing list
> discuss at openvswitch.org
> http://openvswitch.org/mailman/listinfo/discuss
>