[ovs-dev] [PATCH] datapath: compat: fix kernel module reference count.

[William Tu]

Before we introduce dpif-netlink-rtnl, creating a tunnel device
depends on openvswitch/vport kernel module calling kernel's tunnel API.
Example call sequence:
 1) ovs_vport_add (openvswitch.ko)
 2) geneve_create (vport_geneve.ko)
 3) geneve_tnl_create (vport_geneve.ko)
 4) geneve_dev_create_fb (geneve.ko)
As a result, module dependency/refcnt is created: module openvswitch.ko
references vport_geneve.ko and vport_geneve.ko depends on geneve.ko.

After the dpif-netlink-rtnl, a tunnel device can be created by using
rtnetlink, so the creation of fb device comes from the ovs-vswitchd
instead of going through OVS kernel modules.  This breaks the module
dependency between 1) and 2).  As a result, when ovs-vswitchd is running,
the 'lsmod' shows:
  geneve                 27326  1 vport_geneve
  vport_geneve           12560  0
  openvswitch           172469  1 vport_geneve
Problems can be cerated by doing  `rmmod vport_geneve` then
`rmmod geneve`, when ovs-vswitchd expects geneve module should already
be loaded.

The patch introduces 1) load the upstream geneve.ko at init time
from openvswitch.ko and 2) grap the reference count of geneve.
This adds the dependency: openvswitch.ko needs geneve.ko, so that
geneve won't be unloaded accidentally.  As a result, `lsmod` shows
  geneve                 20254  2 vport_geneve
  vport_geneve            1744  0
  openvswitch           159846  1 vport_geneve

Note that when choosing to use upstream tunnel, we don't actually need
vport_geneve, so the patch increment refcnt of geneve instead of
vport_geneve.

Fixes: b6d6830d29e4 ("dpif-netlink-rtnl: Support GENEVE creation")
Signed-off-by: William Tu <u9012063 at gmail.com>
Cc: Eric Garver <e at erig.me>
---
Adding more testing details:
On OVS2.7 without dpif-netlink-rtnl, kernel 4.8
lsmod
  vport_geneve            1744  1 		(this is ref by openvswitch.ko)
  geneve                 20254  1 vport_geneve
  openvswitch           128504  3 vport_geneve

On OVS2.8 with dpif-netlink-rtnl, kernel 4.8, before the patch
lsmod
  geneve                 27326  1 vport_geneve
  vport_geneve           12560  0		(the ref is gone)
  openvswitch           172469  1 vport_geneve

On OVS2.8 with dpif-netlink-rtnl, kernel 4.8, after the patch
lsmod
  geneve                 27326  2 vport_geneve
  vport_geneve           12560  0
  openvswitch           172469  1 vport_geneve

---
 datapath/linux/compat/include/net/geneve.h | 32 ++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/datapath/linux/compat/include/net/geneve.h b/datapath/linux/compat/include/net/geneve.h
index d9c9f0bf7b38..02aaf4f83688 100644
--- a/datapath/linux/compat/include/net/geneve.h
+++ b/datapath/linux/compat/include/net/geneve.h
@@ -8,9 +8,41 @@
 
 #ifdef USE_UPSTREAM_TUNNEL
 #include_next <net/geneve.h>
+static struct module *geneve_mod;
 
 static inline int rpl_geneve_init_module(void)
 {
+	int ret;
+
+	/* When USE_UPSTREAM_TUNNEL, try loading and adding
+	 * module dependency on the upstream geneve module.
+	 */
+	geneve_mod = find_module("geneve");
+
+	if (!geneve_mod) {
+		ret = request_module("geneve");
+		if (ret) {
+			pr_warn("Failed loading geneve module\n");
+			return 0;
+		}
+	}
+
+	/* Geneve module already loaded, find
+	 * it and grap its reference count.
+	 */
+	mutex_lock(&module_mutex);
+
+	geneve_mod = find_module("geneve");
+	if (unlikely(!geneve_mod)) {
+		pr_warn("Failed loading geneve module\n");
+		return 0;
+	}
+
+	ret = ref_module(THIS_MODULE, geneve_mod);
+	if (ret)
+		pr_warn("Failed referencing geneve module\n");
+	mutex_unlock(&module_mutex);
+
 	return 0;
 }
 static inline void rpl_geneve_cleanup_module(void)
-- 
2.7.4