[ovs-dev] Questions about flow put failed(File exist)
Yutao (Simon, Cloud Infrastructure Service Product Dept.)
yu.yutao at huawei.com
Mon Jul 20 06:49:27 UTC 2020
Hi all,
The ovs ran into a problem in our environment: Flow continuous to upcall , because of flow_put_on_pmd failed.
Steps to trigger this problem:
Step1
Add two openflow rules:
#1. ovs-ofctl add-flow br-forward "table=0,in_port=1,dl_dst:3E:5B:51:81:5A:2F,ip,nw_dst=123.1.1.1/32,priority=10 actions=output:2 "
#2. ovs-ofctl add-flow br-forward "table=0,in_port=1,dl_dst:3E:5B:51:81:5A:2F,ip,nw_src=200.175.100.200,priority=20 actions=output:2"
Step2
Two traffic flow from #1 port:
#1. SIP:8.93.223.217 DIP:114.166.27.239
#2. SIP:128.93.223.217 DIP:114.166.27.239
It will generate two megaflow in datapath:
#1. recirc_id(0),in_port(2),eth(dst=3e:5b:51:81:5a:2f),eth_type(0x8100),encap(eth_type(0x0800),ipv4(src=8.93.223.217/128.0.0.0,dst=114.166.27.239/255.0.0.0,frag=no)),
packets:12825613, bytes:3232054476, used:0.000s, actions:ext_action(action=route,args(table_index=0;policy=ip_src))
#2. recirc_id(0),in_port(2),eth(dst=3e:5b:51:81:5a:2f),eth_type(0x8100),encap(eth_type(0x0800),ipv4(src=128.93.223.217/192.0.0.0,dst=114.166.27.239/255.0.0.0,frag=no)),
packets:12825613, bytes:3232054476, used:0.000s, actions:ext_action(action=route,args(table_index=0;policy=ip_src))
Step3
Stop #2 traffic flow in Step 2(SIP:128.93.223.217), wait the #2 megaflow aged;
Step4
Delete the #2 openflow rule in Step1, then start the #2 traffic(SIP:128.93.223.217) again。
Problem occurs---the #2 traffic flow continuous to upcall , because of flow_put_on_pmd failed, error is “File exist”.
Root cause analysis:
1. In Step3, we delete the #2 openflow rule then wait #2 megaflow aged. Now, there is only one megaflow in datapath:
recirc_id(0),in_port(2),eth(dst=3e:5b:51:81:5a:2f),eth_type(0x8100),encap(eth_type(0x0800),ipv4(src=8.93.223.217/128.0.0.0,dst=114.166.27.239/255.0.0.0,frag=no)),
packets:12825613, bytes:3232054476, used:0.000s, actions:ext_action(action=route,args(table_index=0;policy=ip_src))
2. In Step 4 , start the #2 traffic flow again. The flow lookup cls in pmd thread, but it will not match, then packet will upcall to handler.
3. In handler thread, the flow will match "table=0,in_port=1,dl_dst:3E:5B:51:81:5A:2F,ip,nw_dst=123.1.1.1/32,priority=10 actions=output:2 ". So the src_ip in mask will be 0.0.0.0(No openflow rule to match src_ip).
4. Before put megaflow to pmd, handler will check if the megaflow is exist in cls. The key to lookup in cls will be masked in dpif_netdev_flow_put (now,src_ip in key is 0.0.0.0). Unfortunately, It will match the #2 megaflow, File exist, Put faild. Flow continus to upcall
The key to lookup cls in fast_path_processing(-->dp_netdev_pmd_lookup_flow) is different from the one in flow_put_on_pmd(-->dp_netdev_pmd_lookup_flow).
Is there any useful patch to fix this bug? How about the following two modifications:
1. Use the same key to lookup cls in handler thread
---
lib/dpif-netdev.c | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
index e456cc9..facb9f6 100644
--- a/lib/dpif-netdev.c
+++ b/lib/dpif-netdev.c
@@ -3524,10 +3524,12 @@ dpif_netdev_flow_put(struct dpif *dpif, const struct dpif_flow_put *put)
}
/* Must produce a netdev_flow_key for lookup.
- * Use the same method as employed to create the key when adding
- * the flow to the dplcs to make sure they match. */
- netdev_flow_mask_init(&mask, &match);
- netdev_flow_key_init_masked(&key, &match.flow, &mask);
+ * Must generate key by flow to guarantee that key is the same as the key in
+ * fast_path_processing().
+ * ps: key.hash will not be used in lookup. */
+ miniflow_map_init(&key.mf, &match->flow);
+ miniflow_init(&key.mf, &match->flow);
+ key.len = netdev_flow_key_size(miniflow_n_values(&key.mf));
if (put->pmd_id == PMD_ID_NULL) {
if (cmap_count(&dp->poll_threads) == 0) {
--
2. Modify the revalidate_ukey__ to delete the megaflow when mask changes
From c7bda1680832a18946b8459f6b937d62b02d0280 Mon Sep 17 00:00:00 2001
From: f00448292 <fuzhantao at huawei.com<mailto:fuzhantao at huawei.com>>
Date: Mon, 20 Jul 2020 14:44:34 +0800
Subject: [PATCH] NEW
Signed-off-by: f00448292 <fuzhantao at huawei.com<mailto:fuzhantao at huawei.com>>
---
ofproto/ofproto-dpif-upcall.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/ofproto/ofproto-dpif-upcall.c b/ofproto/ofproto-dpif-upcall.c
index 8dfa05b..8a8c233 100644
--- a/ofproto/ofproto-dpif-upcall.c
+++ b/ofproto/ofproto-dpif-upcall.c
@@ -2209,7 +2209,7 @@ revalidate_ukey__(struct udpif *udpif, const struct udpif_key *ukey,
* tells that the datapath flow is now too generic and must be narrowed
* down. Note that we do not know if the datapath has ignored any of the
* wildcarded bits, so we may be overly conservative here. */
- if (flow_wildcards_has_extra(&dp_mask, ctx.wc)) {
+ if (flow_wildcards_equal(&dp_mask, ctx.wc)) {
goto exit;
}
--
Looking forward to your reply.
Thanks.
More information about the dev
mailing list