[ovs-discuss] Issue with modify rule

[Justin Pettit]

> On Dec 12, 2017, at 8:35 AM, Amedeo Sapio <amedeo.sapio at gmail.com> wrote:
> 
> Hi everyone, 
> I am observing an unexpected behavior when I modify an existing flow-rule in OvS. 
> 
> Let me first describe the experiment. 
> I have a simple topology in mininet: H1--S1--S2--S3--H2. 
> 
> I want to change, periodically, the value of the IP TOS bit in packets H1->S1 so that for a period T all the packets have TOS=0, then the next period T all the packets should have TOS=4, and then the next period is back to 0, and so on.
> 
> At the same time, I want to count packets with TOS 0 and 4 respectively in all the switches, both in input and output. 
> 
> I have one rule in table 0 of S1 to write the TOS, that I periodically swap:
> 
> from:
> ovs-ofctl mod-flows s1 "table=0,dl_type=0x0800,nw_proto=17,in_port=1, priority=10,actions=mod_nw_tos:0,resubmit(,1)"
> 
> to:
> ovs-ofctl mod-flows s1 "table=0,dl_type=0x0800,nw_proto=17, in_port=1,priority=10,actions=mod_nw_tos:4,resubmit(,1)"
> 
> and viceversa. 
> 
> Then in table 1 I have permanent rules to count the marked packets:
> 
> ovs-ofctl add-flow s1 "table=1,dl_type=0x0800,nw_proto=17,nw_tos=4,actions=resubmit(,2)"
> ovs-ofctl add-flow s1 "table=1,dl_type=0x0800,nw_proto=17,nw_tos=0,actions=resubmit(,2)"
> 
> and then I have other rules that are not relevant for this problem. 
> 
> In S2 I have permanent rules to count the marked packets received in input:
> 
> ovs-ofctl add-flow s2 "table=0,in_port=2, priority=10,dl_type=0x0800,nw_proto=17,nw_tos=4,actions=resubmit(,2)"
> 
> ovs-ofctl add-flow s2 "table=0,in_port=2, priority=10,dl_type=0x0800,nw_proto=17,nw_tos=0,actions=resubmit(,2)"
> 
> 
> For the majority of the packets everything works as expected. However, few packets (in the order of 4 every 1000) are miscounted. Specifically, some packets that are counted as tos=4 in S1, are counted as tos=0 in S2. The same happen in the opposite case.
> 
> So, after analyzing also the tcpdump traces, it looks like that, when I change the rule, few packets match the old rule and are counted internally (in S1:table1) as having a certain TOS, but when they go out, they actually have a different TOS. 

I'm not an expert on the revalidation code, but I suspect the issue has to do with OVS's flow caching.  OVS will take those OpenFlow flows and generate datapath flows in the fastpath.  Occasionally, OVS will pull the stats from the datapath flows and update the appropriate OpenFlow rule stats.

In your use-case, at any one time, two of those four flows are active and likely generating a single datapath flow that matches UDP traffic and setting the TOS to either 0 or 4.  When you change the active flows, I'm wondering if the datapath flow is getting modified (as opposed to deleted and recreated), and then some of traffic that had the old TOS value is then being associated with the currently active OpenFlow rules.

If you wanted to debug this a bit more, you could try digging into the OVS logs.  Setting some of the "dpif" modules to debug should show you when the stats are pulled and datapath flows are modified.

Good luck!

--Justin