[ovs-dev] assertion failing in commit_set_ipv4_action(), flow->nw_proto != base_flow->nw_proto

[Thomas Morin]

Hi Jarno,

2016-12-02, Jarno Rajahalme:
>
>>>>> 2016-11-29T14:44:40.126Z|00001|odp_util(revalidator45)|WARN|commit_set_ipv4_action
>>>>> assert would fail....
>>>>> 2016-11-29T14:44:40.126Z|00002|odp_util(revalidator45)|WARN| 
>>>>>  base_flow: 
>>>>> ip,in_port=5,dl_vlan=3,dl_vlan_pcp=0,dl_src=fa:16:3e:33:f7:fe,dl_dst=00:00:5e:00:43:64,nw_src=0.0.0.0,nw_dst=0.0.0.0,nw_proto=0,nw_tos=0,nw_ecn=0,nw_ttl=0
>>>>> 2016-11-29T14:44:40.126Z|00003|odp_util(revalidator45)|WARN| 
>>>>>  flow: 
>>>>> tcp,in_port=5,dl_vlan=3,dl_vlan_pcp=0,dl_src=fa:16:3e:33:f7:fe,dl_dst=00:00:5e:00:43:64,nw_src=10.0.1.22,nw_dst=10.0.0.3,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=53295,tp_dst=8080,tcp_flags=psh|ack
>>>>> 2016-11-29T14:44:40.126Z|00004|odp_util(revalidator45)|WARN| 
>>>>>  masks: 
>>>>> recirc_id=0xffffffff,reg0=0xffffffff,in_port=4294967295,dl_vlan=4095,dl_vlan_pcp=7,dl_src=ff:ff:ff:ff:ff:ff,dl_dst=ff:ff:ff:ff:ff:ff,dl_type=0xffff
>>>>> 2016-11-29T14:44:40.126Z|00005|odp_util(revalidator45)|WARN| 
>>>>>  actions: 
>>>>> set(ipv4(src=10.0.1.22,dst=10.0.0.3,ttl=63)),set(eth(src=b8:2a:72:de:1b:e3,dst=00:17:cb:79:2c:01)),push_mpls(label=410384,tc=0,ttl=63,bos=1,eth_type=0x8847),9,set(eth(src=fa:16:3e:33:f7:fe,dst=00:00:5e:00:43:64)),pop_mpls(eth_type=0x800),push_vlan(vid=3,pcp=0),1
>>>
>>> push_mpls clears L3/L4, while pop_mpls re-populates them, and then 
>>> processing the output to port 1 hits the assert?
>>
>> That's what I'm thinking too.
>>
>> Jarno, is this something you have time to look into?  It'd be great, if
>> you do.  I'm way behind.
>
> I’m looking at this.
>
> Based on the trace given it seems that:
> 1. Packet is received on br-int port 32, which outputs it via NORMAL 
> action over a patch port to another bridge. The only patch-port on 
> br-int is 2 (patch-tun). The NORMAL action adds dl_vlan=1.
> 2. br-tun receives the packet on in_port 1 (patch-int), and outputs it 
> on it’s port 2 (patch-to-mpls)
> 3. br-mpls receives the packet on it’s in_port 2 (patch-to-tun), does 
> pop_vlan, and outputs to it’s port 21 (ipvpn-pp-out), which is also an 
> patch port.
> 4. br-mpls (?) receives the packet on it’s in_port 20 (ipvpn-pp-in), 
> does 
> dec_ttl,push_mpls:0x8847,load:0x644c0->OXM_OF_MPLS_LABEL[],set_field:b8:2a:72:de:1b:e3->eth_src,set_field:00:17:cb:79:2c:01->eth_dst,output:1

Yes, step 4 is br-mpls receiving from a patch-port that has both of its 
ends in br-mpls.


>
> All this generates a megaflow: 
> set(ipv4(src=10.0.1.23,dst=10.0.0.3,ttl=63)),set(eth(src=b8:2a:72:de:1b:e3,dst=00:17:cb:79:2c:01)),push_mpls(label=410816,tc=0,ttl=63,bos=1,eth_type=0x8847),9
>
> This is only the beginning part of the trouble-some megaflow, in which 
> br-int sends the packet also to another port (vlan 3), and as part of 
> that pops the MPLS and restores the original ethernet addresses. Maybe 
> this would happen with the trace too, if you flushed MACs before the 
> trace?
>

Yes, I'd think so.
I'll try that later today.


> The patch ports 21 and 20 appear to be in the same bridge and patched 
> to each other. Is this the case?

Yes.
You'll probably tell me it that it's surprising; it's a solution we 
found quite a while ago to keep track of the VPN instance in which the 
packet is to be processed, but we're now implementing a solution using 
registers instead.

>
> The crashing megaflow has in_port=5,dl_vlan=3. Is this also on br-int?


Sorry, I should have mentioned that the crash dump comes from a previous 
run different from the one for which I provided dump-flows/show commands 
output.

The in_port on  the crashing flow is a qvoXXXX port plugged on br-int.


>
> Also, OVS 2.6 is a little bit less aggressive about avoiding 
> recirculation after mpls operations, and I’d be interested to know if 
> your case fails the same way with OVS 2.6?

I didn't succeed in compiling OVS 2.6 on this lab (Trusty).
Is recirculation involved in the crashing flow ?

-Thomas