[ovs-dev] [PATCH v4 1/2] Encap & Decap actions for MPLS packet type.

Eelco Chaudron echaudro at redhat.com
Tue Apr 6 08:54:32 UTC 2021



On 6 Apr 2021, at 10:27, Martin Varghese wrote:

> On Thu, Apr 01, 2021 at 11:32:06AM +0200, Eelco Chaudron wrote:
>>
>>
>> On 1 Apr 2021, at 11:28, Martin Varghese wrote:
>>
>>> On Thu, Apr 01, 2021 at 11:17:14AM +0200, Eelco Chaudron wrote:
>>>>
>>>>
>>>> On 1 Apr 2021, at 11:09, Martin Varghese wrote:
>>>>
>>>>> On Thu, Apr 01, 2021 at 10:54:42AM +0200, Eelco Chaudron wrote:
>>>>>>
>>>>>>
>>>>>> On 1 Apr 2021, at 10:35, Martin Varghese wrote:
>>>>>>
>>>>>>> On Thu, Apr 01, 2021 at 08:59:27AM +0200, Eelco Chaudron wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> On 1 Apr 2021, at 6:10, Martin Varghese wrote:
>>>>>>>>
>>>>>>>>> On Wed, Mar 31, 2021 at 03:59:40PM +0200, Eelco Chaudron 
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 26 Mar 2021, at 7:21, Martin Varghese wrote:
>>>>>>>>>>
>>>>>>>>>>> From: Martin Varghese <martin.varghese at nokia.com>
>>>>>>>>>>>
>>>>>>>>>>> The encap & decap actions are extended to support MPLS
>>>>>>>>>>> packet type.
>>>>>>>>>>> Encap & decap actions adds and removes MPLS
>>>>>>>>>>> header at start of the
>>>>>>>>>>> packet.
>>>>>>>>>>
>>>>>>>>>> Hi Martin,
>>>>>>>>>>
>>>>>>>>>> I’m trying to do some real-life testing, and
>>>>>>>>>> I’m running into
>>>>>>>>>> issues. This
>>>>>>>>>> might be me setting it up wrongly but just
>>>>>>>>>> wanting to confirm…
>>>>>>>>>>
>>>>>>>>>> I’m sending an MPLS packet that contains an ARP packet into 
>>>>>>>>>> a
>>>>>>>>>> physical port.
>>>>>>>>>> This is the packet:
>>>>>>>>>>
>>>>>>>>>> Frame 4: 64 bytes on wire (512 bits), 64 bytes
>>>>>>>>>> captured (512 bits)
>>>>>>>>>>     Encapsulation type: Ethernet (1)
>>>>>>>>>>     [Protocols in frame: eth:ethertype:mpls:data]
>>>>>>>>>> Ethernet II, Src: 00:00:00_00:00:01 (00:00:00:00:00:01), Dst:
>>>>>>>>>> 00:00:00_00:00:02 (00:00:00:00:00:02)
>>>>>>>>>>     Destination: 00:00:00_00:00:02 (00:00:00:00:00:02)
>>>>>>>>>>         Address: 00:00:00_00:00:02 (00:00:00:00:00:02)
>>>>>>>>>>         .... ..0. .... .... .... .... = LG bit: Globally 
>>>>>>>>>> unique
>>>>>>>>>> address
>>>>>>>>>> (factory default)
>>>>>>>>>>         .... ...0 .... .... .... .... = IG bit:
>>>>>>>>>> Individual address
>>>>>>>>>> (unicast)
>>>>>>>>>>     Source: 00:00:00_00:00:01 (00:00:00:00:00:01)
>>>>>>>>>>         Address: 00:00:00_00:00:01 (00:00:00:00:00:01)
>>>>>>>>>>         .... ..0. .... .... .... .... = LG bit: Globally 
>>>>>>>>>> unique
>>>>>>>>>> address
>>>>>>>>>> (factory default)
>>>>>>>>>>         .... ...0 .... .... .... .... = IG bit:
>>>>>>>>>> Individual address
>>>>>>>>>> (unicast)
>>>>>>>>>>     Type: MPLS label switched packet (0x8847)
>>>>>>>>>> MultiProtocol Label Switching Header, Label: 100, Exp: 0, S:
>>>>>>>>>> 1, TTL:
>>>>>>>>>> 64
>>>>>>>>>>     0000 0000 0000 0110 0100 .... .... .... = MPLS Label: 100
>>>>>>>>>>     .... .... .... .... .... 000. .... .... = MPLS 
>>>>>>>>>> Experimental
>>>>>>>>>> Bits: 0
>>>>>>>>>>     .... .... .... .... .... ...1 .... .... = MPLS
>>>>>>>>>> Bottom Of Label
>>>>>>>>>> Stack: 1
>>>>>>>>>>     .... .... .... .... .... .... 0100 0000 = MPLS TTL: 64
>>>>>>>>>> Data (46 bytes)
>>>>>>>>>>
>>>>>>>>>> 0000  ff ff ff ff ff ff 52 54 00 88 51 38 08 06 00 01
>>>>>>>>>> ......RT..Q8....
>>>>>>>>>> 0010  08 00 06 04 00 01 52 54 00 88 51 38 01 01 01 65
>>>>>>>>>> ......RT..Q8...e
>>>>>>>>>> 0020  00 00 00 00 00 00 01 01 01 64 27 98 a0 47
>>>>>>>>>> .........d'..G
>>>>>>>>>>     Data:
>>>>>>>>>> ffffffffffff525400885138080600010800060400015254008851380101016500000000?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I’m trying to use the following rules:
>>>>>>>>>>
>>>>>>>>>>   ovs-ofctl del-flows ovs_pvp_br0
>>>>>>>>>>   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
>>>>>>>>>> "priority=100,dl_type=0x8847,mpls_label=100
>>>>>>>>>> actions=decap(),decap(packet_type(ns=0,type=0x806)),resubmit(,3)"
>>>>>>>>>>   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
>>>>>>>>>> "table=3,priority=10
>>>>>>>>>> actions=normal"
>>>>>>>>>>
>>>>>>>>>> With these, I expect the packet to be sent to vnet0, but
>>>>>>>>>> it’s not.
>>>>>>>>>> Actually,
>>>>>>>>>> the datapath rule looks odd, while the userspace rules seem
>>>>>>>>>> to match:
>>>>>>>>>>
>>>>>>>>>>   $ ovs-dpctl dump-flows
>>>>>>>>>>   recirc_id(0),in_port(1),eth(),eth_type(0x8847),mpls(label=100/0xfffff,tc=0/0,ttl=0/0x0,bos=1/1),
>>>>>>>>>> packets:13, bytes:1118, used:0.322s,
>>>>>>>>>> actions:pop_eth,pop_mpls(eth_type=0x806),recirc(0x19a)
>>>>>>>>>>   recirc_id(0x19a),in_port(1),eth_type(0x0806), packets:13,
>>>>>>>>>> bytes:884,
>>>>>>>>>> used:0.322s, actions:drop
>>>>>>>>>>
>>>>>>>>>>   $ ovs-ofctl dump-flows ovs_pvp_br0 -O OpenFlow13
>>>>>>>>>>   cookie=0x0, duration=85.007s, table=0, n_packets=51,
>>>>>>>>>> n_bytes=4386,
>>>>>>>>>> priority=100,mpls,mpls_label=100
>>>>>>>>>> actions=decap(),decap(packet_type(ns=0,type=0x806)),resubmit(,3)
>>>>>>>>>>   cookie=0x0, duration=84.990s, table=3, n_packets=51,
>>>>>>>>>> n_bytes=3468,
>>>>>>>>>> priority=10 actions=NORMAL
>>>>>>>>>>
>>>>>>>>> The inner packet is ethernet. So the packet type should be
>>>>>>>>> (ns=0,type=0)
>>>>>>>>> ?
>>>>>>>>
>>>>>>>> Forgot to add that I already tried that to start
>>>>>>>> with, based on the
>>>>>>>> example,
>>>>>>>> but as that did not work I tried 0x806.
>>>>>>>>
>>>>>>>> PS: I have this as a remark in my review notes, i.e., to
>>>>>>>> explain the
>>>>>>>> ns and
>>>>>>>> type usage here.
>>>>>>>>
>>>>>>>>
>>>>>>>> This resulted in packets being counted at the open flow
>>>>>>>> level, but it
>>>>>>>> results in NO data path rules. Do get an error though:
>>>>>>>>
>>>>>>>> 2021-04-01T06:53:36.056Z|00141|dpif(handler37)|WARN|system at ovs-system:
>>>>>>>> failed to put[create] (Invalid argument)
>>>>>>>> ufid:3d2d6f6d-5a66-4ace-8b09-7cdcfa5efc8e 
>>>>>>>> recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(1),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=00:00:00:00:00:01/00:00:00:00:00:00,dst=00:00:00:00:00:02/00:00:00:00:00:00),eth_type(0x8847),mpls(label=100/0xfffff,tc=0/0,ttl=64/0x0,bos=1/1),
>>>>>>>> actions:pop_eth,pop_mpls(eth_type=0x6558),set(eth()),recirc(0x4c)
>>>>>>>
>>>>>>> This set(eth) before the recirc is the problem i guesss. I need
>>>>>>> to check
>
> I could reproduce the problem. It has nothing to do with ARP or IP. 
> Unlike my test scripts, in your test you are setting the mac address 
> after the encap action
>
> Ovs-vswitchd is programming a set(eth(dst) action between the  
> pop_mpls and  recirc as it sees a difference in mac address in flow 
> structure and  base_flow structure.
>
> The mac address in flow structure is not cleared in PT_ETH handling of 
> xlate_generic_decap_action but  it is cleared in base_flow in the   
> decap handling of PT_ETH in commit_encap_decap_action function
>
> Due to this difference the set(eth(dst) action will be programmed to 
> the datapath.
>
> Also, I see that in  commit_set_ether_action Function  
> “flow->packet_type != htonl(PT_ETH)” is used to check if the 
> packet is ethernet instead of base_flow->packet_type.
>
> I assume check on base_flow->packet_type make more sense here ?
>
> I tried to fix this issue in 2 different ways.
>
> 1   I have cleared the mac address in flow structure  in PT_ETH 
> handling of xlate_generic_decap action.
>
> 2  In the  commit_set_ether action I changed the check from  
> “flow->packet_type != htonl(PT_ETH)” to  “base_flow->packet_type 
> != htonl(PT_ETH))”.
>
> Though both of them solves this problem, couple of NSH regression 
> tests are failing
>
> 2291: nsh - md1 encap over a veth link                FAILED 
> (nsh.at:85)
>
> 58022292: nsh - md2 encap over a veth link                FAILED 
> (nsh.at:213)
>
> I see that they are failing as they are expecting a set(eth(dst)  
> between the the pop_nsh and the recirc.
>
> Set(eth) action is because of the reasons explained above –
>
> Datapath actions: 
> push_nsh(flags=0,ttl=63,mdtype=1,np=3,spi=0x1234,si=255,c1=0x11223344,c2=0x0,c3=0x0,c4=0x0),push_eth(src=00:00:00:00:00:00,dst=11:22:33:44:55:66),pop_eth,pop_nsh(),set(eth(dst=11:22:33:44:55:66)),recirc(0x1)
>
> In my understanding set(eth) here  is wrong as there is no set 
> ethernet action in the userspace rule
> - Hide quoted text -
>
> table=0,in_port=4,dl_type=0x894f,nsh_mdtype=1,nsh_spi=0x1234,nsh_c1=0x11223344,actions=decap(),decap(),2
>
>
>
> Could someone comment ?

Maybe Jan can answer as he did the NSH implementation, however, what 
would be of interest if you can give me an example of how the encap() 
decap() for this would be used in real life so I’m sure I’m testing 
it correctly?

What I did so far was to encapsulate all traffic going from a VM to the 
physical port in MPLS using the flows like:

ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 
"priority=100,in_port=vnet0,actions=encap(mpls(ether_type=0x8847)),set_mpls_label:100,encap(ethernet),,set_field:00:00:00:00:00:02->dl_ds
t,set_field:00:00:00:00:00:01->dl_src,output:enp5s0f0"

Then I would capture this traffic and sent it back over the same port, 
hoping it would come out as plane traffic with the following rule:

ovs-ofctl add-flow -O OpenFlow15 ovs_pvp_br0 
"priority=100,dl_type=0x8847,mpls_label=100 
actions=decap(),decap(packet_type(ns=0,type=0)),resubmit(,3)"
ovs-ofctl add-flow -O OpenFlow15 ovs_pvp_br0 "table=3,priority=10 
actions=normal"

If this is correct, let me know, and if Jan does not reply, I’ll try 
to understand the code in this area and see if I can find out some 
details…

//Eelco

>>>>>>>> 2021-04-01T06:53:36.056Z|00142|dpif(handler37)|WARN|system at ovs-system:
>>>>>>>> execute 
>>>>>>>> pop_eth,pop_mpls(eth_type=0x6558),set(eth()),recirc(0x4c)
>>>>>>>> failed
>>>>>>>> (Invalid argument) on packet 
>>>>>>>> mpls,vlan_tci=0x0000,dl_src=00:00:00:00:00:01,dl_dst=00:00:00:00:00:02,mpls_label=100,mpls_tc=0,mpls_ttl=64,mpls_bos=1
>>>>>>>>  with metadata skb_priority(0),skb_mark(0),in_port(1) mtu 0
>>>>>>>>
>>>>>>>> Are there missing parts in my kernel that do not get properly
>>>>>>>> detected by
>>>>>>>> the feature detection?
>>>>>>>>
>>>>>>>> $ ovs-appctl dpif/show-dp-features ovs_pvp_br0
>>>>>>>> Masked set action: Yes
>>>>>>>> Tunnel push pop: No
>>>>>>>> Ufid: Yes
>>>>>>>> Truncate action: Yes
>>>>>>>> Clone action: Yes
>>>>>>>> Sample nesting: 10
>>>>>>>> Conntrack eventmask: Yes
>>>>>>>> Conntrack clear: Yes
>>>>>>>> Max dp_hash algorithm: 0
>>>>>>>> Check pkt length action: Yes
>>>>>>>> Conntrack timeout policy: Yes
>>>>>>>> Explicit Drop action: No
>>>>>>>> Optimized Balance TCP mode: No
>>>>>>>> l2 MPLS tunnelling: Yes
>>>>>>>> Max VLAN headers: 2
>>>>>>>> Max MPLS depth: 3
>>>>>>>> Recirc: Yes
>>>>>>>> CT state: Yes
>>>>>>>> CT zone: Yes
>>>>>>>> CT mark: Yes
>>>>>>>> CT label: Yes
>>>>>>>> CT state NAT: Yes
>>>>>>>> CT orig tuple: Yes
>>>>>>>> CT orig tuple for IPv6: Yes
>>>>>>>> IPv6 ND Extension: No
>>>>>>>>
>>>>>>> You are good
>>>>>>>
>>>>>>> I am not sure what is going wrong. Your test case looks same as
>>>>>>> the unit
>>>>>>> test i added.
>>>>>>>
>>>>>>> I tried myself again and this is i get
>>>>>>>
>>>>>>> ovs-ofctl -O OpenFlow13 add-flow br_mpls2
>>>>>>> "in_port=$egress_port,dl_type=0x8847
>>>>>>> +actions=decap(),decap(packet_type(ns=0,type=0),goto_table:1"
>>>>>>> ovs-ofctl -O OpenFlow13 add-flow br_mpls2
>>>>>>> +"table=1,in_port=$egress_port,dl_type=0x0800,nw_dst=1.1.1.2
>>>>>>> +actions=set_field:00:00:00:00:00:02->dl_dst,set_field:00:00:00:00:00:01->dl_sr
>>>>>>> +c output:$ingress_port"
>>>>>>>
>>>>>>> recirc_id(0x3),in_port(6),eth(src=36:b1:ee:7c:01:03,dst=36:b1:ee:7c:01:02),eth_
>>>>>>> +type(0x0800),ipv4(dst=1.1.1.2,frag=no), packets:3, bytes:294,
>>>>>>> used:0.837s,
>>>>>>> +actions:set(eth(src=00:00:00:00:00:01,dst=00:00:00:00:00:02)),4
>>>>>>> recirc_id(0),in_port(6),eth(),eth_type(0x8847),mpls(label=0/0x0,tc=0/0,ttl=0/0x
>>>>>>> +0,bos=1/1), packets:3, bytes:348, used:0.837s,
>>>>>>> +actions:pop_eth,pop_mpls(eth_type=0x6558),recirc(0x3)
>>>>>>>
>>>>>>> The packet to the ovs is
>>>>>>> ETH|MPLS|ETH|IP ?
>>>>>>> How it is differnt from you test case?
>>>>>>
>>>>>> Mine is ETH|MPLS|ETH|ARP, which works fine with pop_mpls
>>>>>>
>>>>> I am wondering how old mpls pop  action works
>>>>> could you please put down the userspace and datapath rules when
>>>>> you used
>>>>> pop_mpls.
>>>>>
>>>>> In my understanding it can never work as what you have after MPLS 
>>>>> is
>>>>> ethernet and not ARP.
>>>>
>>>> It’s ethernet + ARP, but here are my rules:
>>>
>>> To clarify
>>>
>>> The test vector for Decap Test case
>>> ETH|MPLS|ETH|ARP
>>> The test vector for pop mpls test case
>>> ETH|MPLS|ARP|
>>>
>>> The above understanding correct?
>>
>> Guess our emails crossed ;)  I was sending in the same packet for 
>> both the
>> test cases, so
>>
>> ETH|MPLS|ETH|ARP
>>
>> Which with decap() is resulting in the rules not being programmed
>>
>> With popmpls I saw the packets in being received, but did not notice 
>> the
>> incorrect use of popmpls so my packet after popmpls looks like 
>> ETH|ETH|ARP.
>>
>> So I guess all that remains is why the data path rules is not 
>> accepted for
>> ARP with the decap.
>>
>>>>
>>>> dpctl:
>>>>
>>>> recirc_id(0),in_port(2),eth(),eth_type(0x8847),mpls(label=100/0xfffff,tc=0/0,ttl=0/0x0,bos=1/1),
>>>> packets:64, bytes:5504, used:0.444s,
>>>> actions:pop_mpls(eth_type=0x806),recirc(0x80d)
>>>> recirc_id(0x80d),in_port(2),eth(src=00:00:00:00:00:01,dst=00:00:00:00:00:02),eth_type(0x0806),
>>>> packets:64, bytes:5248, used:0.444s, actions:3,1
>>>>
>>>> ofctl:
>>>>
>>>> OFPST_FLOW reply (OF1.5) (xid=0x2):
>>>>  cookie=0x0, duration=178.890s, table=0, n_packets=127,
>>>> n_bytes=10922,
>>>> idle_age=0, priority=100,mpls,mpls_label=100
>>>> actions=pop_mpls:0x0806,resubmit(,3)
>>>>  cookie=0x0, duration=178.873s, table=3, n_packets=127,
>>>> n_bytes=10414,
>>>> idle_age=0, priority=10 actions=NORMAL
>>>>
>>>>
>>>>>>> Thanks for your time.
>>>>>>
>>>>>> Your welcome
>>>>>>
>>>>>>>>>>
>>>>>>>>>> If I use the old way, doing pop_mpls, it works fine:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ovs-ofctl del-flows ovs_pvp_br0
>>>>>>>>>> ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
>>>>>>>>>> "priority=100,dl_type=0x8847,mpls_label=100
>>>>>>>>>> actions=pop_mpls:0x0806,resubmit(,3)"
>>>>>>>>>> ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
>>>>>>>>>> "table=3,priority=10
>>>>>>>>>> actions=normal"
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I also noticed (despite the test example) to make
>>>>>>>>>> encap work, I had
>>>>>>>>>> to set
>>>>>>>>>> the ethernet MAC addresses, or else the packets were not
>>>>>>>>>> getting out.
>>>>>>>>>> So something like:
>>>>>>>>>>
>>>>>>>>>>   ovs-ofctl add-flow -O OpenFlow13
>>>>>>>>>> ovs_pvp_br0 
>>>>>>>>>> "priority=100,in_port=vnet0,actions=encap(mpls(ether_type=0x8847)),set_mpls_label:100,encap(ethernet),,set_field:00:00:00:00:00:02->dl_dst,set_field:00:00:00:00:00:01->dl_src,output:enp5s0f0
>>>>>>>>>>
>>>>>>>
>>>>>>>>>
>>>>>>>>> The packets are not going out because you are sending the 
>>>>>>>>> packet
>>>>>>>>> on a
>>>>>>>>> real nic and not on a virtual inerface (veth pair) ?
>>>>>>>>
>>>>>>>> So for a real NIC we need to set the MAC addresses, maybe
>>>>>>>> some where
>>>>>>>> in the
>>>>>>>> documentation we should add an example on how to use
>>>>>>>> this feature?
>>>>>>>>
>>>>>>>>>> Maybe the test case can be made more realistic? Once I
>>>>>>>>>> understand the
>>>>>>>>>> failure, I can continue with the review.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>>
>>>>>>>>>> Eelco



More information about the dev mailing list