[ovs-dev] [PATCH v4 1/2] Encap & Decap actions for MPLS packet type.

Martin Varghese martinvarghesenokia at gmail.com
Tue Apr 6 09:02:56 UTC 2021


On Tue, Apr 06, 2021 at 10:54:32AM +0200, Eelco Chaudron wrote:
> 
> 
> On 6 Apr 2021, at 10:27, Martin Varghese wrote:
> 
> > On Thu, Apr 01, 2021 at 11:32:06AM +0200, Eelco Chaudron wrote:
> > > 
> > > 
> > > On 1 Apr 2021, at 11:28, Martin Varghese wrote:
> > > 
> > > > On Thu, Apr 01, 2021 at 11:17:14AM +0200, Eelco Chaudron wrote:
> > > > > 
> > > > > 
> > > > > On 1 Apr 2021, at 11:09, Martin Varghese wrote:
> > > > > 
> > > > > > On Thu, Apr 01, 2021 at 10:54:42AM +0200, Eelco Chaudron wrote:
> > > > > > > 
> > > > > > > 
> > > > > > > On 1 Apr 2021, at 10:35, Martin Varghese wrote:
> > > > > > > 
> > > > > > > > On Thu, Apr 01, 2021 at 08:59:27AM +0200, Eelco Chaudron wrote:
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > On 1 Apr 2021, at 6:10, Martin Varghese wrote:
> > > > > > > > > 
> > > > > > > > > > On Wed, Mar 31, 2021 at 03:59:40PM
> > > > > > > > > > +0200, Eelco Chaudron wrote:
> > > > > > > > > > > 
> > > > > > > > > > > 
> > > > > > > > > > > On 26 Mar 2021, at 7:21, Martin Varghese wrote:
> > > > > > > > > > > 
> > > > > > > > > > > > From: Martin Varghese <martin.varghese at nokia.com>
> > > > > > > > > > > > 
> > > > > > > > > > > > The encap & decap actions are extended to support MPLS
> > > > > > > > > > > > packet type.
> > > > > > > > > > > > Encap & decap actions adds and removes MPLS
> > > > > > > > > > > > header at start of the
> > > > > > > > > > > > packet.
> > > > > > > > > > > 
> > > > > > > > > > > Hi Martin,
> > > > > > > > > > > 
> > > > > > > > > > > I’m trying to do some real-life testing, and
> > > > > > > > > > > I’m running into
> > > > > > > > > > > issues. This
> > > > > > > > > > > might be me setting it up wrongly but just
> > > > > > > > > > > wanting to confirm…
> > > > > > > > > > > 
> > > > > > > > > > > I’m sending an MPLS packet that
> > > > > > > > > > > contains an ARP packet into a
> > > > > > > > > > > physical port.
> > > > > > > > > > > This is the packet:
> > > > > > > > > > > 
> > > > > > > > > > > Frame 4: 64 bytes on wire (512 bits), 64 bytes
> > > > > > > > > > > captured (512 bits)
> > > > > > > > > > >     Encapsulation type: Ethernet (1)
> > > > > > > > > > >     [Protocols in frame: eth:ethertype:mpls:data]
> > > > > > > > > > > Ethernet II, Src: 00:00:00_00:00:01 (00:00:00:00:00:01), Dst:
> > > > > > > > > > > 00:00:00_00:00:02 (00:00:00:00:00:02)
> > > > > > > > > > >     Destination: 00:00:00_00:00:02 (00:00:00:00:00:02)
> > > > > > > > > > >         Address: 00:00:00_00:00:02 (00:00:00:00:00:02)
> > > > > > > > > > >         .... ..0. .... .... ....
> > > > > > > > > > > .... = LG bit: Globally unique
> > > > > > > > > > > address
> > > > > > > > > > > (factory default)
> > > > > > > > > > >         .... ...0 .... .... .... .... = IG bit:
> > > > > > > > > > > Individual address
> > > > > > > > > > > (unicast)
> > > > > > > > > > >     Source: 00:00:00_00:00:01 (00:00:00:00:00:01)
> > > > > > > > > > >         Address: 00:00:00_00:00:01 (00:00:00:00:00:01)
> > > > > > > > > > >         .... ..0. .... .... ....
> > > > > > > > > > > .... = LG bit: Globally unique
> > > > > > > > > > > address
> > > > > > > > > > > (factory default)
> > > > > > > > > > >         .... ...0 .... .... .... .... = IG bit:
> > > > > > > > > > > Individual address
> > > > > > > > > > > (unicast)
> > > > > > > > > > >     Type: MPLS label switched packet (0x8847)
> > > > > > > > > > > MultiProtocol Label Switching Header, Label: 100, Exp: 0, S:
> > > > > > > > > > > 1, TTL:
> > > > > > > > > > > 64
> > > > > > > > > > >     0000 0000 0000 0110 0100 .... .... .... = MPLS Label: 100
> > > > > > > > > > >     .... .... .... .... .... 000.
> > > > > > > > > > > .... .... = MPLS Experimental
> > > > > > > > > > > Bits: 0
> > > > > > > > > > >     .... .... .... .... .... ...1 .... .... = MPLS
> > > > > > > > > > > Bottom Of Label
> > > > > > > > > > > Stack: 1
> > > > > > > > > > >     .... .... .... .... .... .... 0100 0000 = MPLS TTL: 64
> > > > > > > > > > > Data (46 bytes)
> > > > > > > > > > > 
> > > > > > > > > > > 0000  ff ff ff ff ff ff 52 54 00 88 51 38 08 06 00 01
> > > > > > > > > > > ......RT..Q8....
> > > > > > > > > > > 0010  08 00 06 04 00 01 52 54 00 88 51 38 01 01 01 65
> > > > > > > > > > > ......RT..Q8...e
> > > > > > > > > > > 0020  00 00 00 00 00 00 01 01 01 64 27 98 a0 47
> > > > > > > > > > > .........d'..G
> > > > > > > > > > >     Data:
> > > > > > > > > > > ffffffffffff525400885138080600010800060400015254008851380101016500000000?
> > > > > > > > > > > 
> > > > > > > > > > > 
> > > > > > > > > > > I’m trying to use the following rules:
> > > > > > > > > > > 
> > > > > > > > > > >   ovs-ofctl del-flows ovs_pvp_br0
> > > > > > > > > > >   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
> > > > > > > > > > > "priority=100,dl_type=0x8847,mpls_label=100
> > > > > > > > > > > actions=decap(),decap(packet_type(ns=0,type=0x806)),resubmit(,3)"
> > > > > > > > > > >   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
> > > > > > > > > > > "table=3,priority=10
> > > > > > > > > > > actions=normal"
> > > > > > > > > > > 
> > > > > > > > > > > With these, I expect the packet to be sent to vnet0, but
> > > > > > > > > > > it’s not.
> > > > > > > > > > > Actually,
> > > > > > > > > > > the datapath rule looks odd, while the userspace rules seem
> > > > > > > > > > > to match:
> > > > > > > > > > > 
> > > > > > > > > > >   $ ovs-dpctl dump-flows
> > > > > > > > > > >   recirc_id(0),in_port(1),eth(),eth_type(0x8847),mpls(label=100/0xfffff,tc=0/0,ttl=0/0x0,bos=1/1),
> > > > > > > > > > > packets:13, bytes:1118, used:0.322s,
> > > > > > > > > > > actions:pop_eth,pop_mpls(eth_type=0x806),recirc(0x19a)
> > > > > > > > > > >   recirc_id(0x19a),in_port(1),eth_type(0x0806), packets:13,
> > > > > > > > > > > bytes:884,
> > > > > > > > > > > used:0.322s, actions:drop
> > > > > > > > > > > 
> > > > > > > > > > >   $ ovs-ofctl dump-flows ovs_pvp_br0 -O OpenFlow13
> > > > > > > > > > >   cookie=0x0, duration=85.007s, table=0, n_packets=51,
> > > > > > > > > > > n_bytes=4386,
> > > > > > > > > > > priority=100,mpls,mpls_label=100
> > > > > > > > > > > actions=decap(),decap(packet_type(ns=0,type=0x806)),resubmit(,3)
> > > > > > > > > > >   cookie=0x0, duration=84.990s, table=3, n_packets=51,
> > > > > > > > > > > n_bytes=3468,
> > > > > > > > > > > priority=10 actions=NORMAL
> > > > > > > > > > > 
> > > > > > > > > > The inner packet is ethernet. So the packet type should be
> > > > > > > > > > (ns=0,type=0)
> > > > > > > > > > ?
> > > > > > > > > 
> > > > > > > > > Forgot to add that I already tried that to start
> > > > > > > > > with, based on the
> > > > > > > > > example,
> > > > > > > > > but as that did not work I tried 0x806.
> > > > > > > > > 
> > > > > > > > > PS: I have this as a remark in my review notes, i.e., to
> > > > > > > > > explain the
> > > > > > > > > ns and
> > > > > > > > > type usage here.
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > This resulted in packets being counted at the open flow
> > > > > > > > > level, but it
> > > > > > > > > results in NO data path rules. Do get an error though:
> > > > > > > > > 
> > > > > > > > > 2021-04-01T06:53:36.056Z|00141|dpif(handler37)|WARN|system at ovs-system:
> > > > > > > > > failed to put[create] (Invalid argument)
> > > > > > > > > ufid:3d2d6f6d-5a66-4ace-8b09-7cdcfa5efc8e recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(1),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=00:00:00:00:00:01/00:00:00:00:00:00,dst=00:00:00:00:00:02/00:00:00:00:00:00),eth_type(0x8847),mpls(label=100/0xfffff,tc=0/0,ttl=64/0x0,bos=1/1),
> > > > > > > > > actions:pop_eth,pop_mpls(eth_type=0x6558),set(eth()),recirc(0x4c)
> > > > > > > > 
> > > > > > > > This set(eth) before the recirc is the problem i guesss. I need
> > > > > > > > to check
> > 
> > I could reproduce the problem. It has nothing to do with ARP or IP.
> > Unlike my test scripts, in your test you are setting the mac address
> > after the encap action
> > 
> > Ovs-vswitchd is programming a set(eth(dst) action between the  pop_mpls
> > and  recirc as it sees a difference in mac address in flow structure and
> > base_flow structure.
> > 
> > The mac address in flow structure is not cleared in PT_ETH handling of
> > xlate_generic_decap_action but  it is cleared in base_flow in the
> > decap handling of PT_ETH in commit_encap_decap_action function
> > 
> > Due to this difference the set(eth(dst) action will be programmed to the
> > datapath.
> > 
> > Also, I see that in  commit_set_ether_action Function
> > “flow->packet_type != htonl(PT_ETH)” is used to check if the packet is
> > ethernet instead of base_flow->packet_type.
> > 
> > I assume check on base_flow->packet_type make more sense here ?
> > 
> > I tried to fix this issue in 2 different ways.
> > 
> > 1   I have cleared the mac address in flow structure  in PT_ETH handling
> > of xlate_generic_decap action.
> > 
> > 2  In the  commit_set_ether action I changed the check from
> > “flow->packet_type != htonl(PT_ETH)” to  “base_flow->packet_type !=
> > htonl(PT_ETH))”.
> > 
> > Though both of them solves this problem, couple of NSH regression tests
> > are failing
> > 
> > 2291: nsh - md1 encap over a veth link                FAILED (nsh.at:85)
> > 
> > 58022292: nsh - md2 encap over a veth link                FAILED
> > (nsh.at:213)
> > 
> > I see that they are failing as they are expecting a set(eth(dst)
> > between the the pop_nsh and the recirc.
> > 
> > Set(eth) action is because of the reasons explained above –
> > 
> > Datapath actions: push_nsh(flags=0,ttl=63,mdtype=1,np=3,spi=0x1234,si=255,c1=0x11223344,c2=0x0,c3=0x0,c4=0x0),push_eth(src=00:00:00:00:00:00,dst=11:22:33:44:55:66),pop_eth,pop_nsh(),set(eth(dst=11:22:33:44:55:66)),recirc(0x1)
> > 
> > In my understanding set(eth) here  is wrong as there is no set ethernet
> > action in the userspace rule
> > - Hide quoted text -
> > 
> > table=0,in_port=4,dl_type=0x894f,nsh_mdtype=1,nsh_spi=0x1234,nsh_c1=0x11223344,actions=decap(),decap(),2
> > 
> > 
> > 
> > Could someone comment ?
> 
> Maybe Jan can answer as he did the NSH implementation, however, what would
> be of interest if you can give me an example of how the encap() decap() for
> this would be used in real life so I’m sure I’m testing it correctly?
> 
> What I did so far was to encapsulate all traffic going from a VM to the
> physical port in MPLS using the flows like:
> 
> ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 "priority=100,in_port=vnet0,actions=encap(mpls(ether_type=0x8847)),set_mpls_label:100,encap(ethernet),,set_field:00:00:00:00:00:02->dl_ds
> t,set_field:00:00:00:00:00:01->dl_src,output:enp5s0f0"
> 
> Then I would capture this traffic and sent it back over the same port,
> hoping it would come out as plane traffic with the following rule:
> 
> ovs-ofctl add-flow -O OpenFlow15 ovs_pvp_br0
> "priority=100,dl_type=0x8847,mpls_label=100
> actions=decap(),decap(packet_type(ns=0,type=0)),resubmit(,3)"
> ovs-ofctl add-flow -O OpenFlow15 ovs_pvp_br0 "table=3,priority=10
> actions=normal"
> 
> If this is correct, let me know, and if Jan does not reply, I’ll try to
> understand the code in this area and see if I can find out some details…
>

You are correct.
> //Eelco
> 
> > > > > > > > > 2021-04-01T06:53:36.056Z|00142|dpif(handler37)|WARN|system at ovs-system:
> > > > > > > > > execute
> > > > > > > > > pop_eth,pop_mpls(eth_type=0x6558),set(eth()),recirc(0x4c)
> > > > > > > > > failed
> > > > > > > > > (Invalid argument) on packet mpls,vlan_tci=0x0000,dl_src=00:00:00:00:00:01,dl_dst=00:00:00:00:00:02,mpls_label=100,mpls_tc=0,mpls_ttl=64,mpls_bos=1
> > > > > > > > >  with metadata skb_priority(0),skb_mark(0),in_port(1) mtu 0
> > > > > > > > > 
> > > > > > > > > Are there missing parts in my kernel that do not get properly
> > > > > > > > > detected by
> > > > > > > > > the feature detection?
> > > > > > > > > 
> > > > > > > > > $ ovs-appctl dpif/show-dp-features ovs_pvp_br0
> > > > > > > > > Masked set action: Yes
> > > > > > > > > Tunnel push pop: No
> > > > > > > > > Ufid: Yes
> > > > > > > > > Truncate action: Yes
> > > > > > > > > Clone action: Yes
> > > > > > > > > Sample nesting: 10
> > > > > > > > > Conntrack eventmask: Yes
> > > > > > > > > Conntrack clear: Yes
> > > > > > > > > Max dp_hash algorithm: 0
> > > > > > > > > Check pkt length action: Yes
> > > > > > > > > Conntrack timeout policy: Yes
> > > > > > > > > Explicit Drop action: No
> > > > > > > > > Optimized Balance TCP mode: No
> > > > > > > > > l2 MPLS tunnelling: Yes
> > > > > > > > > Max VLAN headers: 2
> > > > > > > > > Max MPLS depth: 3
> > > > > > > > > Recirc: Yes
> > > > > > > > > CT state: Yes
> > > > > > > > > CT zone: Yes
> > > > > > > > > CT mark: Yes
> > > > > > > > > CT label: Yes
> > > > > > > > > CT state NAT: Yes
> > > > > > > > > CT orig tuple: Yes
> > > > > > > > > CT orig tuple for IPv6: Yes
> > > > > > > > > IPv6 ND Extension: No
> > > > > > > > > 
> > > > > > > > You are good
> > > > > > > > 
> > > > > > > > I am not sure what is going wrong. Your test case looks same as
> > > > > > > > the unit
> > > > > > > > test i added.
> > > > > > > > 
> > > > > > > > I tried myself again and this is i get
> > > > > > > > 
> > > > > > > > ovs-ofctl -O OpenFlow13 add-flow br_mpls2
> > > > > > > > "in_port=$egress_port,dl_type=0x8847
> > > > > > > > +actions=decap(),decap(packet_type(ns=0,type=0),goto_table:1"
> > > > > > > > ovs-ofctl -O OpenFlow13 add-flow br_mpls2
> > > > > > > > +"table=1,in_port=$egress_port,dl_type=0x0800,nw_dst=1.1.1.2
> > > > > > > > +actions=set_field:00:00:00:00:00:02->dl_dst,set_field:00:00:00:00:00:01->dl_sr
> > > > > > > > +c output:$ingress_port"
> > > > > > > > 
> > > > > > > > recirc_id(0x3),in_port(6),eth(src=36:b1:ee:7c:01:03,dst=36:b1:ee:7c:01:02),eth_
> > > > > > > > +type(0x0800),ipv4(dst=1.1.1.2,frag=no), packets:3, bytes:294,
> > > > > > > > used:0.837s,
> > > > > > > > +actions:set(eth(src=00:00:00:00:00:01,dst=00:00:00:00:00:02)),4
> > > > > > > > recirc_id(0),in_port(6),eth(),eth_type(0x8847),mpls(label=0/0x0,tc=0/0,ttl=0/0x
> > > > > > > > +0,bos=1/1), packets:3, bytes:348, used:0.837s,
> > > > > > > > +actions:pop_eth,pop_mpls(eth_type=0x6558),recirc(0x3)
> > > > > > > > 
> > > > > > > > The packet to the ovs is
> > > > > > > > ETH|MPLS|ETH|IP ?
> > > > > > > > How it is differnt from you test case?
> > > > > > > 
> > > > > > > Mine is ETH|MPLS|ETH|ARP, which works fine with pop_mpls
> > > > > > > 
> > > > > > I am wondering how old mpls pop  action works
> > > > > > could you please put down the userspace and datapath rules when
> > > > > > you used
> > > > > > pop_mpls.
> > > > > > 
> > > > > > In my understanding it can never work as what you have
> > > > > > after MPLS is
> > > > > > ethernet and not ARP.
> > > > > 
> > > > > It’s ethernet + ARP, but here are my rules:
> > > > 
> > > > To clarify
> > > > 
> > > > The test vector for Decap Test case
> > > > ETH|MPLS|ETH|ARP
> > > > The test vector for pop mpls test case
> > > > ETH|MPLS|ARP|
> > > > 
> > > > The above understanding correct?
> > > 
> > > Guess our emails crossed ;)  I was sending in the same packet for
> > > both the
> > > test cases, so
> > > 
> > > ETH|MPLS|ETH|ARP
> > > 
> > > Which with decap() is resulting in the rules not being programmed
> > > 
> > > With popmpls I saw the packets in being received, but did not notice
> > > the
> > > incorrect use of popmpls so my packet after popmpls looks like
> > > ETH|ETH|ARP.
> > > 
> > > So I guess all that remains is why the data path rules is not
> > > accepted for
> > > ARP with the decap.
> > > 
> > > > > 
> > > > > dpctl:
> > > > > 
> > > > > recirc_id(0),in_port(2),eth(),eth_type(0x8847),mpls(label=100/0xfffff,tc=0/0,ttl=0/0x0,bos=1/1),
> > > > > packets:64, bytes:5504, used:0.444s,
> > > > > actions:pop_mpls(eth_type=0x806),recirc(0x80d)
> > > > > recirc_id(0x80d),in_port(2),eth(src=00:00:00:00:00:01,dst=00:00:00:00:00:02),eth_type(0x0806),
> > > > > packets:64, bytes:5248, used:0.444s, actions:3,1
> > > > > 
> > > > > ofctl:
> > > > > 
> > > > > OFPST_FLOW reply (OF1.5) (xid=0x2):
> > > > >  cookie=0x0, duration=178.890s, table=0, n_packets=127,
> > > > > n_bytes=10922,
> > > > > idle_age=0, priority=100,mpls,mpls_label=100
> > > > > actions=pop_mpls:0x0806,resubmit(,3)
> > > > >  cookie=0x0, duration=178.873s, table=3, n_packets=127,
> > > > > n_bytes=10414,
> > > > > idle_age=0, priority=10 actions=NORMAL
> > > > > 
> > > > > 
> > > > > > > > Thanks for your time.
> > > > > > > 
> > > > > > > Your welcome
> > > > > > > 
> > > > > > > > > > > 
> > > > > > > > > > > If I use the old way, doing pop_mpls, it works fine:
> > > > > > > > > > > 
> > > > > > > > > > > 
> > > > > > > > > > > ovs-ofctl del-flows ovs_pvp_br0
> > > > > > > > > > > ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
> > > > > > > > > > > "priority=100,dl_type=0x8847,mpls_label=100
> > > > > > > > > > > actions=pop_mpls:0x0806,resubmit(,3)"
> > > > > > > > > > > ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
> > > > > > > > > > > "table=3,priority=10
> > > > > > > > > > > actions=normal"
> > > > > > > > > > > 
> > > > > > > > > > > 
> > > > > > > > > > > I also noticed (despite the test example) to make
> > > > > > > > > > > encap work, I had
> > > > > > > > > > > to set
> > > > > > > > > > > the ethernet MAC addresses, or else the packets were not
> > > > > > > > > > > getting out.
> > > > > > > > > > > So something like:
> > > > > > > > > > > 
> > > > > > > > > > >   ovs-ofctl add-flow -O OpenFlow13
> > > > > > > > > > > ovs_pvp_br0 "priority=100,in_port=vnet0,actions=encap(mpls(ether_type=0x8847)),set_mpls_label:100,encap(ethernet),,set_field:00:00:00:00:00:02->dl_dst,set_field:00:00:00:00:00:01->dl_src,output:enp5s0f0
> > > > > > > > > > > 
> > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > The packets are not going out because
> > > > > > > > > > you are sending the packet
> > > > > > > > > > on a
> > > > > > > > > > real nic and not on a virtual inerface (veth pair) ?
> > > > > > > > > 
> > > > > > > > > So for a real NIC we need to set the MAC addresses, maybe
> > > > > > > > > some where
> > > > > > > > > in the
> > > > > > > > > documentation we should add an example on how to use
> > > > > > > > > this feature?
> > > > > > > > > 
> > > > > > > > > > > Maybe the test case can be made more realistic? Once I
> > > > > > > > > > > understand the
> > > > > > > > > > > failure, I can continue with the review.
> > > > > > > > > > > 
> > > > > > > > > > > 
> > > > > > > > > > > Cheers,
> > > > > > > > > > > 
> > > > > > > > > > > Eelco
> 


More information about the dev mailing list