[ovs-dev] [RFC PATCH v2 00/13] Add Network Service Header Support
Jan Scheurich
jan.scheurich at ericsson.com
Wed Jul 13 10:25:52 UTC 2016
First of all, Ericsson fully supports the initiative to provide support for NSH encapsulation in OVS.
Based on earlier RFC patch sets and mailing list discussions the solution should be based on the following principles:
A. Align with existing OpenFlow paradigms and OVS implementation architecture
B. Separate NSH encapsulation from the underlying transport encapsulation (Ethernet, VXLAN-GPE, GRE, ...)
C. Handle the NSH encapsulation in the OF pipeline, not as tunnel ports
D. Re-use the TLV option mapping developed for Geneve for NSH metadata format MD2
The traffic cases that OVS should support include the following:
1. Push an NSH header onto an Ethernet packet and forward that with a suitable transport encapsulation (Classifier)
2. Receive an NSH packet with a supported transport encapsulation, remove transport encapsulation, match on NSH header, potentially modify it, and forward NSH packet with a suitable transport encapsulation (SFF)
3. Receive an NSH packet with a supported transport encapsulation, remove transport encapsulation, match on NSH header, pop the NSH header, and forward inner packet potentially using data extracted from the popped NSH header (SFF)
4. (future) Stateful NSH proxy popping NSH header on the way to an non-NSH SF and re-inserting the NSH header on return of the packet.
I will try to structure my general thoughts about the patch set below.
Push/pop NSH
-------------
The present patch implements the new push/pop_nsh actions in a way that their execution is postponed until the xlate context is committed, for example when the packet is sent out on a port. Apparently this is to resemble the behavior of a tunnel port.
In OpenFlow, however, push/pop actions have a well-defined semantic:
In an Apply Actions instruction they are executed immediately. After a push action the inner packet header is no longer visible to OF and all subsequent actions and matches refer to the new outer headers. Typically the ethertype of the modified packets changes with the push. After a pop action, the popped outer headers are no longer accessible (unless the pop action stores them in some metadata fields) and, if the ethertype changes, the retrieved inner packet is parsed for subsequent matches and actions.
In a Write Actions instruction the push action is saved in the packet’s action set for execution at the end of the pipeline (before output to port or group), but also then the OF spec specifies that any push action are executed *before* set_field actions, i.e. any buffered set_field actions would modify the outer header.
I believe that the proposed push/pop_nsh NXM OF actions should adhere to that scheme.
From an implementation perspective it means that push_nsh and pop_nsh actions should trigger xlate_commit_actions() in the same way as push/pop_mpls does today.
NSH packets as non-Ethernet packets in the OF pipeline
-------------------------------------------------------
The moment we introduce stand-alone push/pop_nsh actions we open up new ground in OVS. Until now a packet in the OF pipeline has always been an Ethernet frame. In the OF 1.5 spec this constraint has been removed by the introduction of the Packet type-aware pipeline (EXT-112) but that is not yet implemented in OVS.
Simon Horman’s v3 patch set “userspace: Support for L3 encapsulated packets” is the first step in that direction as it enables the ofproto-dpif slow path to process packets without an Ethernet header but with a valid packet ethertype. Matches on MAC addresses and set_field action for MAC addresses for such packets will fail.
I believe Simon’s patch does allow a clean semantic of a push/pop_nsh actions. Push_nsh would turn a packet into a non-Ethernet packet and set the ethertype to 0x894f for NSH. As such it should be possible to send out such an NSH packet on a non-Ethernet tunnel port such as VXLAN-GPE. Conversely a pop_nsh action should only be allowed if the packet is non-Ethernet with ethertype 0x894f.
I strongly second Jesse's view that the NSH patch set should be built on top of Simon's patch (unless someone implements full EXT-112 first).
Ethernet as transport encapsulation (push/pop_eth)
---------------------------------------------------
Simon’s current v3 patch set transparently pushes a dummy Ethernet header when sending a non-Ethernet packet out to a L2 port. There is no way how a controller could currently set the MAC addresses in this header, so this can only be partial solution to the problem.
For NSH over (non-GPE) VXLAN transport tunnel ports this would be OK as the middle MAC header between VXLAN and NSH headers is only a dummy and has no meaning. But for NSH over Ethernet transport, the controller must set the outer MAC addresses.
An explicit push_eth (optional parameter: ethertype) OF action in accordance with above standard push action semantics should solve this issue as it allows the controller to modify the outer MAC addresses with subsequent set_field actions.
The new push_eth action can be used in two scenarios:
a) On a non-Ethernet frame (e.g. NSH, MPLS, or IP):
This just adds a dummy Ethernet header in front of the non-Ethernet packet. The ethertype should be taken from the existing non-Ethernet frame. SFC will use this.
b) On an Ethernet frame:
This creates raw MAC in MAC encapsulation. The action should provide the ethertype to be used in the outer Ethernet header. Not used in SFC but included for generality.
Note: The standard OpenFlow push_pbb action copies the inner MAC addresses into the outer MAC header. To me this doesn’t make too much sense and I would suggest to not do that.
The new pop_eth action should leave a non-Ethernet frame with the original ethertype of the L2 frame. The only exception is if the inner packet itself is an Ethernet frame (MAC in MAC use case recognized by ethertype ??), in which case the ethertype should be taken from the inner Ethernet header and the inner packet should be re-parsed for further processing in the OF pipeline.
Access to NSH header fields in the OF pipeline
-----------------------------------------------
OpenFlow has two ways for this: packet header and metadata match fields. Packet header match fields are populated when parsing a packet with NSH ethertype and are only valid and accessible while the NSH header is in place (i.e. before pop_nsh or after push_nsh). Metadata fields, in contrast, would be populated by a pop_nsh action to save the content of the popped NSH header and would be used in a push_nsh action to populate the pushed header fields in the packet.
I doubt that the OVS community will accept a single set of NXM fields that can be used both as packet header and metadata fields interchangingly. That is really a fundamental question that we should agree up front. It will very much determine the OF interface that ODL SFC needs to adapt to.
If that is not acceptable, we should focus on one of these approaches, otherwise we would have to create two complete sets of NXM match fields for NSH.
The benefit of NSH packet header match fields is that an SFF can match on e.g. NSP and NSI without having to pop the NSH header and push again for transmission to an SF or SFF. This will be important for SFF performance as this is the most frequent traffic case
A benefit of NSH metadata fields might be that all NSH header data is implicitly kept with the packet after pop so that SFC controller developers don’t have to manually take care of copying NSH headers into NXM registers to use them after pop_nsh. Not sure how frequently this will actually be needed.
My preference would be to go for NSH packet header match fields as these are more general and optimal for the most frequent use case.
Re-use of TLV tunnel metadata mechanism
----------------------------------------
In principle I support the approach to re-use the existing OVS mechanism for mapping TLV protocol options to NXM_NX_TUN_METADATA<IDX> fields.
I have a couple of questions related to this
1. Is it appropriate to use the NXM_NX_TUN_METADATA<IDX> fields as packet header match fields that are populated when the OF pipeline parses an MD2 NSH packet and the controller has set up some TLV mappings for the NSH MD2 options. I’m concerned about the field name that suggests these are tunnel metadata.
2. Should a set_field action on such a mapped NXM_NX_TUN_METADATA<IDX> field be allowed to modify the TLV header of the NSH packet? This should be possible to do in-place as long as the length of the TLV is fixed.
3. The existing TLV mapping commands just include class, type and len. These are unique only within a given protocol like Geneve or NSH. To disambiguate and specify a TLV mapping only for an option of a given protocol, the command should also include some kind of protocol identifier.
Example SFC pipeline
------------------------
The following illustrates how an SFC pipeline could look like with OF-compliant push/pop_nsh and push/pop_eth actions, assuming that NSH match fields are packet header rather than metadata fields. Priorities are not signifcant:
Ingress table
# Classifier (result is an NSH encapsulated non-Ethernet packet with ethertype 0x894f)
table=0, priority=1000,tcp,in_port=2,nw_src=10.0.35.2,nw_dst=10.0.36.4 actions=push_nsh,actions=set_field:0->nsh_mdtype,set_field:3->nsh_np,set_field:4->nsp,set_field:255->nsi,set_field:1->nsh_c1,set_field:0x64->nsh_c2,set_field:3->nsh_c3,set_field:4->nsh_c4
# NSH over Ethernet frame from local vSF connected to port 3
table=0, priority=1000,in_port=3,dl_dst=88:f0:31:b5:12:b5,eth_type=0x894f actions=pop_eth,goto_table:4
Main SFF table (packet must be non-Ethernet packet with NSH header, ethertype=0x894f)
# SFF flow entry for output to local vSF on port 2
table=4, priority=100,eth_type=0x894f,nsi=255,nsp=4 actions=push_eth,set_field:88:f0:31:b5:12:b5->eth_src,set_field:00:00:00:00:35:02->eth_dst,output:2
# SFF flow entry for forwarding to VXLAN tunnel port 100 (pushing a dummy Ethernet header)
table=4, priority=100,eth_type=0x894f,nsi=254,nsp=4 actions=push_eth,output:100
# SFF flow entry for forwarding to VXLAN-GPE tunnel port 200
table=4, priority=100,eth_type=0x894f,nsi=253,nsp=4 actions=output:200
# SFF flow entry for terminating RSP and forwarding decapsulated packet (saving the NSH C2 metadata in register to be used as e.g. VRF tag in IP forwarding decision)
table=4, priority=100,eth_type=0x894f,nsi=252,nsp=4 actions=move:nsh_c2->NXM_NX_REG0[],pop_nsh,goto_table:10
Exit IP forwarding table
table=10, priority=100,NXM_NX_REG0=0x64,ip,nw_dst=10.0.36.4 actions=push_eth,set_field:22:33:44:55:66:77->eth_src,set_field:11:22:33:44:55:66->eth_dst,output:4
Any comments are welcome.
Regards, Jan
> -----Original Message-----
> From: dev [mailto:dev-bounces at openvswitch.org] On Behalf Of Johnson Li
> Sent: Tuesday, 12 July, 2016 19:26
> To: dev at openvswitch.org
> Subject: [ovs-dev] [RFC PATCH v2 00/13] Add Network Service Header
> Support
>
> IETF draft at:
> https://tools.ietf.org/html/draft-ietf-sfc-nsh-01
> defines a new protocol named Network Service Header (NSH) for Service
> Function Chaining. The NSH format looks like below:
>
> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> |Ver|O|C|R|R|R|R|R|R| Length | MD Type | Next Proto |
> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> | Service Path ID | Service Index |
> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> | |
> ~ Mandatory/Optional Context Header ~
> | |
> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>
> In this patch set, we implement the NSH support for the OVS which then can
> be used as Service Function Forwarder. NSH is transport independent by
> design, and VxLAN-GPE and Ethernet are targeted transports being
> supported by OVS initially.
>
> The implementation for VxLAN-GPE is upstreamed by Jiri Benc at Linux
> kernel tree commit <e1e5314de08ba6003b358125eafc9ad9e75a950c>
>
> while adding VxLAN-GPE support, the Ethernet type of the VxLAN-GPE
> tunneling port is set to ARPHRD_NONE, which breaks the assumption that all
> frames communicated between OVS data plane and tunneling ports should
> start from a Ethernet header. Hence Simon Horman submitted a patch set to
> enable the raw protocol support at:
> http://openvswitch.org/pipermail/dev/2016-June/072010.html
>
> In order to support NSH without depending on Simon's patch, we introduced
> new flow actions push_eth and pop_eth to support the Ethernet as a NSH
> transport. The new actions append a new Ethenet header to carry NSH frame
> or remove the Ethernet header from the NSH frame. We reused Simon's
> code for the data path implementation and added explicit flow actions in
> control plane.
>
> Basic NSH steering test case:
>
> 172.168.60.101/24 172.168.60.102/24
> +--------------+ +--------------+
> | br-int | | br-int |
> +--------------+ +--------------+
> | vxlan0 | | vxlan0 |
> +--------------+ +--------------+
> | |
> | |
> | |
> 192.168.50.101/24 192.168.50.102/24
> +--------------+ +---------------+
> | br-eth1 | | br-eth1 |
> +--------------+ +---------------+
> | eth1 |----------------------| eth1 |
> +--------------+ +---------------+
>
> Node 1 with OVS. Node 2 with OVS.
>
> Configure Node 1:
> Step 1: Create VxLAN port
> $ovs-vsctl add-port br-int vxlan0 -- set interface vxlan0 \
> type=vxlan options:remote_ip=192.168.50.102 options:key=flow \
> options:dst_port=4789
> Step 2: Add flows for Egress
> $ovs-ofctl add-flow br-int "table=0, priority=260, in_port=LOCAL \
> actions=load:0x1->NXM_NX_NSP[],load:0xFF->NXM_NX_NSI[],\
> load:1->NXM_NX_NSH_MDTYPE[],load:0x3->NXM_NX_NSH_NP[],\
> load:0x11223344->NXM_NX_NSH_C1[],load:0x55667788-
> >NXM_NX_NSH_C2[],\
> load:0x99aabbcc->NXM_NX_NSH_C3[],load:0xddeeff00-
> >NXM_NX_NSH_C4[],\
> push_nsh,push_eth,output:1"
> Step 3: Add flow for Ingress
> $ovs-ofctl add-flow br-int "table=0, priority=260, in_port=1,\
> nsh_mdtype=1, nsp=0x800001, nsi=0xFF, nshc1=0xddeeff00,\
> nshc2=0x99aabbcc, nshc3=0x55667788, nshc4=0x11223344, \
> actions=pop_eth,pop_nsh,output:LOCAL"
>
> Configure Node 2:
> Step 1: Create VxLAN port
> $ovs-vsctl add-port br-int vxlan0 -- set interface vxlan0 \
> type=vxlan options:remote_ip=192.168.50.101 options:key=flow \
> options:dst_port=4789
> Step 2: Add flows for Egress
> $ovs-ofctl add-flow br-int "table=0, priority=260, in_port=LOCAL \
> actions=load:0x800001->NXM_NX_NSP[],load:0xFF->NXM_NX_NSI[],\
> load:1->NXM_NX_NSH_MDTYPE[],load:0x3->NXM_NX_NSH_NP[],\
> load:0xddeeff00->NXM_NX_NSH_C1[],load:0x99aabbcc-
> >NXM_NX_NSH_C2[],\
> load:0x55667788->NXM_NX_NSH_C3[],load:0x11223344-
> >NXM_NX_NSH_C4[],\
> push_nsh,push_eth,output:1"
> Step 3: Add flow for Ingress
> $ovs-ofctl add-flow br-int "table=0, priority=260, in_port=1,\
> nsh_mdtype=1, nsp=0x1, nsi=0xFF, nshc1=0x11223344,\
> nshc2=0x55667788, nshc3=0x99aabbcc, nshc4=0xddeeff00, \
> actions=pop_eth,pop_nsh,output:LOCAL"
>
> ---
> Change Log:
> V1->V2: 1. Add prototype for MD type 2 support. Since the MD type 2
> uses the same TLV format of Geneve, so this patch set
> implements a prototype for MD type 2 support by reusing
> the TUN_METADATA APIs and data structures for Geneve. BTW,
> all the Geneve specific APIs will be renamed to be more
> generic in the following patch iterations if this approach
> makes sense.
> 2. Add Ethernet transport support. To remove the dependency
> on Simon' patches for raw protocol support, we enable
> Ethernet as a supported NSH transport. Hence explicit flow
> actions push_eth/pop_eth to push/pop Ethernet header are added.
>
> Johnson Li (13):
> Add NSH keys as match fields for user space flow table
> Format NSH keys to readable strings
> Add key attributes of Network Service Header
> Add APIs to set NSH keys for match fields
> Add Meta flow key for NSH header
> Parse NSH header in function flow_extract in user space
> Userspace: Parse NSH header in flow_extract
> Add "pop_nsh/push_nsh" flow action for OVS control plane
> commit control plane action to data plane
> Add push_eth/pop_eth flow actions for kernel data path
> Add control plane command push_eth and pop_eth
> Commit push_eth/pop_eth flow action to data plane
> Format ODP action for push_eth/pop_eth flow actions
>
> datapath/linux/compat/include/linux/openvswitch.h | 44 +++
> include/openvswitch/flow.h | 5 +-
> include/openvswitch/match.h | 12 +
> include/openvswitch/meta-flow.h | 126 +++++++
> include/openvswitch/ofp-actions.h | 7 +
> include/openvswitch/packets.h | 19 +
> lib/dpif-netdev.c | 4 +
> lib/dpif.c | 4 +
> lib/flow.c | 85 +++++
> lib/match.c | 48 +++
> lib/meta-flow.c | 152 ++++++++
> lib/nx-match.c | 18 +
> lib/odp-execute.c | 12 +
> lib/odp-util.c | 418 +++++++++++++++++++++-
> lib/ofp-actions.c | 155 ++++++++
> lib/packets.h | 48 +++
> ofproto/ofproto-dpif-sflow.c | 5 +
> ofproto/ofproto-dpif-xlate.c | 73 ++++
> tests/ofproto.at | 10 +-
> 19 files changed, 1241 insertions(+), 4 deletions(-)
>
> --
> 1.8.4.2
>
> --------------------------------------------------------------
> Intel Research and Development Ireland Limited Registered in Ireland
> Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
> Registered Number: 308263
>
>
> This e-mail and any attachments may contain confidential material for the
> sole use of the intended recipient(s). Any review or distribution by others is
> strictly prohibited. If you are not the intended recipient, please contact the
> sender and delete all copies.
>
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> http://openvswitch.org/mailman/listinfo/dev
More information about the dev
mailing list