[ovs-dev] NSH Option 2 implementation

[Jan Scheurich]

> From: Jesse Gross [mailto:jesse at kernel.org]
> Sent: Friday, 16 September, 2016 01:38
> 
> I think the main issue is that when packets are being tunneled and NSH is in use, the encapsulation format (and therefore OVS port
> type) is not NSH - it's typically VXLAN-GPE. This is conceptually similar to MPLS over GRE, which we don't currently support without an
> intervening Ethernet header. However, if we did it would presumably use the existing push/pop MPLS actions.

I think there is general consensus that the OF pipeline should send NSH-encapsulated packets to a VXLAN-GPE port or a GRE port for transport encapsulation. To avoid the unwanted sandwich MAC header, we are dependent on Simon's L3 tunneling patch and the tunnel port must be configured as L3 (or rather non-Ethernet).

> It seems like the only way to do this cleanly for both the NSH and MPLS cases for tunneled and non-tunneled packets is to have
> push/pop Ethernet and push/pop MPLS/NSH actions. This would allow the headers to be built up incrementally in whatever form is
> appropriate.
> Outputting to a tunnel or a physical port wouldn't need any actions that are different at the tagging layer.

Outputting an NSH packet to a physical (or, in fact, any Ethernet) port does require an outer Ethernet header with Ethertype 0x894F.

We need to decide between two fundamental approaches (just looking at the encap direction, the decap direction is analogous):

1) Combined push_nsh(_eth) action to push both NSH and outer MAC header with Ethertype 0x894F.

This is straightforward and maintains the current model of OVS as Ethernet-only OpenFlow pipeline. The outer MAC addresses can be modified with normal set_field actions between push and output. When sending the NSH packet to a VXLAN-GPE or GRE L3 tunnel, the outer MAC header would be stripped. Internally, in the datapath, one could optimize the temporary Ethernet header insertion away.

The push/pop_pbb actions standardized in OF (though not implemented in OVS) follow the same model.


2) Modular push_nsh and push_eth actions

This would give the controller the flexibility to push an NSH header alone, e.g. for transmission to a L2 tunnel port. The push_eth action would only be needed when sending to an Ethernet port.

A stand-alone push_nsh action creates a non-Ethernet packet in the OF pipeline. OVS today is not prepared to handle non-Ethernet packets in the pipeline (The packet-type aware pipeline has been standardized in OF 1.5 but is not implemented yet). 

Even if we find a way to avoid dependency on the actual OF 1.5 control plane impacts, I think some infrastructure will be needed in OVS (datapaths and slow path) to deal with non-Ethernet packets correctly. The current work to support L3 tunneling would also have to be reconsidered in when we go that way.

It is not clear what use cases there are for a stand-alone push_eth action, as a raw MAC in MAC encapsulation is not really standardized in IETF.


Option 1) would be sufficient to cover all NSH use cases known today. It is straightforward and only depends on the L3 tunneling patches that are close to being finalized.

Option 2) implies more work and additional dependencies of NSH support on generic infrastructure work. It might be more flexible but it is not clear if there are use cases that would benefit from the additional flexibility.

Given all that, my suggestion is to go for option 1) now with the chance of adding more modular actions once OVS provides the infrastructure and OF control plane to deal with non-Ethernet packets.

> 
> One other comment, unrelated to the above discussion, is that NSH is continuing to make incompatible wire format changes.
> Specifically, the option length field was recently changed from representing four byte multiples to be the actual length. This brings up
> two questions:
>  * Obviously the patches will need to be updated to handle this but also it means that the option format is no longer exactly the same
> as Geneve options. This will require some thought on how to support both protocols while maximizing code reuse.
>  * OVS strives to maintain consistent behavior for external interfaces and it's not clear how this would be possible if the protocol
> continues to change in the future.

My take on this is that we could declare the NSH support in OVS "experimental" at least until the NSH drafts have reached RFC stage. Until then everybody should be prepared for non-backward-compatible changes in OVS to follow the NSH drafts.

Another approach could be to start with the stable MD1 header format first, but that has other drawbacks as discussed. I would strive to implement at least experimental support for MD2 TLV headers, too.