[ovs-dev] NSH Option 2 implementation

[Jesse Gross]

On Tue, Sep 20, 2016 at 3:52 AM, Jan Scheurich
<jan.scheurich at ericsson.com> wrote:
>> From: Jesse Gross [mailto:jesse at kernel.org]
>> Sent: Friday, 16 September, 2016 01:38
>>
>> I think the main issue is that when packets are being tunneled and NSH is in use, the encapsulation format (and therefore OVS port
>> type) is not NSH - it's typically VXLAN-GPE. This is conceptually similar to MPLS over GRE, which we don't currently support without an
>> intervening Ethernet header. However, if we did it would presumably use the existing push/pop MPLS actions.
>
> I think there is general consensus that the OF pipeline should send NSH-encapsulated packets to a VXLAN-GPE port or a GRE port for transport encapsulation. To avoid the unwanted sandwich MAC header, we are dependent on Simon's L3 tunneling patch and the tunnel port must be configured as L3 (or rather non-Ethernet).

Yes, I agree.

>> It seems like the only way to do this cleanly for both the NSH and MPLS cases for tunneled and non-tunneled packets is to have
>> push/pop Ethernet and push/pop MPLS/NSH actions. This would allow the headers to be built up incrementally in whatever form is
>> appropriate.
>> Outputting to a tunnel or a physical port wouldn't need any actions that are different at the tagging layer.
>
> Outputting an NSH packet to a physical (or, in fact, any Ethernet) port does require an outer Ethernet header with Ethertype 0x894F.
>
> We need to decide between two fundamental approaches (just looking at the encap direction, the decap direction is analogous):
>
> 1) Combined push_nsh(_eth) action to push both NSH and outer MAC header with Ethertype 0x894F.
>
> This is straightforward and maintains the current model of OVS as Ethernet-only OpenFlow pipeline. The outer MAC addresses can be modified with normal set_field actions between push and output. When sending the NSH packet to a VXLAN-GPE or GRE L3 tunnel, the outer MAC header would be stripped. Internally, in the datapath, one could optimize the temporary Ethernet header insertion away.
>
> The push/pop_pbb actions standardized in OF (though not implemented in OVS) follow the same model.
>
>
> 2) Modular push_nsh and push_eth actions
>
> This would give the controller the flexibility to push an NSH header alone, e.g. for transmission to a L2 tunnel port. The push_eth action would only be needed when sending to an Ethernet port.
>
> A stand-alone push_nsh action creates a non-Ethernet packet in the OF pipeline. OVS today is not prepared to handle non-Ethernet packets in the pipeline (The packet-type aware pipeline has been standardized in OF 1.5 but is not implemented yet).
>
> Even if we find a way to avoid dependency on the actual OF 1.5 control plane impacts, I think some infrastructure will be needed in OVS (datapaths and slow path) to deal with non-Ethernet packets correctly. The current work to support L3 tunneling would also have to be reconsidered in when we go that way.
>
> It is not clear what use cases there are for a stand-alone push_eth action, as a raw MAC in MAC encapsulation is not really standardized in IETF.
>
>
> Option 1) would be sufficient to cover all NSH use cases known today. It is straightforward and only depends on the L3 tunneling patches that are close to being finalized.
>
> Option 2) implies more work and additional dependencies of NSH support on generic infrastructure work. It might be more flexible but it is not clear if there are use cases that would benefit from the additional flexibility.
>
> Given all that, my suggestion is to go for option 1) now with the chance of adding more modular actions once OVS provides the infrastructure and OF control plane to deal with non-Ethernet packets.

I think one that we've seen as a result of existing tag implements
like VLAN and MPLS in OVS is that static layering like has been
defined in OpenFlow to date does not really work very well. There are
simply too many different ways that these tags have been used in
different situations and the problem becomes much worse as new tags
are introduce and potentially layered together. You can find some
examples if you search the mailing list archives for MPLS development
and some later questions from users - and that was with only two tags.
I think the only possible way to solve this is by independently
manipulating each layer and allowing them to be built up
incrementally.

Given that NSH is both trying to operate at multiple places in the
protocol stack and wants to operate on non-Ethernet packets,
infrastructure for packet types other than Ethernet seems pretty core
to what it is trying to accomplish.

>> One other comment, unrelated to the above discussion, is that NSH is continuing to make incompatible wire format changes.
>> Specifically, the option length field was recently changed from representing four byte multiples to be the actual length. This brings up
>> two questions:
>>  * Obviously the patches will need to be updated to handle this but also it means that the option format is no longer exactly the same
>> as Geneve options. This will require some thought on how to support both protocols while maximizing code reuse.
>>  * OVS strives to maintain consistent behavior for external interfaces and it's not clear how this would be possible if the protocol
>> continues to change in the future.
>
> My take on this is that we could declare the NSH support in OVS "experimental" at least until the NSH drafts have reached RFC stage. Until then everybody should be prepared for non-backward-compatible changes in OVS to follow the NSH drafts.

I think that's basically fine. I suppose the most important thing to
do is get a patch set that this ready to be applied and then we can
make a determination based on the current status of the draft. The
largest pieces of work that we are discussing here are not depending
on the specific bit layout anyways.

> Another approach could be to start with the stable MD1 header format first, but that has other drawbacks as discussed. I would strive to implement at least experimental support for MD2 TLV headers, too.

I don't think that this solves the problem. MD type 1 is no more
standardized than type 2 so while this change happened to be in type
2, there's no guarantee that the same will be true for future changes.