[ovs-dev] Sync on PTAP, EXT-382 and NSH

Mon Jan 9 11:15:22 UTC 2017

Jan, do you think when your proposal can be merged into ovs? The old NSH implementation and new proposal aren't contradictory, they can coexist, your new proposal isn't just for NSH, new proposal won't have push_nsh & pop_nsh, so we can let them coexist, let users decide which one is better.

We need to let users have one NSH version available before your proposal is implemented. I support your proposal, but I have no way to do anything helpful before your implementation for generic encap/decap and packet_type are available.

I don't know what other guys are thinking about this, it seems only you and I are caring this :)

From: Jan Scheurich [mailto:jan.scheurich at web.de]
Sent: Monday, January 9, 2017 8:23 AM
To: Yang, Yi Y <yi.y.yang at intel.com>; Jan Scheurich <jan.scheurich at ericsson.com>; Zoltán Balogh <zoltan.balogh at ericsson.com>; Jiri Benc (jbenc at redhat.com) <jbenc at redhat.com>; Pravin Shelar <pshelar at ovn.org>; Simon Horman (simon.horman at netronome.com) <simon.horman at netronome.com>; 'jpettit at ovn.org' <jpettit at ovn.org>; 'jarno at ovn.org' <jarno at ovn.org>; 'Ben Pfaff' <bpfaff at vmware.com>; 'ben.mackcrane at corsa.com' <ben.mackcrane at corsa.com>; dev at openvswitch.org; Zhou, Danny <danny.zhou at intel.com>
Subject: Re: [ovs-dev] Sync on PTAP, EXT-382 and NSH

Hi Yi,

I fully agree that support for NSH has been dragging along for to long. The prime reason for this (in addition to the dependency on the L3 tunneling) have been the mentioned conceptual problems with the current patches. Our initiative is the attempt to put NSH on a solid basis in OF/OVS and make faster progress by bundling all forces that agree on the solution documented in the Google doc.

We wouldn't pursue this if we saw a chance of upstreaming the NSH patches as they are now. We still hope that you can agree on the proposed approach and help making NSH happen soon.

Please find some more answers below.

BR, Jan

On 2017-01-03 01:59, Yang, Yi Y wrote:
Jan, we can't always be waiting endlessly, L3 patchset has been in Linux kernel no matter you like it or not, we're just to make sure ovs can work with it, VxLAN-gpe is not only for NSH.

If you think L3 patchset in Linux kernel has any issue, please send out your fix patches as soon as possible, it can be ported to ovs, this isn't an excuse we wait endlessly. I don't think we need to wait it.

About several NSH-related issues you mentioned, I don't think they are big issues,

Jan: "The NXM fields for NSH are used both as packet match fields and as packet metadata fields (after decap). This ambiguity leads to problems, latest when dealing with NSH in NSH packets."

Yi: VxLAN tunnel metadata used the same way, isn't it? What problems do you mean? I believe we can fix them even if they are really so-called "problems".
Tunnel metadata do not have that issue. They are only valid after the tunnel packet has been decapsulated or before it is encapsulated, but never when it is encapsulated. However, if you use OXM fields both for matching present packet headers and for keeping their content as packet metadata after decapsulation, both the meaning of matching on them and manipulating them is essentially undefined. This is most obvious for a packet with two NSH headers in a row (hierarchical SFC).

Jan: "They introduce push/pop_nsh OpenFlow actions without dealing with the resulting non-Ethernet packets in the pipeline. The behavior is not at all well defined."
Yi: L3 patchset is just for this, isn't it? Your new proposal will also depend on L3 patchset, right?
No, the L3 patch set merely allows to connect L3 ports to an Ethernet only (non-PTAP) OF pipeline. The packets in the OF pipeline are logically always L2 packets, even though the representation inside ofproto-dpif may temporarily omit the Ethernet header from the packet. But the L2 fields dl_src, dl_dst and dl_type can be matched on and manipulated by the OF controller at any time. When a packet is sent to an Ethernet port, the L2 header containing these fields is always present.

There is nothing in the L3 tunneling patch set that would allow an OF controller to modify the packet type in the pipeline. For this we need the concept of packet-type aware pipeline and the packet_type match field.

The new proposal based on PTAP does indeed depend on "versatile" tunnel ports, as e.g. already implemented in the kernel datapath. Zoltan and I are very close to having the user-space parts of the L3 tunneling patch sets adapted to the packet_type concept. These replace the user-space patches in your latest L3 tunneling patch set. Then we'd have work packages 1,2,3 and 6 of the document in place and we could focus in parallel on 4 (PTAP), 5 (EXT-382) and NSH MD1 (7 and 8).

Jan: "The re-use of the Geneve tunnel metada fields for NSH MD2 TLVs is problematic because
a) it again mixes packet metadata and header fields and
b) it couldn't handle NSH MD2 in Geneve tunnels."
Yi: You have to admit this is the existing best solution for MD type 2, it is not perfect, but it is ready for use. I don't think people will use GENEVE for NSH now, we can modify it to adapt to such use case if people really would like to do that way.
Using Tunnel TLV Metadata fields for NSH MD2 packet headers is conceptually wrong and not general as it would break if one carried NSH over Geneve. We should not upstream a broken solution that will have to be redone (in an incompatible way) sooner or later.

 Jan, I don't think the new proposal fixed the above issues you mentioned, on the contrary, it will make things more complicated. Why don't we go fats path instead take a from-a-scratch way?
I don't really see where things are getting more complicated. Large parts of the solution actually carry over easily. From the OF controller perspective, the NSH pipeline can be just as simple as before, as we have tried to demonstrate with the example flows in the document.

From: Jan Scheurich [mailto:jan.scheurich at web.de]
Sent: Friday, December 30, 2016 6:34 PM
To: Yang, Yi Y <yi.y.yang at intel.com><mailto:yi.y.yang at intel.com>; Jan Scheurich <jan.scheurich at ericsson.com><mailto:jan.scheurich at ericsson.com>; Zoltán Balogh <zoltan.balogh at ericsson.com><mailto:zoltan.balogh at ericsson.com>; Jiri Benc (jbenc at redhat.com<mailto:jbenc at redhat.com>) <jbenc at redhat.com><mailto:jbenc at redhat.com>; Pravin Shelar <pshelar at ovn.org><mailto:pshelar at ovn.org>; Simon Horman (simon.horman at netronome.com<mailto:simon.horman at netronome.com>) <simon.horman at netronome.com><mailto:simon.horman at netronome.com>; 'jpettit at ovn.org<mailto:jpettit at ovn.org>' <jpettit at ovn.org><mailto:jpettit at ovn.org>; 'jarno at ovn.org<mailto:jarno at ovn.org>' <jarno at ovn.org><mailto:jarno at ovn.org>; 'Ben Pfaff' <bpfaff at vmware.com><mailto:bpfaff at vmware.com>; 'ben.mackcrane at corsa.com<mailto:ben.mackcrane at corsa.com>' <ben.mackcrane at corsa.com><mailto:ben.mackcrane at corsa.com>; dev at openvswitch.org<mailto:dev at openvswitch.org>; Zhou, Danny <danny.zhou at intel.com><mailto:danny.zhou at intel.com>
Subject: Re: [ovs-dev] Sync on PTAP, EXT-382 and NSH

Hi Yi,

Thanks for the confirmation and for rebasing the existing L3 tunneling patches to include VXLAN-GPE.

Unfortunately, Simon's original user-space implementation in patches 9/17 through 11/17 using base_layer and offset fields in dp_packet is not compatible to our ongoing implementation of versatile tunnel ports in PTAP and non-PTAP bridges, which is based on an explicit packet_type field.

To avoid extensive rework, I would rather not merge these changes into master now but substitute them with the final implementation. This is work-package "3 - L3 ports in non-PTAP pipeline" in our Google doc and Zoltan and I will have that ready soon.

Regarding the implementation of NSH support, we should work together to implement what is described in the Google doc: https://docs.google.com/document/d/1oWMYUH8sjZJzWa72o2q9kU0N6pNE-rwZcLH3-kbbDR8/edit#heading=h.wp4o2op1lp9z

In our opinion the earlier NSH patches cannot be upstreamed because of a couple of fundamental conceptual problems:

  1.  The NXM fields for NSH are used both as packet match fields and as packet metadata fields (after decap).
This ambiguity leads to problems, latest when dealing with NSH in NSH packets.
  2.  They introduce push/pop_nsh OpenFlow actions without dealing with the resulting non-Ethernet packets in the pipeline. The behavior is not at all well defined.
  3.  The re-use of the Geneve tunnel metada fields for NSH MD2 TLVs is problematic because
a) it again mixes packet metadata and header fields and
b) it couldn't handle NSH MD2 in Geneve tunnels.
All these issues are addressed in the proposed new solution built on PTAP and EXT-382. The fundamentals are aligned with ONF (OXM classes already assigned), so that there is a good chance that we can feed the OVS implementation of NSH into the next OpenFlow standard. The first phase covering the fixed MD1 NSH header should also be possible to upstream in Q1/17, quite soon after the basic patches for PTAP and EXT-382.

Let's have a direct talk when I'm back in office after New Year.

Regards, Jan

On 2016-12-23 01:51, Yang, Yi Y wrote:

Hi, Jan

I confirm I can take VxLAN-gpe and NSH  related work, now I'm pushing Jiri's L3 patches ot ovs in order that it can be ported into ovs as early as possible, Pravin, Joe and Jarno found some vlan-related issues in Jiri's L3 patches in net-next and worked out several patches for net-next, but they are not merged yet.

But I have had  a workable local ovs version with Jiri's l3 patches and Jarno's fix patches merged, I have worked out several patches to make sure VxLAN-gpe can work in layer3 and layer 2 mode, now they are ready except DPDK userspace has some issues which I'm debugging.

So I think L3 patches and VxLAN-gpe will be ready very soon.

I remember every guys agreed our old NSH implementation with MD type 2 support, I think that will be the fastest path we can take for NSH support, I dare to guarantee it can be ready to merge about one month after (including kernel patches and ovs patches)

I'm wondering if you guys can make a form to list pros and cons for the old implementation way and this new one in order that every people can clearly know what the advantages and disadvantages for them are.

From: Jan Scheurich [mailto:jan.scheurich at ericsson.com]

Sent: Thursday, December 22, 2016 6:51 PM

To: Zoltán Balogh <zoltan.balogh at ericsson.com><mailto:zoltan.balogh at ericsson.com>; Yang, Yi Y <yi.y.yang at intel.com><mailto:yi.y.yang at intel.com>; Jiri Benc (jbenc at redhat.com<mailto:jbenc at redhat.com>) <jbenc at redhat.com><mailto:jbenc at redhat.com>; Pravin Shelar <pshelar at ovn.org><mailto:pshelar at ovn.org>; Simon Horman (simon.horman at netronome.com<mailto:simon.horman at netronome.com>) <simon.horman at netronome.com><mailto:simon.horman at netronome.com>; 'jpettit at ovn.org<mailto:jpettit at ovn.org>' <jpettit at ovn.org><mailto:jpettit at ovn.org>; 'jarno at ovn.org<mailto:jarno at ovn.org>' <jarno at ovn.org><mailto:jarno at ovn.org>; 'Ben Pfaff' <bpfaff at vmware.com><mailto:bpfaff at vmware.com>; 'ben.mackcrane at corsa.com<mailto:ben.mackcrane at corsa.com>' <ben.mackcrane at corsa.com><mailto:ben.mackcrane at corsa.com>; dev at openvswitch.org<mailto:dev at openvswitch.org>

Subject: RE: Sync on PTAP, EXT-382 and NSH

Thanks for the good meeting. Here are my notes:

Date: 2016-12-21, 17-18:30 CET

Participants: Jarno R, Ben P, Ben M-C, Jiri B, Simon H, Zoltan B, Jan S

Summary:

·        Discussed making PTAP, EXT-382 and NSH available as extensions to OF 1.3.

·        No big deal for the match fields and the encap/decap actions

·        Potential problem could be the missing packet_type information in OF 1.3 Packet In, Packet Out and Table Features (Note: Closer inspection of OF 1.5.1 spec reveals that OXM packet_type is part of struct ofp_match in Packet In and Packet Out. It should be OK for an OF 1.3 controller extended with PTAP support)

·        Is it simpler for the controllers to upgrade to OF 1.5?

·        Deferred the decision

·        Agreed to use the (to be assigned) OXM field code points for packet_type and NSH in OVS for all OF versions

·        Agreed to allow all NS=1 packet types received from/sent to a tunnel port that uses the Ethertype name space in its protocol field (like GRE). Other versatile tunnel ports (like VXLAN-GPE) which have their own code points require explicit mapping and must drop packets for which no such mapping exists.

·        Discussed introduction of a new OXM class for the proposed GEN_TLV fields

·        No problem to reserve an OXM class even before those fields are standardized

·        For standardization we also need to specify a dynamic binding mechanism between protocol TLVs and GEN_TLV fields. We can submit the mechanism to be developed in OVS for standardization when its stable.

·        Walk-through of division into work packages:

·        Some follow up needed for the L3 packet support kernel datapath patches

·        Rest OK

·        Technical discussion around GEN_TLV and use for NSH MD2 in conjunction with encap(NSH) to be continued in the Google doc.

·        Time line:

·        The entire work should be targeting OVS 2.8 with feature freeze around July

·        OVS 2.7 is having feature freeze already in early January

·        Work packages can be upstreamed individually. NSH MD1 support doesn't have to wait for MD2

·        Basically agreed to the work split proposed in the document:

·        RedHat is taking the patches for L3 tunnel configuration (including use of RTNETLINK for config)

·        Ericsson take the infrastructure components (L3-tunnels, PTAP, Basic EXT-382)

·        Jarno (VMware) will handle the GEN_TLV infrastructure

·        Confirm with Yi Yang (Intel): Can they take VXLAN-GPE and the NSH-specific WPs? Or do they need help?

·        Way of working

·        Continue the meeting series for coordination of effort

·        Can use a feature integration branch to ease the joint development and test

·        Review of patches mainly through the ovs-dev mailing list

·        Use tools like git citool to break up larger patches into a series of smaller patches for review

-----Original Appointment-----

From: Jan Scheurich

Sent: Sunday, 18 December, 2016 15:34

To: Jan Scheurich; Zoltán Balogh; Yang, Yi Y (yi.y.yang at intel.com<mailto:yi.y.yang at intel.com><mailto:yi.y.yang at intel.com><mailto:yi.y.yang at intel.com>); Jiri Benc (jbenc at redhat.com<mailto:jbenc at redhat.com><mailto:jbenc at redhat.com><mailto:jbenc at redhat.com>); Pravin Shelar; Simon Horman (simon.horman at netronome.com<mailto:simon.horman at netronome.com><mailto:simon.horman at netronome.com><mailto:simon.horman at netronome.com>); jpettit at ovn.org<mailto:jpettit at ovn.org><mailto:jpettit at ovn.org><mailto:jpettit at ovn.org>; jarno at ovn.org<mailto:jarno at ovn.org><mailto:jarno at ovn.org><mailto:jarno at ovn.org>; Ben Pfaff; 'ben.mackcrane at corsa.com<mailto:ben.mackcrane at corsa.com>'; dev at openvswitch.org<mailto:dev at openvswitch.org><mailto:dev at openvswitch.org><mailto:dev at openvswitch.org>

Subject: Sync on PTAP, EXT-382 and NSH

When: Wednesday, 21 December, 2016 17:00-18:30 (UTC+01:00) Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna.

Where: Skype Meeting

Moved to Wednesday same time to accommodate Jiri.

Hope this is still OK for the others.

Hello all,

I would like to call for a final sync meeting before the Christmas break.

Now that we have gone through the main aspects of the design, I would like to focus on how to divide the entire function into manageable pieces, discuss the potential work split, an integration anatomy and a rough time plane for upstreaming. I will try to prepare input in our Google doc for this.

https://docs.google.com/document/d/1oWMYUH8sjZJzWa72o2q9kU0N6pNE-rwZcLH3-kbbDR8/edit

If there are questions left regarding the design, please bring them up as well. You can also comment on the document at any time.

Regards, Jan

.........................................................................................................................................

--> Join Skype Meeting<https://meet.ericsson.com/jan.scheurich/HJ2NTF23><https://meet.ericsson.com/jan.scheurich/HJ2NTF23>

This is an online meeting for Skype for Business, the professional meetings and communications app formerly known as Lync.

Join by phone

+492115343925<tel:+492115343925,70849799%23><tel:+492115343925,70849799%23> (Germany)          English (United States)

89925<tel:+89925,70849799%23><tel:+89925,70849799%23> (Germany)          English (United States)

Find a local number<https://dialin.ericsson.com?id=70849799><https://dialin.ericsson.com?id=70849799>

Conference ID: 70849799

Forgot your dial-in PIN?<https://dialin.ericsson.com><https://dialin.ericsson.com> |Help<http://o15.officeredir.microsoft.com/r/rlidLync15?clid=1033&p1=5&p2=2009><http://o15.officeredir.microsoft.com/r/rlidLync15?clid=1033&p1=5&p2=2009>

To join a Lync / Skype for Business meeting from an Ericsson standard video room, add 77 before the Conference ID (e.g. 771234567 where 1234567 is the conference ID).    To join from a video room outside of Ericsson add one of the domains after 77 and Conference ID (e.g. 771234567@ xxxx.ericsson.net, where xxxx=emea/apac/amcs).  For assistance contact the IT Service Desk.

[!OC([1033])!]

.........................................................................................................................................

_______________________________________________

dev mailing list

dev at openvswitch.org<mailto:dev at openvswitch.org>

https://mail.openvswitch.org/mailman/listinfo/ovs-dev