[ovs-dev] [PATCH 1/2] ovn: Avoid tunneling for VLAN packets redirected to a gateway chassis

Ankur Sharma ankur.sharma at nutanix.com
Fri Nov 2 00:50:56 UTC 2018


Hi Numan, Mark,

Thanks for the patch.
Description explains the problem statement and solution really well.

I have following comments:

a. Regarding the solution:
   i. I think referring to a non connected port in a logical datapath pipeline is probably not the right way.
      i.e as per the description, in ls0 pipeline we are referring to lr0-public.
      lr0-public is not a peer interface of ls0-lr0 patch port pair, hence ls0 pipeline should be totally agnostic to it (just my 2 cents).

      table=16(ls_in_l2_lkup), priority=50, match=(eth.dst == 00:00:00:00:af:12 && is_chassis_resident("cr-lr0-public")),  action=(outport = "sw0-lr0"; output;)

b. Nit: Patch 2 in this series does not look related to patch  1 (unless I am missing something),
            if current then we should have a separate patch for it.

c. Nit: Any new config that we are adding, we should add corresponding ovn-nbctl CLI.
           This way non openstack deployments can also play with the feature 😊

===================================================

Regarding the solution:
------------------------------

As I understand, we are trying to achieve following here:
a. For a distributed virtual router, we want some router interface to be fully centralized, rather than distributed.
    i.e instead of creating 2 versions of a router port (lrp-* and cr-lrp*), one distributed and one centralized, 
    We want the lrp-*  itself to be centralized.
    i.e in your case, from vlan backed logical switch, you want to enter router pipeline only on the gateway chassis and NOT on the source chassis.

b. Just wanted to propose an alternate approach here, which we implemented for a slightly different use case. 
     Looks like this alternate approach would help in your scenario as well.

Alternate Approach:
--------------------------
a. Convert the port pair between logical switch and logical router to be of type "l3gateway", rather than "patch".
b. i.e in your configuration, lr0-sw0 and sw0-lr0 should be implemented as type l3gateway rather than patch.
c. In other words, we are simulating a centralized router (for specific peer logical switches) in a distributed router.
We will be discussing this approach in OVS Conf as well:
https://ovsfall2018.sched.com/event/IO9w/connectivity-for-external-networks-on-the-overlay?iframe=no&w=100%&sidebar=yes&bg=no

===================================================

Please feel free to point, if I missed something here.
Please feel free to comment on the proposed alternative.

Thanks

Regards,
Ankur


-----Original Message-----
From: ovs-dev-bounces at openvswitch.org <ovs-dev-bounces at openvswitch.org> On Behalf Of Numan Siddique
Sent: Monday, October 15, 2018 2:38 AM
To: Mark Michelson <mmichels at redhat.com>
Cc: ovs dev <dev at openvswitch.org>
Subject: Re: [ovs-dev] [PATCH 1/2] ovn: Avoid tunneling for VLAN packets redirected to a gateway chassis

On Sat, Oct 13, 2018 at 1:26 AM Mark Michelson <mailto:mmichels at redhat.com> wrote:

> Hi Numan,
>
> The patch does a good job of explaining the routing behavior and the 
> tunneling problem solved within.
>
> Prior to the patch, you can have a distributed gateway router with a 
> redirect-chassis port set on it. This allows for east-west traffic to 
> have an optimal direct path between hypervisors, but for the 
> north-south use case, when the traffic is redirected to the 
> redirect-chassis, the traffic is encapsulated.
>
> With this patch, you add the reside-on-redirect-chassis option to 
> router ports. This essentially makes all traffic destined for the 
> router port get redirected to the gateway chassis prior to running the 
> router pipeline. This removes the encapsulation issue, but it also 
> means that east-west traffic is now also centralized.
>

That's right. That's the trade off to solve this issue for VLAN tenant networks.


>
> I'm curious what the current behavior is when you specify a gateway 
> router by setting options:chassis. Specifically, I'm curious about how 
> it compares if you define a router where the "external" port has 
> options:redirect-chassis set on it and all other ports have 
> options:reside-on-redirect-chassis set on them. Have you essentially 
> just created the same thing? Or is there some subtle difference?
>

There is a difference. In the case of gateway router (options:chassis set) scenario, it is expected that the tenant VLAN logical switches will be connected to a normal router and this normal router will be connected to the gateway router via a transit switch.
So the east west traffic will be distributed, but for the North/South traffic, the packet on the source chassis enters logical switch pipeline -> normal router pipeline -> transit switch pipeline. And then the packet is sent to the chassis hosting the gateway router via the tunnel port. On the gateway chassis, packet enters the transit switch pipeline ->  gateway router pipeline -> provider network pipeline.

Thanks
Numan






> On 10/05/2018 01:14 PM, mailto:nusiddiq at redhat.com wrote:
> > From: Numan Siddique <mailto:nusiddiq at redhat.com>
> >
> > An OVN deployment can have multiple logical switches each with a 
> > localnet port connected to a distributed logical router with one 
> > logical router port providing external connectivity (provider 
> > network) and others used as tenant networks with VLAN tagging.
> >
> > As reported in [1], external traffic from these VLAN tenant networks 
> > are tunnelled to the gateway chassis (chassis hosting a distributed 
> > gateway port which applies NAT rules). As part of the discussion in 
> > [1], there were few possible solutions proposed by Russell [2]. This 
> > patch implements the first option in [2].
> >
> > With this patch, a new option 'reside-on-redirect-chassis' in 'options'
> > column of Logical_Router_Port table is added. If the value of this 
> > option is set to 'true' and if the logical router also have a 
> > distributed gateway port, then routing for this logical router port 
> > is centralized in the chassis hosting the distributed gateway port.
> >
> > If a logical switch 'sw0' is connected to a router 'lr0' with the 
> > router port - 'lr0-sw0' with the address - "00:00:00:00:af:12
> 192.168.1.1"
> > , and it has a distributed logical port - 'lr0-public', then the 
> > below logical flow is added in the logical switch pipeline of 'sw0' 
> > if the 'reside-on-redirect-chassis' option is set on 'lr-sw0' -
> >
> > table=16(ls_in_l2_lkup), priority=50, match=(eth.dst == 
> > 00:00:00:00:af:12 &&
> is_chassis_resident("cr-lr0-public")),
> > action=(outport = "sw0-lr0"; output;)
> >
> > With the above flow, the packet doesn't enter the router pipeline in 
> > the source chassis. Instead the packet is sent out via the localnet 
> > port of 'sw0'. The gateway chassis upon receiving this packet, runs 
> > the logical router pipeline applying NAT rules and sends the traffic 
> > out via the localnet port of the provider network. The gateway 
> > chassis will also reply to the ARP requests for the router port IPs.
> >
> > With this approach, we avoid redirecting the external traffic to the 
> > gateway chassis via the tunnel port. There are a couple of drawbacks 
> > with this approach:
> >
> >    - East - West routing is no more distributed for the VLAN tenant
> >      networks if 'reside-on-redirect-chassis' option is defined
> >
> >    - 'dnat_and_snat' NAT rules with 'logical_mac' and 'logical_port'
> >      columns defined will not work for the VLAN tenant networks.
> >
> > This approach is taken for now as it is simple. If there is a 
> > requirement to support distributed routing for these VLAN tenant 
> > networks, we can explore other possible solutions.
> >
> > [1] -
> https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.
> org_pipermail_ovs-2Ddiscuss_2018-2DApril_046543.html&d=DwICAg&c=s883Gp
> UCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=mE2qy
> wxdjIadgBjj3-Xsjbt7jiXYD543pSHvwWZn5sg&s=TupRua9WlqBQG00wHQofyUYjEPymD
> aCMGNnZisJ-KyY&e=
> > [2] -
> https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.
> org_pipermail_ovs-2Ddiscuss_2018-2DApril_046557.html&d=DwICAg&c=s883Gp
> UCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=mE2qy
> wxdjIadgBjj3-Xsjbt7jiXYD543pSHvwWZn5sg&s=ufwXW9yvvqyU0Uc4YG3VaNekB5ieu
> 5EpBrRGcK_j0-k&e=
> >
> > Reported-at:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.
> org_pipermail_ovs-2Ddiscuss_2018-2DApril_046543.html&d=DwICAg&c=s883Gp
> UCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=mE2qy
> wxdjIadgBjj3-Xsjbt7jiXYD543pSHvwWZn5sg&s=TupRua9WlqBQG00wHQofyUYjEPymD
> aCMGNnZisJ-KyY&e=
> > Reported-by: venkata anil <mailto:vkommadi at redhat.com>
> > Co-authored-by: venkata anil <mailto:vkommadi at redhat.com>
> > Signed-off-by: Numan Siddique <mailto:nusiddiq at redhat.com>
> > Signed-off-by: venkata anil <mailto:vkommadi at redhat.com>
> > ---
> >   ovn/northd/ovn-northd.8.xml |  30 ++++
> >   ovn/northd/ovn-northd.c     |  71 +++++++---
> >   ovn/ovn-architecture.7.xml  | 160 +++++++++++++++++++++
> >   ovn/ovn-nb.xml              |  43 ++++++
> >   tests/ovn.at                | 273 ++++++++++++++++++++++++++++++++++++
> >   5 files changed, 561 insertions(+), 16 deletions(-)
> >
> > diff --git a/ovn/northd/ovn-northd.8.xml 
> > b/ovn/northd/ovn-northd.8.xml index 7352c6764..f52699bd3 100644
> > --- a/ovn/northd/ovn-northd.8.xml
> > +++ b/ovn/northd/ovn-northd.8.xml
> > @@ -874,6 +874,25 @@ output;
> >               resident.
> >             </li>
> >           </ul>
> > +
> > +        <p>
> > +          For the Ethernet address on a logical switch port of type
> > +          <code>router</code>, when that logical switch port's
> > +          <ref column="addresses" table="Logical_Switch_Port"
> > +          db="OVN_Northbound"/> column is set to <code>router</code> and
> > +          the connected logical router port specifies a
> > +          <code>reside-on-redirect-chassis</code> and the logical router
> > +          to which the connected logical router port belongs to has a
> > +          <code>redirect-chassis</code> distributed gateway logical
> router
> > +          port:
> > +        </p>
> > +
> > +        <ul>
> > +          <li>
> > +            The flow for the connected logical router port's Ethernet
> > +            address is only programmed on the
> <code>redirect-chassis</code>.
> > +          </li>
> > +        </ul>
> >         </li>
> >
> >         <li>
> > @@ -1179,6 +1198,17 @@ output;
> >             upstream MAC learning to point to the
> >             <code>redirect-chassis</code>.
> >           </p>
> > +
> > +        <p>
> > +          For the logical router port with the option
> > +          <code>reside-on-redirect-chassis</code> set (which is
> centralized),
> > +          the above flows are only programmed on the gateway port
> instance on
> > +          the <code>redirect-chassis</code> (if the logical router has a
> > +          distributed gateway port). This behavior avoids generation
> > +          of multiple ARP responses from different chassis, and allows
> > +          upstream MAC learning to point to the
> > +          <code>redirect-chassis</code>.
> > +        </p>
> >         </li>
> >
> >         <li>
> > diff --git a/ovn/northd/ovn-northd.c b/ovn/northd/ovn-northd.c index 
> > 31ea5f410..3998a898c 100644
> > --- a/ovn/northd/ovn-northd.c
> > +++ b/ovn/northd/ovn-northd.c
> > @@ -4426,13 +4426,32 @@ build_lswitch_flows(struct hmap *datapaths,
> struct hmap *ports,
> >                   ds_put_format(&match, "eth.dst == "ETH_ADDR_FMT,
> >                                 ETH_ADDR_ARGS(mac));
> >                   if (op->peer->od->l3dgw_port
> > -                    && op->peer == op->peer->od->l3dgw_port
> > -                    && op->peer->od->l3redirect_port) {
> > -                    /* The destination lookup flow for the router's
> > -                     * distributed gateway port MAC address should only
> be
> > -                     * programmed on the "redirect-chassis". */
> > -                    ds_put_format(&match, " && is_chassis_resident(%s)",
> > -
> op->peer->od->l3redirect_port->json_key);
> > +                    && op->peer->od->l3redirect_port
> > +                    && op->od->localnet_port) {
> > +                    bool add_chassis_resident_check = false;
> > +                    if (op->peer == op->peer->od->l3dgw_port) {
> > +                        /* The peer of this port represents a
> distributed
> > +                         * gateway port. The destination lookup 
> > + flow
> for the
> > +                         * router's distributed gateway port MAC
> address should
> > +                         * only be programmed on the
> "redirect-chassis". */
> > +                        add_chassis_resident_check = true;
> > +                    } else {
> > +                        /* Check if the option
> 'reside-on-redirect-chassis'
> > +                         * is set to true on the peer port. If set 
> > + to
> true
> > +                         * and if the logical switch has a localnet
> port, it
> > +                         * means the router pipeline for the 
> > + packets
> from
> > +                         * this logical switch should be run on the
> chassis
> > +                         * hosting the gateway port.
> > +                         */
> > +                        add_chassis_resident_check = smap_get_bool(
> > +                            &op->peer->nbrp->options,
> > +                            "reside-on-redirect-chassis", false);
> > +                    }
> > +
> > +                    if (add_chassis_resident_check) {
> > +                        ds_put_format(&match, " &&
> is_chassis_resident(%s)",
> > +
> op->peer->od->l3redirect_port->json_key);
> > +                    }
> >                   }
> >
> >                   ds_clear(&actions); @@ -5197,15 +5216,35 @@ 
> > build_lrouter_flows(struct hmap *datapaths,
> struct hmap *ports,
> >                             op->lrp_networks.ipv4_addrs[i].network_s,
> >                             op->lrp_networks.ipv4_addrs[i].plen,
> >                             op->lrp_networks.ipv4_addrs[i].addr_s);
> > -            if (op->od->l3dgw_port && op == op->od->l3dgw_port
> > -                && op->od->l3redirect_port) {
> > -                /* Traffic with eth.src = l3dgw_port->lrp_networks.ea_s
> > -                 * should only be sent from the "redirect-chassis", so
> that
> > -                 * upstream MAC learning points to the
> "redirect-chassis".
> > -                 * Also need to avoid generation of multiple ARP
> responses
> > -                 * from different chassis. */
> > -                ds_put_format(&match, " && is_chassis_resident(%s)",
> > -                              op->od->l3redirect_port->json_key);
> > +
> > +            if (op->od->l3dgw_port && op->od->l3redirect_port &&
> op->peer
> > +                && op->peer->od->localnet_port) {
> > +                bool add_chassis_resident_check = false;
> > +                if (op == op->od->l3dgw_port) {
> > +                    /* Traffic with eth.src =
> l3dgw_port->lrp_networks.ea_s
> > +                     * should only be sent from the 
> > + "redirect-chassis",
> so that
> > +                     * upstream MAC learning points to the
> "redirect-chassis".
> > +                     * Also need to avoid generation of multiple 
> > + ARP
> responses
> > +                     * from different chassis. */
> > +                    add_chassis_resident_check = true;
> > +                } else {
> > +                    /* Check if the option 'reside-on-redirect-chassis'
> > +                     * is set to true on the router port. If set to true
> > +                     * and if peer's logical switch has a localnet
> port, it
> > +                     * means the router pipeline for the packets from
> > +                     * peer's logical switch is be run on the chassis
> > +                     * hosting the gateway port and it should reply 
> > + to
> the
> > +                     * ARP requests for the router port IPs.
> > +                     */
> > +                    add_chassis_resident_check = smap_get_bool(
> > +                        &op->nbrp->options,
> > +                        "reside-on-redirect-chassis", false);
> > +                }
> > +
> > +                if (add_chassis_resident_check) {
> > +                    ds_put_format(&match, " && is_chassis_resident(%s)",
> > +                                  op->od->l3redirect_port->json_key);
> > +                }
> >               }
> >
> >               ds_clear(&actions);
> > diff --git a/ovn/ovn-architecture.7.xml b/ovn/ovn-architecture.7.xml 
> > index 6ed2cf132..998470c34 100644
> > --- a/ovn/ovn-architecture.7.xml
> > +++ b/ovn/ovn-architecture.7.xml
> > @@ -1372,6 +1372,166 @@
> >       https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.openvswitch.org_en_latest_topics_high-2Davailability&d=DwICAg&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=mE2qywxdjIadgBjj3-Xsjbt7jiXYD543pSHvwWZn5sg&s=3YEL0T7qW3h-GbKKAAcQ2q6kFtMqXliOiuOLrpKVQsg&e=.
> >     </p>
> >
> > +  <h2>Tenant VLAN networks connected to a Logical Router</h2>
> > +
> > +  <p>
> > +    It is possible to have multiple logical switches each with a
> localnet port
> > +    (representing physical networks) connected to a logical router 
> > + in
> which one
> > +    may provide the external connectivity via a distributed gatewat
> port and
> > +    the rest of them are used internally (with VLAN tagged). It is
> expected
> > +    that <code>ovn-bridge-mappings</code> is configured 
> > + appropriately
> on the
> > +    chassis.
> > +  </p>
> > +
> > +  <h3>East West routing</h3>
> > +  <p>
> > +    East-West routing between these tenant VLAN logical switches 
> > + works
> almost
> > +    the same way as normal logical switches. When the VM sends such 
> > + a
> packet,
> > +    then:
> > +  </p>
> > +  <ol>
> > +    <li>
> > +      The packet enters the ingress pipeline of the logical router
> datapath
> > +      via the logical router port in the source chassis.
> > +    </li>
> > +
> > +    <li>
> > +      Routing decision is taken.
> > +    </li>
> > +
> > +    <li>
> > +      The packet goes out of the integration bridge to the provider
> bridge (
> > +      belonging to the destination logical switch) via the localnet
> port.
> > +    </li>
> > +
> > +    <li>
> > +      The destination chassis receives the packet via the localnet port
> > +      and delivers to the destination VM.
> > +    </li>
> > +  </ol>
> > +
> > +  <h3>External traffic</h3>
> > +
> > +  <p>
> > +    The following happens when a VM sends an external traffic 
> > + (which
> requires
> > +    NATting) and the chassis hosting the VM doesn't have a 
> > + distributed
> gateway
> > +    port.
> > +  </p>
> > +
> > +  <ol>
> > +    <li>
> > +      The packet enters the ingress pipeline of the logical router
> datapath
> > +      via the logical router port in the source chassis.
> > +    </li>
> > +
> > +    <li>
> > +      Routing decision is taken. Since the gateway router or the
> distributed
> > +      gateway port doesn't reside in the source chassis, the traffic is
> > +      redirected to the gateway chassis via the tunnel port.
> > +    </li>
> > +
> > +    <li>
> > +      The gateway chassis receives the packet, applies the NAT rules and
> > +      forwards it via the localnet port.
> > +    </li>
> > +  </ol>
> > +
> > +  <p>
> > +    Although this works, the VM traffic is tunnelled. In order for it to
> > +    work properly, the MTU of the VLAN tenant networks must be 
> > + lowered
> to
> > +    account for the tunnel encapsulation.
> > +  </p>
> > +
> > +  <h2>Centralized routing for VLAN tenant networks</h2>
> > +
> > +  <p>
> > +    To overcome the tunnel encapsulation problem described in the
> previous
> > +    section, <code>OVN</code> supports the option of enabling
> centralized
> > +    routing for VLAN tenant networks. CMS can configure the option
> > +    <ref column="options:reside-on-redirect-chassis"
> > +    table="Logical_Router_Port" db="OVN_NB"/> to <code>true</code> 
> > + for
> each
> > +    <ref table="Logical_Router_Port" db="OVN_NB"/> which connects to the
> > +    logical switch of the VLAN tenant network. This causes the gateway
> > +    chassis (hosting the distributed gateway port) to handle all the
> > +    routing for these networks, making it centralized. It will reply to
> > +    the ARP requests for the logical router port IPs.
> > +  </p>
> > +
> > +  <p>
> > +    If the logical router doesn't have a distributed gateway port
> connecting
> > +    to the provider network, then this option is ignored by
> <code>OVN</code>.
> > +  </p>
> > +
> > +  <p>
> > +    The following happens when a VM sends an east-west traffic 
> > + which
> needs to
> > +    be routed:
> > +  </p>
> > +
> > +  <ol>
> > +    <li>
> > +      The packet from the VM enters the logical datapath pipeline 
> > + of
> the source
> > +      VLAN network in the source chassis and is sent out via the
> localnet port
> > +      (instead of sending it to router pipeline).
> > +    </li>
> > +
> > +    <li>
> > +      The packet enters the logical datapath pipeline of the source VLAN
> > +      network in the gateway chassis and is sent to the logical datapath
> > +      pipeline belonging to the logical router.
> > +    </li>
> > +
> > +    <li>
> > +      Routing decision is taken.
> > +    </li>
> > +
> > +    <li>
> > +      The packet enters the logical datapath pipeline of the destination
> > +      VLAN network. The packet is delivered to the destination VM 
> > + if it
> resides
> > +      in the same chassis. Otherwise the packet is sent out via the
> localnet
> > +      port of the destination VLAN network.
> > +    </li>
> > +
> > +    <li>
> > +      The destination chassis receives the packet via the localnet port
> > +      and delivers to the destination VM.
> > +    </li>
> > +  </ol>
> > +
> > +  <p>
> > +    The following happens when a VM sends an external traffic which
> requires
> > +    NATting:
> > +  </p>
> > +
> > +  <ol>
> > +    <li>
> > +      The packet from the VM enters the logical datapath pipeline 
> > + of
> the source
> > +      VLAN network in the source chassis and is sent out via the
> localnet port
> > +      (instead of sending it to router pipeline).
> > +    </li>
> > +
> > +    <li>
> > +      The packet enters the logical datapath pipeline of the source VLAN
> > +      network in the gateway chassis and is sent to the logical datapath
> > +      pipeline belonging to the logical router.
> > +    </li>
> > +
> > +    <li>
> > +      Routing decision is taken and NAT rules are applied.
> > +    </li>
> > +
> > +    <li>
> > +      The packet enters the logical datapath pipeline of the 
> > + provider
> network
> > +      and is sent out via the localnet port of the provider network.
> > +    </li>
> > +  </ol>
> > +
> > +  <p>
> > +    For the reverse external traffic, the gateway chassis applies 
> > + the
> unNATting
> > +    rules and sends the packet via the localnet port of the VLAN tenant
> > +    network and the destination chassis receives the packet and
> delivers to
> > +    the VM.
> > +  </p>
> > +
> >     <h2>Life Cycle of a VTEP gateway</h2>
> >
> >     <p>
> > diff --git a/ovn/ovn-nb.xml b/ovn/ovn-nb.xml index 
> > 8564ed39c..13ae56e13 100644
> > --- a/ovn/ovn-nb.xml
> > +++ b/ovn/ovn-nb.xml
> > @@ -1635,6 +1635,49 @@
> >             chassis to enable high availability.
> >           </p>
> >         </column>
> > +
> > +      <column name="options" key="reside-on-redirect-chassis">
> > +        <p>
> > +          Generally routing is distributed in <code>OVN</code>. The
> packet
> > +          from a logical port which needs to be routed hits the 
> > + router
> pipeline
> > +          in the source chassis. For the East-West traffic, the 
> > + packet
> is
> > +          sent directly to the destination chassis. For the outside
> traffic
> > +          the packet is sent to the gateway chassis.
> > +        </p>
> > +
> > +        <p>
> > +          When this option is set, <code>OVN</code> considers this 
> > + only
> if
> > +        </p>
> > +
> > +        <ul>
> > +          <li>
> > +            The logical router to which this logical router port
> belongs to
> > +            has a distributed gateway port.
> > +          </li>
> > +
> > +          <li>
> > +            The peer's logical switch has a localnet port (representing
> > +            a tenant VLAN network)
> > +          </li>
> > +        </ul>
> > +
> > +        <p>
> > +          When this option is set to <code>true</code>, then the packet
> > +          which needs to be routed hits the router pipeline in the
> chassis
> > +          hosting the distributed gateway router port. The source
> chassis
> > +          pushes out this traffic via the localnet port. With this the
> > +          East-West traffic is no more distributed and will always 
> > + go
> through
> > +          the gateway chassis.
> > +        </p>
> > +
> > +        <p>
> > +          Without this option set, for any traffic destined to 
> > + outside
> from a
> > +          logical port which belongs to a logical switch with 
> > + localnet
> port,
> > +          the source chassis will send the traffic to the gateway
> chassis via
> > +          the tunnel port instead of the localnet port and this 
> > + could
> cause MTU
> > +          issues.
> > +        </p>
> > +      </column>
> >       </group>
> >
> >       <group title="Attachment">
> > diff --git a/tests/ovn.at b/tests/ovn.at index 769e09f81..504ba228d 
> > 100644
> > --- a/tests/ovn.at
> > +++ b/tests/ovn.at
> > @@ -8537,6 +8537,279 @@ OVN_CLEANUP([hv1],[hv2],[hv3])
> >
> >   AT_CLEANUP
> >
> > +# VLAN traffic for external network redirected through distributed
> router
> > +# gateway port should use vlans(i.e input network vlan tag) across
> hypervisors
> > +# instead of tunneling.
> > +AT_SETUP([ovn -- vlan traffic for external network with distributed
> router gateway port])
> > +AT_SKIP_IF([test $HAVE_PYTHON = no]) ovn_start
> > +
> > +# Logical network:
> > +# # One LR R1 that has switches foo (192.168.1.0/24) and # # alice 
> > +(172.16.1.0/24) connected to it.  The logical port # # between R1 
> > +and alice has a "redirect-chassis" specified, # # i.e. it is the 
> > +distributed router gateway port(172.16.1.6).
> > +# # Switch alice also has a localnet port defined.
> > +# # An additional switch outside has the same subnet as alice # # 
> > +(172.16.1.0/24), a localnet port and nexthop port(172.16.1.1) # # 
> > +which will receive the packet destined for external network # # 
> > +(i.e 8.8.8.8 as destination ip).
> > +
> > +# Physical network:
> > +# # Three hypervisors hv[123].
> > +# # hv1 hosts vif foo1.
> > +# # hv2 is the "redirect-chassis" that hosts the distributed router
> gateway port.
> > +# # hv3 hosts nexthop port vif outside1.
> > +# # All other tests connect hypervisors to network n1 through 
> > +br-phys
> for tunneling.
> > +# # But in this test, hv1 won't connect to n1(and no br-phys in 
> > +hv1),
> and
> > +# # in order to show vlans(instead of tunneling) used between hv1 
> > +and
> hv2,
> > +# # a new network n2 created and hv1 and hv2 connected to this 
> > +network
> through br-ex.
> > +# # hv2 and hv3 are still connected to n1 network through br-phys.
> > +net_add n1
> > +
> > +# We are not calling ovn_attach for hv1, to avoid adding br-phys.
> > +# Tunneling won't work in hv1 as ovn-encap-ip is not added to any
> bridge in hv1
> > +sim_add hv1
> > +as hv1
> > +ovs-vsctl \
> > +    -- set Open_vSwitch . external-ids:system-id=hv1 \
> > +    -- set Open_vSwitch .
> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> > +    -- set Open_vSwitch . external-ids:ovn-encap-type=geneve,vxlan \
> > +    -- set Open_vSwitch . external-ids:ovn-encap-ip=192.168.0.1 \
> > +    -- add-br br-int \
> > +    -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true \
> > +    -- set Open_vSwitch . 
> > + external-ids:ovn-bridge-mappings=public:br-ex
> > +
> > +start_daemon ovn-controller
> > +ovs-vsctl -- add-port br-int hv1-vif1 -- \
> > +    set interface hv1-vif1 external-ids:iface-id=foo1 \
> > +    ofport-request=1
> > +
> > +sim_add hv2
> > +as hv2
> > +ovs-vsctl add-br br-phys
> > +ovn_attach n1 br-phys 192.168.0.2
> > +ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings="public:br-ex,phys:br-phys"
> > +
> > +sim_add hv3
> > +as hv3
> > +ovs-vsctl add-br br-phys
> > +ovn_attach n1 br-phys 192.168.0.3
> > +ovs-vsctl -- add-port br-int hv3-vif1 -- \
> > +    set interface hv3-vif1 external-ids:iface-id=outside1 \
> > +    options:tx_pcap=hv3/vif1-tx.pcap \
> > +    options:rxq_pcap=hv3/vif1-rx.pcap \
> > +    ofport-request=1
> > +ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings="phys:br-phys"
> > +
> > +# Create network n2 for vlan connectivity between hv1 and hv2 
> > +net_add n2
> > +
> > +as hv1
> > +ovs-vsctl add-br br-ex
> > +net_attach n2 br-ex
> > +
> > +as hv2
> > +ovs-vsctl add-br br-ex
> > +net_attach n2 br-ex
> > +
> > +OVN_POPULATE_ARP
> > +
> > +ovn-nbctl create Logical_Router name=R1
> > +
> > +ovn-nbctl ls-add foo
> > +ovn-nbctl ls-add alice
> > +ovn-nbctl ls-add outside
> > +
> > +# Connect foo to R1
> > +ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24 ovn-nbctl 
> > +lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> > +    type=router options:router-port=foo \
> > +    -- lsp-set-addresses rp-foo router
> > +
> > +# Connect alice to R1 as distributed router gateway port 
> > +(172.16.1.6)
> on hv2
> > +ovn-nbctl lrp-add R1 alice 00:00:02:01:02:03 172.16.1.6/24 \
> > +    -- set Logical_Router_Port alice options:redirect-chassis="hv2"
> > +ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
> > +    type=router options:router-port=alice \
> > +    -- lsp-set-addresses rp-alice router \
> > +
> > +# Create logical port foo1 in foo
> > +ovn-nbctl lsp-add foo foo1 \
> > +-- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> > +
> > +# Create logical port outside1 in outside, which is a nexthop 
> > +address # for 172.16.1.0/24 ovn-nbctl lsp-add outside outside1 \
> > +-- lsp-set-addresses outside1 "f0:00:00:01:02:04 172.16.1.1"
> > +
> > +# Set default gateway (nexthop) to 172.16.1.1 ovn-nbctl 
> > +lr-route-add R1 "0.0.0.0/0" 172.16.1.1 alice AT_CHECK([ovn-nbctl 
> > +lr-nat-add R1 snat 172.16.1.6 192.168.1.1/24]) ovn-nbctl set 
> > +Logical_Switch_Port rp-alice options:nat-addresses=router
> > +
> > +ovn-nbctl lsp-add foo ln-foo
> > +ovn-nbctl lsp-set-addresses ln-foo unknown ovn-nbctl 
> > +lsp-set-options ln-foo network_name=public ovn-nbctl lsp-set-type 
> > +ln-foo localnet AT_CHECK([ovn-nbctl set Logical_Switch_Port ln-foo 
> > +tag=2])
> > +
> > +# Create localnet port in alice
> > +ovn-nbctl lsp-add alice ln-alice
> > +ovn-nbctl lsp-set-addresses ln-alice unknown ovn-nbctl lsp-set-type 
> > +ln-alice localnet ovn-nbctl lsp-set-options ln-alice 
> > +network_name=phys
> > +
> > +# Create localnet port in outside
> > +ovn-nbctl lsp-add outside ln-outside ovn-nbctl lsp-set-addresses 
> > +ln-outside unknown ovn-nbctl lsp-set-type ln-outside localnet 
> > +ovn-nbctl lsp-set-options ln-outside network_name=phys
> > +
> > +# Allow some time for ovn-northd and ovn-controller to catch up.
> > +# XXX This should be more systematic.
> > +ovn-nbctl --wait=hv --timeout=3 sync
> > +
> > +# Check that there is a logical flow in logical switch foo's 
> > +pipeline # to set the outport to rp-foo (which is expected).
> > +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl dump-flows foo | grep 
> > +ls_in_l2_lkup
> | \
> > +grep rp-foo | grep -v is_chassis_resident | wc -l`])
> > +
> > +# Set the option 'reside-on-redirect-chassis' for foo ovn-nbctl set 
> > +logical_router_port foo
> options:reside-on-redirect-chassis=true
> > +# Check that there is a logical flow in logical switch foo's 
> > +pipeline # to set the outport to rp-foo with the condition is_chassis_redirect.
> > +ovn-sbctl dump-flows foo
> > +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl dump-flows foo | grep 
> > +ls_in_l2_lkup
> | \
> > +grep rp-foo | grep is_chassis_resident | wc -l`])
> > +
> > +echo "---------NB dump-----"
> > +ovn-nbctl show
> > +echo "---------------------"
> > +ovn-nbctl list logical_router
> > +echo "---------------------"
> > +ovn-nbctl list nat
> > +echo "---------------------"
> > +ovn-nbctl list logical_router_port
> > +echo "---------------------"
> > +
> > +echo "---------SB dump-----"
> > +ovn-sbctl list datapath_binding
> > +echo "---------------------"
> > +ovn-sbctl list port_binding
> > +echo "---------------------"
> > +ovn-sbctl dump-flows
> > +echo "---------------------"
> > +ovn-sbctl list chassis
> > +echo "---------------------"
> > +
> > +for chassis in hv1 hv2 hv3; do
> > +    as $chassis
> > +    echo "------ $chassis dump ----------"
> > +    ovs-vsctl show br-int
> > +    ovs-ofctl show br-int
> > +    ovs-ofctl dump-flows br-int
> > +    echo "--------------------------"
> > +done
> > +
> > +ip_to_hex() {
> > +    printf "%02x%02x%02x%02x" "$@"
> > +}
> > +
> > +foo1_ip=$(ip_to_hex 192 168 1 2)
> > +gw_ip=$(ip_to_hex 172 16 1 6)
> > +dst_ip=$(ip_to_hex 8 8 8 8)
> > +nexthop_ip=$(ip_to_hex 172 16 1 1)
> > +
> > +foo1_mac="f00000010203"
> > +foo_mac="000001010203"
> > +gw_mac="000002010203"
> > +nexthop_mac="f00000010204"
> > +
> > +# Send ip packet from foo1 to 8.8.8.8 src_mac="f00000010203"
> > +dst_mac="000001010203"
> >
> +packet=${foo_mac}${foo1_mac}08004500001c0000000040110000${foo1_ip}${d
> +st_ip}0035111100080000
> > +
> > +as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet sleep 2
> > +
> > +# ARP request packet for nexthop_ip to expect at outside1
> >
> +arp_request=ffffffffffff${gw_mac}08060001080006040001${gw_mac}${gw_ip
> +}000000000000${nexthop_ip}
> > +echo $arp_request >> hv3-vif1.expected cat hv3-vif1.expected > 
> > +expout $PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv3/vif1-tx.pcap 
> > +| grep
> ${nexthop_ip} | uniq > hv3-vif1
> > +AT_CHECK([sort hv3-vif1], [0], [expout])
> > +
> > +# Send ARP reply from outside1 back to the router 
> > +reply_mac="f00000010204"
> >
> +arp_reply=${gw_mac}${nexthop_mac}08060001080006040002${nexthop_mac}${
> +nexthop_ip}${gw_mac}${gw_ip}
> > +
> > +as hv3 ovs-appctl netdev-dummy/receive hv3-vif1 $arp_reply 
> > +OVS_WAIT_UNTIL([
> > +    test `as hv2 ovs-ofctl dump-flows br-int | grep table=66 | \ 
> > +grep actions=mod_dl_dst:f0:00:00:01:02:04 | wc -l` -eq 1
> > +    ])
> > +
> > +# VLAN tagged packet with router port(192.168.1.1) MAC as 
> > +destination
> MAC
> > +# is expected on bridge connecting hv1 and hv2
> >
> +expected=${foo_mac}${foo1_mac}8100000208004500001c0000000040110000${f
> +oo1_ip}${dst_ip}0035111100080000
> > +echo $expected > hv1-br-ex_n2.expected
> > +
> > +# Packet to Expect at outside1 i.e nexthop(172.16.1.1) port.
> > +# As connection tracking not enabled for this test, snat can't be 
> > +done
> on the packet.
> > +# We still see foo1 as the source ip address. But source 
> > +mac(gateway
> MAC) and
> > +# dest mac(nexthop mac) are properly configured.
> >
> +expected=${nexthop_mac}${gw_mac}08004500001c000000003f110100${foo1_ip
> +}${dst_ip}0035111100080000
> > +echo $expected > hv3-vif1.expected
> > +
> > +reset_pcap_file() {
> > +    local iface=$1
> > +    local pcap_file=$2
> > +    ovs-vsctl -- set Interface $iface options:tx_pcap=dummy-tx.pcap 
> > +\ options:rxq_pcap=dummy-rx.pcap
> > +    rm -f ${pcap_file}*.pcap
> > +    ovs-vsctl -- set Interface $iface
> options:tx_pcap=${pcap_file}-tx.pcap \
> > +options:rxq_pcap=${pcap_file}-rx.pcap
> > +}
> > +
> > +as hv1 reset_pcap_file br-ex_n2 hv1/br-ex_n2 as hv3 reset_pcap_file 
> > +hv3-vif1 hv3/vif1 sleep 2 as hv1 ovs-appctl netdev-dummy/receive 
> > +hv1-vif1 $packet sleep 2
> > +
> > +# On hv1, the packet should not go from vlan switch pipleline to 
> > +router # pipleine as hv1 ovs-ofctl dump-flows br-int
> > +
> > +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=65 | grep
> "priority=100,reg15=0x1,metadata=0x2" \
> > +| grep actions=clone | grep -v n_packets=0 | wc -l], [0], [[0
> > +]])
> > +
> > +# On hv1, table 32 check that no packet goes via the tunnel port 
> > +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=32 \
> > +| grep "NXM_NX_TUN_ID" | grep -v n_packets=0 | wc -l], [0], [[0
> > +]])
> > +
> > +ip_packet() {
> > +    grep "1010203f00000010203"
> > +}
> > +
> > +# Check vlan tagged packet on the bridge connecting hv1 and hv2 
> > +with the # foo1's mac.
> > +$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/br-ex_n2-tx.pcap |
> ip_packet | uniq > hv1-br-ex_n2
> > +cat hv1-br-ex_n2.expected > expout
> > +AT_CHECK([sort hv1-br-ex_n2], [0], [expout])
> > +
> > +# Check expected packet on nexthop interface $PYTHON 
> > +"$top_srcdir/utilities/ovs-pcap.in" hv3/vif1-tx.pcap | grep
> ${foo1_ip}${dst_ip} | uniq > hv3-vif1
> > +cat hv3-vif1.expected > expout
> > +AT_CHECK([sort hv3-vif1], [0], [expout])
> > +
> > +OVN_CLEANUP([hv1],[hv2],[hv3])
> > +AT_CLEANUP
> > +
> >   AT_SETUP([ovn -- IPv6 ND Router Solicitation responder])
> >   AT_KEYWORDS([ovn-nd_ra])
> >   AT_SKIP_IF([test $HAVE_PYTHON = no])
> >
>
>
_______________________________________________
dev mailing list
mailto:dev at openvswitch.org
https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwICAg&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=mE2qywxdjIadgBjj3-Xsjbt7jiXYD543pSHvwWZn5sg&s=vk8-2EI8-njSdNsgLyP81K8HEZOJfSxugzH3JpXsMUM&e=


More information about the dev mailing list