[ovs-dev] reside-on-redirect-chassis option

[Numan Siddique]

On Thu, Mar 12, 2020 at 3:06 AM Ben Pfaff <blp at ovn.org> wrote:
>
> On Wed, Mar 11, 2020 at 02:29:59PM -0700, Ben Pfaff wrote:
> > Hi Numan.  I am trying to properly understand the
> > reside-on-redirect-chassis option.  I think that it has two effects:
> >
> > 1. Traffic over the router port is always sent to the gateway chassis.
> >
> > 2. Traffic to the gateway chassis over the router port is always sent via
> >    localnet rather than a tunnel.
> >
> > Is that correct?  #2 seems to be stated by the documentation and the
> > commit messages, but I haven't been able to work out exactly why it has
> > that effect.
>
> And how does this interact with options:redirect-type?

Hi Ben,

The points (1) and (2) are  correct.

The reason this option was added is because of South/North traffic
from provider tenant networks
was tunnelled  to the gateway chassis (hosting the distributed gw router port).

Here's an example. Suppose we have the below logical setup

 *****************
Switch 5716af60-25d8-43ec-b70f-6b5fbefa6df9 (v4)
    port v4-ln
        type: localnet
        tag: 4
        addresses: ["unknown"]
    port v4-lr0
        type: router
        addresses: ["00:00:00:00:ff:01"]
        router-port: lr0-v4
    port v4-p1
        addresses: ["50:54:00:00:00:03 10.0.0.3"]
switch c9bbae86-f38a-49a2-85a0-6ed1d7c25077 (public)
    port public-lr0
        type: router
        router-port: lr0-public
    port ln-public
        type: localnet
        tag: 100
        addresses: ["unknown"]
switch 35cf00ce-0a40-4e79-9104-f1e8e0f59b66 (v5)
    port v5-p1
        addresses: ["40:54:00:00:00:03 20.0.0.3"]
    port v5-lr0
        type: router
        addresses: ["00:00:00:00:ff:02"]
        router-port: lr0-v5
    port v5-ln
        type: localnet
        tag: 5
        addresses: ["unknown"]
router 560b3171-ed27-4841-b034-3d7e43a9dc7a (lr0)
    port lr0-v4
        mac: "00:00:00:00:ff:01"
        networks: ["10.0.0.1/24"]
    port lr0-v5
        mac: "00:00:00:00:ff:02"
        networks: ["20.0.0.1/24"]
    port lr0-public
        mac: "00:00:20:20:12:13"
        networks: ["172.168.0.100/24"]
        gateway chassis: [ovn-gw]
    nat 25bd1545-c587-4e0f-a3e7-b142fa8689c7
        external ip: "172.168.0.100"
        logical ip: "20.0.0.0/24"
        type: "snat"
    nat a201f98b-389d-4055-ba66-806630382a6e
        external ip: "172.168.0.100"
        logical ip: "10.0.0.0/24"
        type: "snat"
***********************

You can see that all the logical switches have localnet ports. logical
switch 'v4' and 'v5' represents
tenant vlan provider networks (with vlan tags 4 nd 5 respectively) and
the logical switch 'public' provides
external connectivity.

Prior to the reside-on-redirect-chassis option, the traffic from vlan
tenant networks destined to outside
was tunnelled to the chassis hosting the distributed gateway router
port - lr0-public. And this was an issue.

'reside-on-redirect-chassis' is supposed to be set on the normal
logical router ports. In the above example
lr0-v4 and lr0-v5. When this is set, the routing is centralized on the
gateway chassis (where lr0-public resides).

Suppose if VIF of logical port v4-p1 sends arp request for 10.0.0.1,
the packet will go out via the localnet port
and since its broadcast traffic, it will reach the gateway chassis and
the ovn-controller their responds to the
ARP request. And the routing is handled on this chassis.

If the packet is destined outside, it is pushed out via the localnet
port of "public" logical switch on the
gateway chassis.

The downside is that the routing is centralized on the gw chassis - be
it E/W or N/S. Note that all the traffic will go
out via the localnet ports and no tunneling is involved. If we don't
centralize the E/W traffic, then the physical switch
will have issues with the router ip-mac learning as it will the router
ip-mac from many chassis.

Some time back, to overcome this centralized routing problem, Ankur
(from Nutanix) proposed another solution
which expects a separate mac is set for each chassis. With this
approach the E/W traffic is no longer centralized.
https://github.com/ovn-org/ovn/commit/1fed74cfc1a1e3b29cf86eab2e96048813019b57

ovn-architecture has more details on this approach.

Hope this helps. Let me know if you have any questions.

Regarding your question on how it interacts with redirect-type - If
the logical router doesn't have a gateway router port
(be it ha chassis, gateway chassis or redirect-type),
'reside-on-redirect-chassis' option set on other router ports is
ignored as
we can't centralize the routing.

Thanks
Numan


>
> Thanks,
>
> Ben.
>