[ovs-dev] OVN Load Balancing Feedback

[Guru Shetty]

On 13 June 2017 at 13:26, Ethan Jackson <ethan at quilt.io> wrote:

> Sure that makes sense.  I guess what's a little weird is that the load
> balancer lives on the logical switch instead of the logical router.
> Typically, when I think of a load balancer in the real world, it *is*
> a router that just happens to rewrite IP addresses.  It wouldn't be a
> router behind a switch that rewrites IPs, but not macs.
>

That is fair. As Kevin mentioned we do support load-balancers in routers,
but then it won't be distributed but rather in a gateway. The few reasons
why I went with the switch for distributed load-balancers were
1. Conntrack recirculations are expensive in OVS. We are already
recirculating for firewall, so I wanted to use the same recirculation and
same conntrack zone for load-balancing too.
2. The nature of OVN pipeline is that routers are not symmetric. i.e we
calculate everything the routers and switches are supposed to do in the
host where the VM/container resides (in either direction of traffic). So we
cannot share conntrack state for load-balancing on the same machine, if we
do it in the router.
3. Google does it, so there is a precedent.



> That said, not a huge deal ... it seems to work for us.  My only worry
> is that another team wouldn't be able to figure it out.
>

We have examples in tests/system-ovn.at. We should probably improve
documentation.


>
> Ethan
>
> On Tue, Jun 13, 2017 at 12:30 PM, Guru Shetty <guru at ovn.org> wrote:
> >
> >
> > On 12 June 2017 at 16:54, Kevin Lin <kevin at quilt.io> wrote:
> >>
> >> Hello, My name is Kevin Lin and I work with Ethan on Quilt (quilt.io).
> We
> >> just started using load balancing for the project -- Ethan wanted me to
> >> write to
> >> you all with feedback on the load balancer, and to get some feedback on
> >> our
> >> approach.
> >>
> >> For context, we have a number of containers connected to a single
> logical
> >> switch. We would like to create load balancers across groups of these
> >> containers. The load balancer should have it's own IP and MAC address,
> and
> >> be accessible from any of the containers connected to the switch.
> >>
> >> The point that confused me about the load balancer was that it only
> >> rewrites
> >> the IP addresses, and doesn't handle MAC addresses for you. As a
> >> result, it's very easy to set up a load balancer on a logical switch,
> that
> >> changes the IP to the appropriate desitination container, but can't
> >> respond
> >> ARPs or rewrite MAC addresses. This got us thinking, that what we really
> >> want
> >> is a load balancer attached to a logical router, but the documentation
> >> seems to
> >> indicate that this can't be set up without using a gateway.
> >>
> >> Anyways, we got it working, but we ended up with this rather convoluted
> >> design:
> >> - The load balancer is associated with the logical switch.
> >> - A logical router is connected to the logical switch. That just
> responds
> >> to
> >>   ARPs and forwards traffic sent to it back onto the logical switch
> (after
> >>   rewriting the MAC).
> >> - The IP of the load balancer is associated with this logical router's
> >> port.
> >
> >
> > This is how we use it for kubernetes too. The general thought process I
> went
> > with was that you won't have just a logical switch without some connected
> > gateway (either logical router or external default gateway) and we
> offload
> > the hairpin to the connected router. This is how kube-proxy works in
> google
> > cloud too. I did not want to add the additional complexity of ARP
> responses
> > from the load-balancer.
> >
> >>
> >>
> >> This way, when containers ARP for and send traffic to a load balanced
> IP,
> >> it
> >> gets routed to the logical router, but the load balancer rules rewrite
> the
> >> destination IP. The router then receives this packet, and routes it
> >> through to
> >> the rewritten IP. It works, but it's a bit of an unnatural hack.
> >>
> >> Hope this is helpful,
> >> Kevin
> >>
> >
>
>
>
> --
> Ethan J. Jackson
> quilt.io
>