[ovs-discuss] Integration of ovn/ovs with kubernetes

[Guru Shetty]

On 15 November 2017 at 05:02, Sébastien Bernard <sbernard at nerim.net> wrote:

> On 15/11/2017 01:22, Guru Shetty wrote:
>
>
>
> On 14 November 2017 at 14:40, Sébastien Bernard <sbernard at nerim.net>
> wrote:
>
>> Hello,
>>
>> I'm looking for some pieces of advise to use a network based on
>> openvswitch with kubernetes.
>>
>> I've tried to follow the following document
>> https://github.com/openvswitch/ovn-kubernetes, with some success and
>> some failures.
>
>
>> First, it's not really clear what version of kubernetes is supported with
>> this software. I followed all the recipe, and at the end when starting the
>> ovs-k8s-watcher, I get error about the system:anonymous-user not having the
>> right to list services (tried with kubernetes 1.8).
>>
>
> I have seen it work till k8s 1.7. Haven't tried k8s 1.8 yet.
> This is most likely some permission issue. Haven't seen it before. Are you
> running it as a root? Can you use kubectl to list services? How about curl.
> For e.g:
> curl http://127.0.0.1:8080/api/v1/watch/endpoints
>
> I'm indeed running as root. The setup is ok up to the point of interacting
> with the kubernetes cluster. At this point it breaks with the error message.
> The install Doc may be amended for the new way of building kube cluster
> the kubernetes team is pushing (RBAC + kubeadm setup see below).
> I'll post a followup with the errors later.
>
>
>
>>
>> Second, I was puzzled by the install procedure, I don't really know where
>> the kubernetes configuration is modified. I was expecting some yaml to
>> apply with the kubectl, and nothing seems to change the kube configuration.
>> Where's the link between the pods and the ovs ?
>>
>
> When you do the "minion-init", it installs a OVN CNI plugin. The plugin
> gets invoked by kubelet when a pod gets scheduled. The plugin will setup
> the IP address and also add the pod's network interface to OVS.
>
>
>
>>
>> Third, is the 'ovn-k8s-overlay minion-init ' to be run on all minion and
>> the master also or only on the nodes ?
>>
> minion-init only on the nodes.
>
> The kubernetes setup is now done through the kubeadm.
> A master is an ordinary node with only pods of kube-system namespace
> scheduled. apiserver / controller-manager / scheduler are just pods
> scheduled statically.
>
> Let me ask this in a different way : should the ovn-k8s-overlay
> minion-init be run on each machine running a kubelet service ?
>

The current scripts assume that the kubernetes daemons run in host and not
inside pods. I will spend some time to see the changes in the script needed
to make it work with kubeadm too. It is unlikely to work as-is with kubeadm.

The OVN watcher needs access to kubernetes API server's IP address. All the
CNI plugins running in minions need access to the API server too. Those are
the only 2 OVN requirements.



>
>
>
>
>
>>
>> And last, what is the ovn-kube exectutable and how do you use it ?
>>
>
> This is a golang watcher which right now is only for advanced users, which
> calls things like "minion-init", "master-init" etc on its own, allocating
> subnets etc. We need to do a better job documenting it.
>
> I would suggest starting from the vagrant here. To get familiar with
> installation procedure. I often run it on my mac and it works.
> https://github.com/openvswitch/ovn-kubernetes/tree/master/vagrant
>
> You can then look at the installation scripts the vagrant uses.
>
> e.g:
> https://github.com/openvswitch/ovn-kubernetes/blob/master/vagrant/
> provisioning/setup-master.sh
> https://github.com/openvswitch/ovn-kubernetes/blob/master/vagrant/
> provisioning/setup-k8s-master.sh
>
>
> Thanks for the links. Vagrant setup is working ok. I'll try to reproduce
> it on a real setup (i.e. by hand).
>
> Seb
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20171115/c568dbb1/attachment.html>