[ovs-discuss] HA OVN "Central" as a kubernetes service

Brendan Doyle brendan.doyle at oracle.com
Mon Jul 6 15:37:27 UTC 2020


So I've tried the steps in 
https://github.com/ovn-org/ovn-kubernetes/commit/a07b1a01af7e37b15c2e5f179ffad2b9f25a083d:

|cd $HOME/work/src/github.com/ovn-org/ovn-kubernetes/dist/images 
./daemonset.sh --image=docker.io/ovnkube/ovn-daemonset-u:latest \ 
--net-cidr=192.168.0.0/16 --svc-cidr=172.16.1.0/24 \ 
--gateway-mode="local" \ --k8s-apiserver=https://$MASTER_IP:6443|


|# Create OVN namespace, service accounts, ovnkube-db headless service, 
configmap, and policies kubectl create -f 
$HOME/work/src/github.com/ovn-org/ovn-kubernetes/dist/yaml/ovn-setup.yaml 
# Run ovnkube-db deployment. kubectl create -f 
$HOME/work/src/github.com/ovn-org/ovn-kubernetes/dist/yaml/ovnkube-db.yaml 
# Run ovnkube-master deployment. kubectl create -f 
$HOME/work/src/github.com/ovn-org/ovn-kubernetes/dist/yaml/ovnkube-master.yaml 
# Run ovnkube daemonset for nodes kubectl create -f 
$HOME/work/src/github.com/ovn-org/ovn-kubernetes/dist/yaml/ovnkube-node.yaml 
And I see nothing in my k8 cluster, seems like it does nothing 
[root at ca-rain01 yaml]# kubectl create -f ovnkube-master.yaml deployment 
"ovnkube-master" created [root at ca-rain01 yaml]# kubectl delete 
deployment ovnkube-master Error from server (NotFound): 
deployments.extensions "ovnkube-master" not found Has anybody got this 
working, or used any other means to deploy an OVN cluster as a K8 
Statefulset? Brendan |



On 06/07/2020 12:33, Brendan Doyle wrote:
> Hi,
>
> So I'm really confused by what you have pointed me to here. As stated 
> I do NOT
> want to use OVN as a CNI. I have a k8s cluster that use flannel as the 
> CNI. I simply
> want to create an OVN "central" cluster as a Stateful set in my 
> *existing* K8
> config.
>
> This repo:
> https://github.com/ovn-org/ovn-kubernetes/commit/a07b1a01af7e37b15c2e5f179ffad2b9f25a083d
>
> Seems to be for setting up a K8s cluster to use OVN as the CNI??
> Have you tried this?
> What IP do the ovn-controllers use to reach the OVN "central cluster?
> It seems to use an OVN docker image from docker.io, I want to use my 
> own OVN src
> Do I use/modify the dist/images/Dockerfile in this repo? that has 
> loads of references to CNI
> like I said I don't want to use OVN as the CNI??
>
>
> The instructions here 
> https://github.com/ovn-org/ovn/blob/d6b56b1629d5984ef91864510f918e232efb89de/Documentation/intro/install/general.rst
> seem more promising, if not a little confusing:
>
> IN the section "Starting OVN Central services in containers"
>>
>> Export following variables in .env and place it under project root:
>>
>> $ OVN_BRANCH=<BRANCH>
>> $ OVN_VERSION=<VERSION>
>> $ DISTRO=<LINUX_DISTRO>
>> $ KERNEL_VERSION=<LINUX_KERNEL_VERSION>
>> $ GITHUB_SRC=<GITHUB_URL>
>> $ DOCKER_REPO=<REPO_TO_PUSH_IMAGE>
>
> Does it mean create a file called ".env" and place it in the toplevel 
> dir of the cloned ovn repo?
> Or does it mean just add these to you shell environment (i.e put them 
> in .bashrc)?
>
> Then we have:
>
> 1)
>>
>> Start OVN containers using below command:
>>
>> $ docker run -itd --net=host --name=ovn-nb \
>>    <docker_repo>:<tag> ovn-nb-tcp
>>
>> $ docker run -itd --net=host --name=ovn-sb \
>>    <docker_repo>:<tag> ovn-sb-tcp
>>
>> $ docker run -itd --net=host --name=ovn-northd \
>>    <docker_repo>:<tag> ovn-northd-tcp
> followed by
>
> 2)
>> $ docker run -e "host_ip=<host_ip>" -e "nb_db_port=<port>" -itd \
>>    --name=ovn-nb-raft --net=host --privileged <docker_repo>:<tag> \
>>    ovn-nb-cluster-create
>>
>> $ docker run -e "host_ip=<host_ip>" -e "sb_db_port=<port>" -itd \
>>    --name=ovn-sb-raft --net=host --privileged <docker_repo>:<tag> \
>>    ovn-sb-cluster-create
>>
>> $ docker run -e "OVN_NB_DB=tcp:<node1>:6641,tcp:<node2>:6641,\
>>    tcp:<node3>:6641" -e "OVN_SB_DB=tcp:<node1>:6642,tcp:<node2>:6642,\
>>    tcp:<node3>:6642" -itd --name=ovn-northd-raft <docker_repo>:<tag> \
>>    ovn-northd-cluster
> Does it mean do 1), then 2) or does it mean do 1) for non HA OVN 
> central *OR* 2)
> for HA/clustered OVN Central?
>
> It's not clear
>
> Thanks
>
>
>
>
>
>
> On 25/06/2020 17:36, aginwala wrote:
>> Hi:
>>
>> There are a couple of options as I have been exploring this too:
>>
>> 1. Upstream ovn-k8s patches 
>> (https://github.com/ovn-org/ovn-kubernetes/commit/a07b1a01af7e37b15c2e5f179ffad2b9f25a083d) 
>> uses statefulset and headless service for starting ovn central raft 
>> cluster with 3 replicas. Cluster startup code and pod specs are 
>> pretty neat that addresses most of the doubts.
>>
>> OVN components have been containerized too to start them in pods. You 
>> can also refer to 
>> https://github.com/ovn-org/ovn/blob/d6b56b1629d5984ef91864510f918e232efb89de/Documentation/intro/install/general.rst 
>>  for the same and use them to make it work in pod specs too.
>>
>>
>> 2. Write a new ovn operator similar to etcd operator 
>> https://github.com/coreos/etcd-operator which just takes the count of 
>> raft replicas and does the job in the background.
>>
>> I also added ovn-k8s group so they can comment on any other ideas 
>> too. Hope it helps.
>>
>>
>>
>> On Thu, Jun 25, 2020 at 7:15 AM Brendan Doyle 
>> <brendan.doyle at oracle.com <mailto:brendan.doyle at oracle.com>> wrote:
>>
>>     Hi,
>>
>>     So I'm trying to find information on setting up an OVN
>>     "Central/Master"
>>     high availability (HA)
>>     Not as Active-Backup with Pacemaker, but as a cluster. But I want to
>>     deploy and manage that
>>     cluster as a Kubernetes service .
>>
>>     I can find lots of stuff on "ovn-kube" but this seems to be using
>>     OVN as
>>     a  kubernetes CNI instead of
>>     Flannel etc.  But this is not what I want to do, I have a kubernetes
>>     cluster using Flannel as the CNI,
>>     now  I want to deploy a HA OVN "Central" as a kubernetes service.
>>     Kind
>>     of like how you can deploy
>>     a MySQL cluster in kubernetes using a SatefulSet deployment.
>>
>>     I have found this:
>>     https://github.com/ovn-org/ovn-kubernetes#readme
>>
>>     But it is not clear to me if this is how to setup OVN as a
>>     kubernetes
>>     CNI or it's how to setup a HA
>>     OVN central as kubernetes service.
>>
>>     Can anybody comment, has anyone done this?
>>
>>
>>     I guess I could run an OVN central as standalone and use a
>>     kubernetes
>>     deployment with 3
>>       replica sets and "export" as a NodePort service. And have a
>>     floating/VIP on my kubernetes
>>     nodes. And direct ovn-controllers to the VIP. So only the pod
>>     that holds
>>     the VIP would service
>>     requests. This would work and give HA, but you don't get the
>>     performance
>>     of an OVN
>>     clustered Database Model, where each OVN central could service
>>     requests.
>>
>>
>>
>>
>>     Thanks
>>
>>
>>     Rdgs
>>     Brendan
>>
>>     _______________________________________________
>>     discuss mailing list
>>     discuss at openvswitch.org <mailto:discuss at openvswitch.org>
>>     https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>>
>
>
> _______________________________________________
> discuss mailing list
> discuss at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20200706/53f748de/attachment.html>


More information about the discuss mailing list