[ovs-discuss] ovn ovsdb clustering question

Thu May 27 01:21:06 UTC 2021

It's always useful to get feedback from people new to the project.  It
would be even better if you would send some an update to or a bug report
for the documentation, so that the next person to try it will have a
better experience.  A blog post is great but it's something external to
the project so it's harder to find.

On Wed, May 26, 2021 at 10:05:56AM -0400, Satish Patel wrote:
> Hi Numan,
> 
> Finally i have created small blog on OVSDB clustering so it help out
> new folks - https://satishdotpatel.github.io/openstack-ansible-ovn-clustering/
> 
> On Mon, May 24, 2021 at 5:58 PM Satish Patel <satish.txt at gmail.com> wrote:
> >
> > Hi Numan,
> >
> > Sorry for delay reply but I figured it out how to setup cluster basically I have to remove everything from /var/lib/ovn directory on each new nodes before I join them to cluster.
> >
> > Now I have other issue where I messed up my cluster using leave command and don’t know how to rejoin it. So basically I destroy cluster and rebuild it but in that process i lost my ovsdb database and now I don’t know how to restore previous data? I found some backup files inside /var/lib/ovn/ directories and I did restore using ovsdb-client utility but they didn’t help.
> >
> > I love ovn design but missing documents specially related cluster operation and whenever I google something all I found people asking question but not good answer. It would be great to have nice cluster operation and management doc. (I’m planning to write blog once I get everything working in lab)
> >
> > Sent from my iPhone
> >
> > > On May 21, 2021, at 4:35 PM, Numan Siddique <numans at ovn.org> wrote:
> > >
> > > On Fri, May 21, 2021 at 9:53 AM Satish Patel <satish.txt at gmail.com> wrote:
> > >>
> > >> Folks,
> > >>
> > >> I have 3 controller nodes and am trying to setup clustering for OVN
> > >> deployment, but lack enough documentation and have some issues and
> > >> questions regarding it.
> > >>
> > >> I found this document but again its little confusing and very old -
> > >> https://mail.openvswitch.org/pipermail/ovs-discuss/2018-March/046470.html
> > >>
> > >> # controller 1
> > >>
> > >> /usr/share/ovn/scripts/ovn-ctl --db-nb-addr=172.30.40.93 \
> > >> --db-nb-create-insecure-remote=yes \
> > >> --db-sb-addr=172.30.40.93 \
> > >> --db-sb-create-insecure-remote=yes \
> > >> --db-nb-cluster-local-addr=172.30.40.93 \
> > >> --db-sb-cluster-local-addr=172.30.40.93 \
> > >> --ovn-northd-nb-db=tcp:172.30.40.93:6641,tcp:172.30.40.25:6641,tcp:172.30.40.177:6641
> > >> \
> > >> --ovn-northd-sb-db=tcp:172.30.40.93:6642,tcp:172.30.40.25:6642,tcp:172.30.40.177:6642
> > >> \
> > >> start_northd
> > >>
> > >>
> > >> # controller 2
> > >>
> > >> /usr/share/ovn/scripts/ovn-ctl --db-nb-addr=172.30.40.25 \
> > >> --db-nb-create-insecure-remote=yes \
> > >> --db-sb-addr=172.30.40.25 \
> > >> --db-sb-create-insecure-remote=yes \
> > >> --db-nb-cluster-local-addr=172.30.40.25 \
> > >> --db-sb-cluster-local-addr=172.30.40.25 \
> > >> --db-nb-cluster-remote-addr=172.30.40.93 \
> > >> --db-sb-cluster-remote-addr=172.30.40.93 \
> > >> --ovn-northd-nb-db=tcp:172.30.40.93:6641,tcp:172.30.40.25:6641,tcp:172.30.40.177:6641
> > >> \
> > >> --ovn-northd-sb-db=tcp:172.30.40.93:6642,tcp:172.30.40.25:6642,tcp:172.30.40.177:6642
> > >> \
> > >> start_northd
> > >>
> > >>
> > >> # controller 3
> > >>
> > >> /usr/share/ovn/scripts/ovn-ctl --db-nb-addr=172.30.40.177 \
> > >> --db-nb-create-insecure-remote=yes \
> > >> --db-nb-cluster-local-addr=172.30.40.177 \
> > >> --db-sb-addr=172.30.40.177 \
> > >> --db-sb-create-insecure-remote=yes \
> > >> --db-sb-cluster-local-addr=172.30.40.177 \
> > >> --db-nb-cluster-remote-addr=172.30.40.93 \
> > >> --db-sb-cluster-remote-addr=172.30.40.93 \
> > >> --ovn-northd-nb-db=tcp:172.30.40.93:6641,tcp:172.30.40.25:6641,tcp:172.30.40.177:6641
> > >> \
> > >> --ovn-northd-sb-db=tcp:172.30.40.93:6642,tcp:172.30.40.25:6642,tcp:172.30.40.177:6642
> > >> \
> > >> start_northd
> > >>
> > >> ## Validation steps
> > >>
> > >> controller-2# export\
> > >> remote="tcp:172.30.40.93:6641,tcp:172.30.40.25:6641,tcp:172.30.40.177:6641"
> > >>
> > >> controller-2# ovn-nbctl --db=$remote show
> > >> controller-2#
> > >>
> > >> In the above command i am seeing output only when it hit controller-1
> > >> node, but for node-2 and note-3 giving me empty output that means data
> > >> replication doesn't work. what is the command to verify
> > >> synchronization working between all 3 nodes?
> > >>
> > >> Do I need to restart any other services?
> > >
> > > Hi Satish,
> > >
> > > Can you check out this script and try it -
> > > https://github.com/ovn-org/ovn-fake-multinode/blob/master/ovn_cluster.sh#L353
> > >
> > > Probably there is no need for you to start northd on all three nodes.
> > > If you want to start northd on 3 nodes,
> > > then they should be configured to connect to the clustered databases.
> > > Probably in your case, ovn-norths
> > > are connecting to the local unix sockets of ovsdb servers.
> > >
> > > Thanks
> > > Numan
> > >
> > >> _______________________________________________
> > >> discuss mailing list
> > >> discuss at openvswitch.org
> > >> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
> > >>
> _______________________________________________
> discuss mailing list
> discuss at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss