[ovs-discuss] raft ovsdb clustering

[Numan Siddique]

Hi Aliasgar,

On Tue, Mar 13, 2018 at 7:11 AM, aginwala <aginwala at asu.edu> wrote:

> Hi Ben/Noman:
>
> I am trying to setup 3 node southbound db cluster  using raft10
> <https://patchwork.ozlabs.org/patch/854298/> in review.
>
> # Node 1 create-cluster
> ovsdb-tool create-cluster /etc/openvswitch/ovnsb_db.db
> /root/ovs-reviews/ovn/ovn-sb.ovsschema tcp:10.99.152.148:6642
>

A different port is used for RAFT. So you have to choose another port like
6644 for example.

>
>
> # Node 2
> ovsdb-tool join-cluster /etc/openvswitch/ovnsb_db.db OVN_Southbound tcp:
> 10.99.152.138:6642 tcp:10.99.152.148:6642 --cid 5dfcb678-bb1d-4377-b02d-
> a380edec2982
>
> #Node 3
> ovsdb-tool join-cluster /etc/openvswitch/ovnsb_db.db OVN_Southbound tcp:
> 10.99.152.101:6642 tcp:10.99.152.138:6642 tcp:10.99.152.148:6642 --cid
> 5dfcb678-bb1d-4377-b02d-a380edec2982
>
> # ovn remote is set to all 3 nodes
> external_ids:ovn-remote="tcp:10.99.152.148:6642, tcp:10.99.152.138:6642,
> tcp:10.99.152.101:6642"
>

> # Starting sb db on node 1 using below command on node 1:
>
> ovsdb-server --detach --monitor -vconsole:off -vraft -vjsonrpc
> --log-file=/var/log/openvswitch/ovsdb-server-sb.log --pidfile=/var/run/openvswitch/ovnsb_db.pid
> --remote=db:OVN_Southbound,SB_Global,connections --unixctl=ovnsb_db.ctl
> --private-key=db:OVN_Southbound,SSL,private_key --certificate=db:OVN_Southbound,SSL,certificate
> --ca-cert=db:OVN_Southbound,SSL,ca_cert --ssl-protocols=db:OVN_Southbound,SSL,ssl_protocols
> --ssl-ciphers=db:OVN_Southbound,SSL,ssl_ciphers --remote=punix:/var/run/openvswitch/ovnsb_db.sock
> /etc/openvswitch/ovnsb_db.db
>
> # check-cluster is returning nothing
> ovsdb-tool check-cluster /etc/openvswitch/ovnsb_db.db
>
> # ovsdb-server-sb.log below shows the leader is elected with only one
> server and there are rbac related debug logs with rpc replies and empty
> params with no errors
>
> 2018-03-13T01:12:02Z|00002|raft|DBG|server 63d1 added to configuration
> 2018-03-13T01:12:02Z|00003|raft|INFO|term 6: starting election
> 2018-03-13T01:12:02Z|00004|raft|INFO|term 6: elected leader by 1+ of 1
> servers
>
>
> Now Starting the ovsdb-server on the other clusters fails saying
> ovsdb-server: ovsdb error: /etc/openvswitch/ovnsb_db.db: cannot identify
> file type
>
>
> Also noticed that man ovsdb-tool is missing cluster details. Might want to
> address it in the same patch or different.
>
>
> Please advise to what is missing here for running ovn-sbctl show as this
> command hangs.
>
>
>

I think you can use the ovn-ctl command "start_cluster_sb_ovsdb" for your
testing (atleast for now)

For your setup, I think you can start the cluster as

# Node 1
ovn-ctl --db-sb-addr=10.99.152.148 --db-sb-port=6642
--db-sb-create-insecure-remote=yes --db-sb-cluster-local-addr="tcp:
10.99.152.148:6644" start_cluster_sb_ovsdb

# Node 2
ovn-ctl --db-sb-addr=10.99.152.138 --db-sb-port=6642
--db-sb-create-insecure-remote=yes
--db-sb-cluster-local-addr="tcp:10.99.152.138:6644"
--db-sb-cluster-remote-addr="tcp:10.99.152.148:6644" start_cluster_sb_ovsdb

# Node 3
ovn-ctl --db-sb-addr=10.99.152.101 --db-sb-port=6642
--db-sb-create-insecure-remote=yes
--db-sb-cluster-local-addr="tcp:10.99.152.101:6644"
--db-sb-cluster-remote-addr="tcp:10.99.152.148:6644" start_cluster_sb_ovsdb


Let me know how it goes.

Thanks
Numan



>
>
>
>
>
> _______________________________________________
> discuss mailing list
> discuss at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20180313/d9779113/attachment-0001.html>