[ovs-discuss] [OVN] How to set inactivity_probe between members in RAFT cluster

Sat Feb 8 09:24:39 UTC 2020

Hi,Aliasgar,
                Thanks for your reply. I am not clear actually. I have config 6641/6642 port to make client to connect by VIP from haproxy, the ports are configed in "OVN_NORTHD_OPTS",which you can search my mail.  And, "inactivity_probe"  problem comes from 6643/6644 port. If you can explain in more detail it would be great. Thanks very much.

regards,
Yun

在 2020-02-08 16:40:08，"aginwala" <aginwala at asu.edu> 写道：

On Fri, Feb 7, 2020 at 6:26 PM taoyunupt <taoyunupt at 126.com> wrote:

Hi,Aliasgar,
                   Maybe I need to tell you the way how I deployed the RAFT cluster, to make you understand my situation. I have 3 servers ,IPs are  192.5.0.8, 192.5.0.9, 192.5.0.10.
                   After  reading my steps,you may know why  my RAFT cluste do not have output of   "ovn-sbctl get-connection",but It also works most of time.           
                    If the way I used to deployed cluster is not good , please point it out. Thanks very much.
Hi Yun:

Your approach to start the cluster seems correct. The reason you don't see the connection entry after creating cluster is because it's needed for clients to allow them to connect to the cluster and not to form the cluster. Hence, you just need an additional step to create one nb and one sb connection entry for which you can set connection to ptcp:6641/42 so that clients like northd, ovn-controller, etc can connect to the cluster. Please also refer to cluster db section in https://github.com/openvswitch/ovs/blob/master/Documentation/ref/ovsdb.7.rst. for more details.

                  1.First step to create cluster by ovsdb-tool commands 

                        Create a cluster on the first node, and the IP address of this node is 192.5.0.8
                       # ovsdb-tool create-cluster /etc/openvswitch/ovnsb_db.db /usr/share/openvswitch/ovn-sb.ovsschema tcp:192.5.0.8:6644
                      # ovsdb-tool create-cluster /etc/openvswitch/ovnnb_db.db /usr/share/openvswitch/ovn-nb.ovsschema tcp:192.5.0.8:6643

                      Join the cluster on the second node, and the IP address of this node is 192.5.0.9
                      # ovsdb-tool join-cluster /etc/openvswitch/ovnsb_db.db OVN_Southbound tcp:192.5.0.9:6644 tcp:192.5.0.8:6644 tcp:192.5.0.10:6644
                     # ovsdb-tool join-cluster /etc/openvswitch/ovnnb_db.db OVN_Northbound tcp:192.5.0.9:6643 tcp:192.5.0.8:6643 tcp:192.5.0.10:6643

                  Join the cluster on the third node, and the IP address of this node is 192.5.0.10
                  #ovsdb-tool join-cluster /etc/openvswitch/ovnsb_db.db OVN_Southbound tcp:192.5.0.10:6644 tcp:192.5.0.8:6644 tcp:192.5.0.9:6644
                    #ovsdb-tool join-cluster /etc/openvswitch/ovnnb_db.db OVN_Northbound tcp:192.5.0.10:6643 tcp:192.5.0.8:6643 tcp:192.5.0.9:6643

                 2.Second step to conifg cluster

                   Edit the / etc / sysconfig / ovn-northd file of each node, add the OVN_NORTHD_OPTS option and content,
                   The IP of first node is 192.5.0.8, the added content is,Other nodes are similar:

                   OVN_NORTHD_OPTS="--db-nb-addr=192.5.0.8 --db-nb-create-insecure-remote=yes --db-sb-addr=192.5.0.8 \

--db-sb-create-insecure-remote=yes --db-nb-cluster-local-addr=192.5.0.8 --db-sb-cluster-local-addr=192.5.0.8 --ovn-northd-nb-db=tcp:192.5.0.8:6641,tcp:192.5.0.9:6641,tcp:192.5.0.10:6641 \

--ovn-northd-sb-db=tcp:192.5.0.8:6642,tcp:192.5.0.9:6642,tcp:192.5.0.10:6642"

                   3.Third step to start cluster

                     Execute the following command to start the cluster

                    #systemctl restart openvswitch ovn-northd

Regards,
Yun

在 2020-02-07 22:45:36，"taoyunupt" <taoyunupt at 126.com> 写道：

Hi,Aliasgar,

               Thanks for your reply.  I have tried your suggestion. But I found that  it just could create one NB connection or one SB connection.  In RAFT, we need at least two.
               That means  the output  of 'ovn-nbctl get-connection' has two lines. What do you think if I want to fix this problem?
               May be you don't need to consider how to have two connections for NB. Actually, I want to know how to solve the  "inactivity_probe"  problem.  

Regards,
Yun

At 2020-02-07 03:05:37, "aginwala" <aginwala at asu.edu> wrote:

Hi Yun:

For changing inactivity probe which is 5 sec default, you need to create connection entry both for sb and nb db.
ovn-nbctl -- --id=@conn_uuid create Connection \
target="<ptcp/ssl>\:<your-pord>\:<vip-ip>" \
inactivity_probe=<value> -- set NB_Global . connections=@conn_uuid

ovn-nbctl set connection . inactivity_probe=<value> will then work!

To tune the election timer for raft on say nb db, you can tune with below command:
ovs-appctl -t /var/run/openvswitch/ovnnb_db.ctl cluster/change-election-timer OVN_Northbound <value>
You can run similar settings for sb db for tuning the value

On Wed, Feb 5, 2020 at 4:00 AM taoyunupt <taoyunupt at 126.com> wrote:

Hi,Numan,
                I happend the problem that there are frequently elections in RAFT cluster members . I think it was cause by the not good connection between members of RARF cluster. As the log shows.
                Becase  the output of  "ovn-sbctl get-connection"  is none in RAFT cluster member,  So the command "ovn-sbctl set connection . inactivity_probe=180000"  not works.
                Do you know how to set "inactivity_probe"  when we use RAFT cluster?   It will be appreciateed  if you have more suggestions.

2020-02-05T01:37:29.178Z|03424|reconnect|ERR|tcp:10.254.8.210:52048: no response to inactivity probe after 5 seconds, disconnecting
2020-02-05T01:37:30.519Z|03425|raft|INFO|tcp:10.xxx.8.210:59300: learned server ID cdec
2020-02-05T01:37:30.519Z|03426|raft|INFO|tcp:10.xxx.8.210:59300: learned remote address tcp:10.254.8.210:6643
2020-02-05T03:52:02.791Z|03427|raft|INFO|received leadership transfer from 3e2e in term 64
2020-02-05T03:52:02.791Z|03428|raft|INFO|term 65: starting election
2020-02-05T03:52:02.792Z|03429|reconnect|INFO|tcp:10.xxx.8.208:6643: connection closed by peer
2020-02-05T03:52:02.869Z|03430|raft|INFO|term 65: elected leader by 2+ of 3 servers
2020-02-05T03:52:03.210Z|03431|raft|INFO|tcp:10.xxx.8.208:46140: learned server ID 3e2e
2020-02-05T03:52:03.210Z|03432|raft|INFO|tcp:10.xxx.8.208:46140: learned remote address tcp:10.xxx.8.208:6643
2020-02-05T03:52:03.793Z|03433|reconnect|INFO|tcp:10.254.8.208:6643: connecting...
2020-02-05T03:52:03.793Z|03434|reconnect|INFO|tcp:10.254.8.208:6643: connected

Thanks,
Yun
_______________________________________________
discuss mailing list
discuss at openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20200208/79cc32c1/attachment-0001.html>