[ovs-discuss] [ovn][clustered] Confusing to create ovsdb-server clustered databases

Zufar Dhiyaulhaq zufar at onf-ambassador.org
Wed Aug 28 11:49:08 UTC 2019


Hi Numan,

Sorry for double reply (because not cc to ovs-discuss)

I trying to build a cluster with documentation provided in ovn-ctl manual
page. But the ovn-nbctl output nothing. The port is listening in 6641,
6642, 6643, and 6644.
Below the logs:

[root at zu-ovn-controller0 ~]# /usr/share/openvswitch/scripts/ovn-ctl
start_northd $OVN_NORTHD_OPTS --db-nb-addr=10.101.101.100
--db-nb-create-insecure-remote=yes --db-sb-addr=10.101.101.100
--db-sb-create-insecure-remote=yes
--db-nb-cluster-local-addr=10.101.101.100
--db-sb-cluster-local-addr=10.101.101.100 --ovn-northd-nb-db=tcp:
10.101.101.100:6641,tcp:10.101.101.101:6641,tcp:10.101.101.102:6641
--ovn-northd-sb-db=tcp:10.101.101.100:6642,tcp:10.101.101.101:6642,tcp:
10.101.101.102:6642
Creating cluster database /etc/openvswitch/ovnnb_db.db     [  OK  ]
Waiting for OVN_Northbound to come up
2019-08-28T11:28:50Z|00001|reconnect|INFO|unix:/var/run/openvswitch/ovnnb_db.sock:
connecting...
2019-08-28T11:28:50Z|00002|reconnect|INFO|unix:/var/run/openvswitch/ovnnb_db.sock:
connected
                                                           [  OK  ]
Creating cluster database /etc/openvswitch/ovnsb_db.db     [  OK  ]
Waiting for OVN_Southbound to come up
2019-08-28T11:28:51Z|00001|reconnect|INFO|unix:/var/run/openvswitch/ovnsb_db.sock:
connecting...
2019-08-28T11:28:51Z|00002|reconnect|INFO|unix:/var/run/openvswitch/ovnsb_db.sock:
connected
                                                           [  OK  ]
Starting ovn-northd                                        [  OK  ]
[root at zu-ovn-controller0 ~]# ovn-nbctl --db=tcp:10.101.101.100:6641,tcp:
10.101.101.101:6641,tcp:10.101.101.102:6641 show
[root at zu-ovn-controller0 ~]# ovn-sbctl --db=tcp:10.101.101.100:6642,tcp:
10.101.101.101:6642,tcp:10.101.101.102:6642 show
Chassis "a5b59592-f511-4a7a-b37d-93f933c35ea5"
    hostname: "zu-ovn-compute1"
    Encap geneve
        ip: "10.101.101.104"
        options: {csum="true"}
Chassis "cd1a2535-522a-4571-8eac-8394681846a3"
    hostname: "zu-ovn-compute2"
    Encap geneve
        ip: "10.101.101.105"
        options: {csum="true"}
Chassis "1ee48dd1-d520-476d-82d3-3d4651132f47"
    hostname: "zu-ovn-compute0"
    Encap geneve
        ip: "10.101.101.103"
        options: {csum="true"}

[root at zu-ovn-controller1 ~]# /usr/share/openvswitch/scripts/ovn-ctl
start_northd $OVN_NORTHD_OPTS --db-nb-addr=10.101.101.101
--db-nb-create-insecure-remote=yes --db-sb-addr=10.101.101.101
--db-sb-create-insecure-remote=yes
--db-nb-cluster-local-addr=10.101.101.101
--db-sb-cluster-local-addr=10.101.101.101
--db-nb-cluster-remote-addr=10.101.101.100
--db-sb-cluster-remote-addr=10.101.101.100 --ovn-northd-nb-db=tcp:
10.101.101.100:6641,tcp:10.101.101.101:6641,tcp:10.101.101.102:6641
--ovn-northd-sb-db=tcp:10.101.101.100:6642,tcp:10.101.101.101:6642,tcp:
10.101.101.102:6642
Waiting for OVN_Northbound to come up
2019-08-28T11:29:03Z|00001|reconnect|INFO|unix:/var/run/openvswitch/ovnnb_db.sock:
connecting...
2019-08-28T11:29:03Z|00002|reconnect|INFO|unix:/var/run/openvswitch/ovnnb_db.sock:
connected
                                                           [  OK  ]
Waiting for OVN_Southbound to come up
2019-08-28T11:29:03Z|00001|reconnect|INFO|unix:/var/run/openvswitch/ovnsb_db.sock:
connecting...
2019-08-28T11:29:03Z|00002|reconnect|INFO|unix:/var/run/openvswitch/ovnsb_db.sock:
connected
                                                           [  OK  ]
Starting ovn-northd                                        [  OK  ]

[root at zu-ovn-controller2 ~]# /usr/share/openvswitch/scripts/ovn-ctl
start_northd $OVN_NORTHD_OPTS --db-nb-addr=10.101.101.102
--db-nb-create-insecure-remote=yes
--db-nb-cluster-local-addr=10.101.101.102 --db-sb-addr=10.101.101.102
--db-sb-create-insecure-remote=yes
--db-sb-cluster-local-addr=10.101.101.102
--db-nb-cluster-remote-addr=10.101.101.100
--db-sb-cluster-remote-addr=10.101.101.100 --ovn-northd-nb-db=tcp:
10.101.101.100:6641,tcp:10.101.101.101:6641,tcp:10.101.101.102:6641
--ovn-northd-sb-db=tcp:10.101.101.100:6642,tcp:10.101.101.101:6642,tcp:
10.101.101.102:6642
Waiting for OVN_Northbound to come up
2019-08-28T11:29:13Z|00001|reconnect|INFO|unix:/var/run/openvswitch/ovnnb_db.sock:
connecting...
2019-08-28T11:29:13Z|00002|reconnect|INFO|unix:/var/run/openvswitch/ovnnb_db.sock:
connected
                                                           [  OK  ]
Waiting for OVN_Southbound to come up
2019-08-28T11:29:14Z|00001|reconnect|INFO|unix:/var/run/openvswitch/ovnsb_db.sock:
connecting...
2019-08-28T11:29:14Z|00002|reconnect|INFO|unix:/var/run/openvswitch/ovnsb_db.sock:
connected
                                                           [  OK  ]
Starting ovn-northd                                        [  OK  ]

Best Regards,
Zufar Dhiyaulhaq


On Wed, Aug 28, 2019 at 6:14 PM Zufar Dhiyaulhaq <zufar at onf-ambassador.org>
wrote:

> Hi Numan,
>
> I have tried the command but output nothing.
>
> [root at zu-ovn-controller0 ~]# ovn-nbctl --db=tcp:10.101.101.100:6641,tcp:
> 10.101.101.101:6641,tcp:10.101.101.102:6641 show
> [root at zu-ovn-controller0 ~]# ovn-sbctl --db=tcp:10.101.101.100:6642,tcp:
> 10.101.101.101:6642,tcp:10.101.101.102:6642 show
> Chassis "1ee48dd1-d520-476d-82d3-3d4651132f47"
>     hostname: "zu-ovn-compute0"
>     Encap geneve
>         ip: "10.101.101.103"
>         options: {csum="true"}
> Chassis "cd1a2535-522a-4571-8eac-8394681846a3"
>     hostname: "zu-ovn-compute2"
>     Encap geneve
>         ip: "10.101.101.105"
>         options: {csum="true"}
> Chassis "a5b59592-f511-4a7a-b37d-93f933c35ea5"
>     hostname: "zu-ovn-compute1"
>     Encap geneve
>         ip: "10.101.101.104"
>         options: {csum="true"}
> [root at zu-ovn-controller0 ~]# tail -f
> /var/log/openvswitch/ovsdb-server-nb.log
> 2019-08-28T09:12:31.190Z|00031|reconnect|INFO|tcp:10.101.101.102:6643:
> connection attempt failed (No route to host)
> 2019-08-28T09:12:31.190Z|00032|reconnect|INFO|tcp:10.101.101.102:6643:
> waiting 2 seconds before reconnect
> 2019-08-28T09:12:33.191Z|00033|reconnect|INFO|tcp:10.101.101.102:6643:
> connecting...
> 2019-08-28T09:12:33.192Z|00034|reconnect|INFO|tcp:10.101.101.102:6643:
> connection attempt failed (No route to host)
> 2019-08-28T09:12:33.192Z|00035|reconnect|INFO|tcp:10.101.101.102:6643:
> waiting 4 seconds before reconnect
> 2019-08-28T09:12:37.192Z|00036|reconnect|INFO|tcp:10.101.101.102:6643:
> connecting...
> 2019-08-28T09:12:37.192Z|00037|reconnect|INFO|tcp:10.101.101.102:6643:
> connection attempt failed (No route to host)
> 2019-08-28T09:12:37.192Z|00038|reconnect|INFO|tcp:10.101.101.102:6643:
> continuing to reconnect in the background but suppressing further logging
> 2019-08-28T09:22:52.597Z|00039|reconnect|INFO|tcp:10.101.101.101:6643:
> connected
>
> 2019-08-28T09:23:01.279Z|00040|reconnect|INFO|tcp:10.101.101.102:6643:
> connected
>
> I have tried with ovn-ctl to create the clustered-databases, but the
> problem is same, stuck when creating neutron resources. I think it because
> the ovn-northd run in 3 nodes, but neutron only run on a single controller.
>
> this is the step: http://paste.openstack.org/show/766470/
> should I try the step first? but check with passing the remote URL to
> command?
>
> Best Regards,
> Zufar Dhiyaulhaq
>
>
> On Wed, Aug 28, 2019 at 6:06 PM Numan Siddique <nusiddiq at redhat.com>
> wrote:
>
>>
>>
>> On Wed, Aug 28, 2019 at 4:04 PM Zufar Dhiyaulhaq <
>> zufar at onf-ambassador.org> wrote:
>>
>>> [ovn][clustered] Confusing to create ovsdb-server clustered databases
>>>
>>> Hi Everyone, I have successfully created OpenStack with OVN enabled. But
>>> the problem comes when I try to cluster the ovsdb-server. My scenario is
>>> trying to cluster the ovsdb-server databases but only using single
>>> ovn-northd.
>>>
>>> My cluster:
>>> - controller0 : 10.100.100.100 / 10.101.101.100 (ovn-northd,
>>> ovsdb-server, neutron server)
>>> - controller1 : 10.100.100.101 / 10.101.101.101 (ovsdb-server)
>>> - controller2 : 10.100.100.102 / 10.101.101.102 (ovsdb-server)
>>> - compute1 : 10.100.100.103 / 10.101.101.103
>>> - compute2 : 10.100.100.104 / 10.101.101.104
>>> - compute3 : 10.100.100.105 / 10.101.101.105
>>>
>>> 10.100.100.0/24 : Management Network
>>> 10.101.101.0/24 : Data Network
>>>
>>> I have installed the OpenStack using the manual method. Below is the
>>> step to create ovsdb-server and neutron services.
>>> - step 1: bootstrapping ovsdb-server cluster :
>>> http://paste.openstack.org/show/766463/
>>> - step 2: creating neutron service in controller :
>>> http://paste.openstack.org/show/766464/
>>> - step 3: creating neutron service in compute :
>>> http://paste.openstack.org/show/766465/
>>>
>>> But when I try to create neutron resource, Its always hang (only neutron
>>> resource).
>>>
>>> This is the full logs of all nodes, contain:
>>> http://paste.openstack.org/show/766461/
>>> - all openvswitch logs
>>> - port (via netstat)
>>> - step bootstraping ovsdb-server
>>>
>>> Neutron logs in controller0: paste.openstack.org/show/766462/
>>>
>>> Anyone know why stuck? Are my step is wrong?
>>>
>>>
>> Hi Zufar,
>>
>> Have you tried connecting to the clustered dbs using ovn-nbctl ?
>> Does it work fine when you pass the remotes i.e
>> $ovn-nbctl --db=tcp:10.101.101.100:6641,tcp:10.101.101.101:6641,tcp:
>> 10.101.101.102:6641 show
>>
>> If so, then it seems to me you have configured the clustered db properly.
>>
>> Also please take a look at - "man ovn-ctl" and grep for "Creating a
>> clustered db on 3 nodes".
>>
>> May be the python IDL code or the ovsdbapp which networking-ovn uses
>> doesn't support connecting
>> with multiple remotes ?
>>
>> I remember adding the support in the python IDL to accept multiple
>> remotes [1], but not sure
>> what's the status now.
>>
>> @Daniel - Do you have any comments/pointers ?
>>
>> [1] -
>> https://github.com/openvswitch/ovs/commit/31e434fc985c682708f5d92bde2ceae452bdaa4f
>>
>> https://github.com/openvswitch/ovs/commit/257edb1ae07c150023575dfb38ea8e539ad713de
>>
>> Thanks
>> Numan
>>
>> Thank you.
>>>
>>> Best Regards,
>>> Zufar Dhiyaulhaq
>>> _______________________________________________
>>> discuss mailing list
>>> discuss at openvswitch.org
>>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20190828/b37d8ce9/attachment-0001.html>


More information about the discuss mailing list