[ovs-discuss] OVN Scale with RAFT: how to make raft cluster clients to balanced state again

Winson Wang windson.wang at gmail.com
Thu Aug 6 00:45:27 UTC 2020


Hi Han,


On Wed, Aug 5, 2020 at 3:05 PM Han Zhou <hzhou at ovn.org> wrote:

>
>
> On Wed, Aug 5, 2020 at 12:51 PM Winson Wang <windson.wang at gmail.com>
> wrote:
>
>> Hello OVN Experts:
>>
>> With large scale ovn-k8s cluster,  there are several conditions that
>> would make ovn-controller clients connect SB central from a balanced state
>> to an unbalanced state.
>> Is there an ongoing project to address this problem?
>> If not,  I have one proposal not sure if it is doable.
>> Please share your thoughts.
>>
>> The issue:
>>
>> OVN SB RAFT 3 node cluster,  at first all the ovn-controller clients will
>> connect all the 3 nodes in a balanced state.
>>
>> The following conditions will make the connections become unbalanced.
>>
>>    -
>>
>>    One RAFT node restart,  all the ovn-controller clients to reconnect
>>    to the two remaining cluster nodes.
>>
>>
>>    -
>>
>>    Ovn-k8s,  after SB raft pods rolling upgrade, the last raft pod has
>>    no client connections.
>>
>>
>> RAFT clients in an unbalanced state would trigger more stress to the raft
>> cluster,  which makes the raft unstable under stress compared to a balanced
>> state.
>> The proposal solution:
>>
>> Ovn-controller adds next unix commands “reconnect” with argument of
>> preferred SB node IP.
>>
>> When unbalanced state happens,  the UNIX command can trigger
>> ovn-controller reconnect
>>
>> To new SB raft node with fast sync which doesn’t trigger the whole DB
>> downloading process.
>>
>>
> Thanks Winson. The proposal sounds good to me. Will you implement it?
>

Thanks for reviewing my proposal solution.
I am hoping someone from the OVN team who more familiar with the
ovn-controller code to deliver the feature if possible:).

Regards,
Winson


> Han
>
>
>
>>
>> --
>> Winson
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "ovn-kubernetes" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to ovn-kubernetes+unsubscribe at googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/ovn-kubernetes/CAMu6iS--iOW0LxxtkOhJpRT49E-9bJVy0iXraC1LMDUWeu6kLA%40mail.gmail.com
>> <https://groups.google.com/d/msgid/ovn-kubernetes/CAMu6iS--iOW0LxxtkOhJpRT49E-9bJVy0iXraC1LMDUWeu6kLA%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>

-- 
Winson
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20200805/15ad26c0/attachment.html>


More information about the discuss mailing list