[ovs-dev] [PATCH ovn v2] controller: Monitor all logical flows that refer to datapath groups.

Dumitru Ceara dceara at redhat.com
Wed Apr 21 07:40:01 UTC 2021


On 4/20/21 8:54 PM, Han Zhou wrote:
> On Tue, Apr 20, 2021 at 12:43 AM Dumitru Ceara <dceara at redhat.com> wrote:
>>
>> On 4/19/21 7:24 PM, Han Zhou wrote:
>>> On Mon, Apr 19, 2021 at 12:27 AM Dumitru Ceara <dceara at redhat.com>
> wrote:
>>>>
>>>> On 4/16/21 7:16 PM, Mark Michelson wrote:
>>>>> Thanks for the explanation Dumitru. It made good sense out of what had
>>>>> to be a difficult thing to debug. I'm assuming you went with
>>>>> unconditional monitoring since this will work with any change to a DP
>>>>> group, and not just the addition new datapaths to the group.
>>>>
>>>> There's also a case I didn't mention in the commit log (but I can add
> it
>>>> if you think it's worth adding):
>>>>
>>>> If a logical flow LF1 is not shared between datapaths, so it's only
>>>> applied to DP1, when DP2 is added that needs an identical flow,
>>>> ovn-northd might generate a fresh new DPG = {.datapaths=[DP1, DP2]},
> and
>>>> create a new LF1' applied on datapath_group=DPG.
>>>>
>>>> Without unconditional monitoring for datapath groups there's no easy
> way
>>>> to get the notification about DPG and LF1' being added in the same
>>>> jsonrpc update from SB with the delete for LF.
>>>>
>>>> Given that the number of datapath groups is expected to be relatively
>>>> low compared to the total number of logical flows, unconditional
>>>> monitoring seems like the best solution.
>>>>
>>>>>
>>>>> Acked-by: Mark Michelson <mmichels at redhat.com>
>>>>
>>>> Thanks!
>>>>
>>>
>>> +1 (although late)
>>>
>>> I am catching up the DPG implementation, and here are some thoughts on
> the
>>> use cases. For my understanding, except for the DPG that contains all
> DPs,
>>> most DPGs are mapping to port-groups, which usually map to
>>> tenants/namespaces. The conditional monitoring feature (when
>>> ovn-monitor-all is disabled) is useful mainly when there are a large
> number
>>> of small tenants and they don't need to talk to each other, so their
>>> workloads may reside on different HVs and DPs, so conditional monitoring
>>> can result in a smaller number of lflows on each HV in this scenario.
>>> However, when DPG related flows are unconditionally monitored, it makes
>>> conditional monitoring not as useful as it should be when there are a
> lot
>>> of ACLs, because ACLs are the major user of the port-group based DPGs,
>>> which may contribute to a significant portion of lflows. In such
> scenarios
>>> (when there are many small tenants), maybe it is better to disable DPG
> (and
>>> disable ovn-monitor-all) to benefit more from the conditional monitoring
>>
>> I think DPGs may be useful in this case too (many small tenants) with
>> conditional monitoring enabled.
>>
>> There's a reasonable number of "trivial" logical flows, e.g., the flows
>> that advance to next table by default, that are shared between all
>> logical datapaths and DPGs will optimize those.
>>
>> For example, on one such large downstream OpenStack deployment we saw
>> IIRC a reduction of ~20% w.r.t. number of logical flows and SB size when
>> enabling DPG.  It's not a huge factor but it's still a relevant
>> reduction in number of objects in the SB that need to be sent to all
>> ovn-controllers.
> 
> It is true that DPG will still reduce SB size on the central nodes, but my
> above point is about the lflows being conditionally monitored by each
> individual HV, because usually central DB size is not a concern if
> ovn-monitor-all is disabled. In the particular scenario of large amount of
> small tenants, without DPG, a HV may be interested in only a small amount
> of DPs which has a small amount of flows in total. With DPG enabled,
> although sharing the common flows between the DPs would reduce some
> redundant flows, but since the number of interested DPs is small for each
> HV, this savings may be small. On the other hand, since all PG related ACL
> flows for all tenants (all DPs) are now monitored, each HV could have a
> huge increase of unnecessary ACL flows from unrelated tenants. That's why I
> think in such a scenario DPG should be disabled.

I see, you're right, makes sense.  Another way to make this work in all
scenarios is to add two-level deep conditional monitoring to the IDL and
ovsdb-server, that is, allow ovn-controller to conditionally monitor
"all logical flows that refer to DPG that contain datapaths equal to a
specific value".

> 
>>
>>> feature. In the contrary use cases when there are few but large tenants,
>>> each of them maps to a large port-group, crossing a large amount of
>>> datapaths, then enabling DPG may be a better choice, and at the same
> time
>>> enabling ovn-monitor-all may be recommended.
>>>
>>> Is the above understanding correct?
>>>
>>
>> I think so.
>>
>> Maybe it makes sense to try to add some guidelines about when to enable
>> the datapath groups feature and how to combine it with conditional
>> monitoring, maybe as part of the OVN documentation?  What do you think?
>>
> Yes, I agree we should document the guidelines.
> 
>> Thanks,
>> Dumitru
>>
>>
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> 



More information about the dev mailing list