[ovs-dev] [PATCH ovn v2] controller: Monitor all logical flows that refer to datapath groups.

Ilya Maximets i.maximets at ovn.org
Wed Apr 21 10:43:23 UTC 2021


On 4/21/21 9:40 AM, Dumitru Ceara wrote:
> On 4/20/21 8:54 PM, Han Zhou wrote:
>> On Tue, Apr 20, 2021 at 12:43 AM Dumitru Ceara <dceara at redhat.com> wrote:
>>>
>>> On 4/19/21 7:24 PM, Han Zhou wrote:
>>>> On Mon, Apr 19, 2021 at 12:27 AM Dumitru Ceara <dceara at redhat.com>
>> wrote:
>>>>>
>>>>> On 4/16/21 7:16 PM, Mark Michelson wrote:
>>>>>> Thanks for the explanation Dumitru. It made good sense out of what had
>>>>>> to be a difficult thing to debug. I'm assuming you went with
>>>>>> unconditional monitoring since this will work with any change to a DP
>>>>>> group, and not just the addition new datapaths to the group.
>>>>>
>>>>> There's also a case I didn't mention in the commit log (but I can add
>> it
>>>>> if you think it's worth adding):
>>>>>
>>>>> If a logical flow LF1 is not shared between datapaths, so it's only
>>>>> applied to DP1, when DP2 is added that needs an identical flow,
>>>>> ovn-northd might generate a fresh new DPG = {.datapaths=[DP1, DP2]},
>> and
>>>>> create a new LF1' applied on datapath_group=DPG.
>>>>>
>>>>> Without unconditional monitoring for datapath groups there's no easy
>> way
>>>>> to get the notification about DPG and LF1' being added in the same
>>>>> jsonrpc update from SB with the delete for LF.
>>>>>
>>>>> Given that the number of datapath groups is expected to be relatively
>>>>> low compared to the total number of logical flows, unconditional
>>>>> monitoring seems like the best solution.
>>>>>
>>>>>>
>>>>>> Acked-by: Mark Michelson <mmichels at redhat.com>
>>>>>
>>>>> Thanks!
>>>>>
>>>>
>>>> +1 (although late)
>>>>
>>>> I am catching up the DPG implementation, and here are some thoughts on
>> the
>>>> use cases. For my understanding, except for the DPG that contains all
>> DPs,
>>>> most DPGs are mapping to port-groups, which usually map to
>>>> tenants/namespaces. The conditional monitoring feature (when
>>>> ovn-monitor-all is disabled) is useful mainly when there are a large
>> number
>>>> of small tenants and they don't need to talk to each other, so their
>>>> workloads may reside on different HVs and DPs, so conditional monitoring
>>>> can result in a smaller number of lflows on each HV in this scenario.
>>>> However, when DPG related flows are unconditionally monitored, it makes
>>>> conditional monitoring not as useful as it should be when there are a
>> lot
>>>> of ACLs, because ACLs are the major user of the port-group based DPGs,
>>>> which may contribute to a significant portion of lflows. In such
>> scenarios
>>>> (when there are many small tenants), maybe it is better to disable DPG
>> (and
>>>> disable ovn-monitor-all) to benefit more from the conditional monitoring
>>>
>>> I think DPGs may be useful in this case too (many small tenants) with
>>> conditional monitoring enabled.
>>>
>>> There's a reasonable number of "trivial" logical flows, e.g., the flows
>>> that advance to next table by default, that are shared between all
>>> logical datapaths and DPGs will optimize those.
>>>
>>> For example, on one such large downstream OpenStack deployment we saw
>>> IIRC a reduction of ~20% w.r.t. number of logical flows and SB size when
>>> enabling DPG.  It's not a huge factor but it's still a relevant
>>> reduction in number of objects in the SB that need to be sent to all
>>> ovn-controllers.
>>
>> It is true that DPG will still reduce SB size on the central nodes, but my
>> above point is about the lflows being conditionally monitored by each
>> individual HV, because usually central DB size is not a concern if
>> ovn-monitor-all is disabled. In the particular scenario of large amount of
>> small tenants, without DPG, a HV may be interested in only a small amount
>> of DPs which has a small amount of flows in total. With DPG enabled,
>> although sharing the common flows between the DPs would reduce some
>> redundant flows, but since the number of interested DPs is small for each
>> HV, this savings may be small. On the other hand, since all PG related ACL
>> flows for all tenants (all DPs) are now monitored, each HV could have a
>> huge increase of unnecessary ACL flows from unrelated tenants. That's why I
>> think in such a scenario DPG should be disabled.
> 
> I see, you're right, makes sense.  Another way to make this work in all
> scenarios is to add two-level deep conditional monitoring to the IDL and
> ovsdb-server, that is, allow ovn-controller to conditionally monitor
> "all logical flows that refer to DPG that contain datapaths equal to a
> specific value".

Another (probably, easier in implementation?) option is to avoid flow
updates if we have in-flight condition changes, i.e. if we do not have
a full view over logical flows.

> 
>>
>>>
>>>> feature. In the contrary use cases when there are few but large tenants,
>>>> each of them maps to a large port-group, crossing a large amount of
>>>> datapaths, then enabling DPG may be a better choice, and at the same
>> time
>>>> enabling ovn-monitor-all may be recommended.
>>>>
>>>> Is the above understanding correct?
>>>>
>>>
>>> I think so.
>>>
>>> Maybe it makes sense to try to add some guidelines about when to enable
>>> the datapath groups feature and how to combine it with conditional
>>> monitoring, maybe as part of the OVN documentation?  What do you think?
>>>
>> Yes, I agree we should document the guidelines.
>>
>>> Thanks,
>>> Dumitru
>>>
>>>
>> _______________________________________________
>> dev mailing list
>> dev at openvswitch.org
>> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>>
> 



More information about the dev mailing list