[ovs-dev] Hitless resynchronisation of forwarding state

[Jarno Rajahalme]

> On Jul 1, 2016, at 2:48 AM, Jan Scheurich <jan.scheurich at ericsson.com> wrote:
> 
>>> Refreshing the 250K flow entries using the bundle mechanism increases
>>> the vswitchd memory linearly up to 1.9 GB, significantly more than the 910
>>> MB one would expect for accommodating two versions of each rule at the
>>> moment of the atomic activation.
>> 
>> The reason for the high(er) memory use in the bundle case is that the bundle
>> message storage has not been optimized for size, and that the processing
>> itself has also not been optimized for memory consumption. Bundled
>> messages could use the same compressed match format that the rules in the
>> tables use, and the messages could be freed much earlier in the process.
>> These two strategies should get the memory use much closer to the
>> "accommodating the two versions" case. I started that at one point but did
>> not have a benchmark to work with so this work was not finished at the time.
> ...
>> Given that the only downside of the bundle mechanism seems to be the
>> memory cost, and on other dimensions the bundle mechanism is actually
>> better, would you be willing to reconsider your proposal if the bundle
>> memory consumption was significantly improved? 
> 
> Twice the memory of the steady state footprint would still be a significant burden, in particular if that memory would have to reside on hugepages that are statically reserved for ovs-vswitchd. I must say I haven't fully understood which part of the OVS data structures are actually allocated from hugepage memory when running OVS with DPDK netdev datapath. Can you clarify this?
> 

Not an expert on huge pages myself, but I asked around and was told DPDK use of huge pages does not affect the size of pages used for the rest of OVS userspace.


> How much effort do you think it would require to optimize the bundle mechanism the way you suggest? Is that something that could still fit into OVS 2.6?
> 
> We would also need to include the missing support for groups and meters in the bundle implementation. Groups would be crucial for us, meters we do not currently use in our pipelines yet. Any estimate on the complexity/cost of those?
> 

I have just posted a new patch series that implements both group support and also reduces the memory burden of bundle message storage and execution. I have not tested how much this helps, and would be happy if you could repeat your testing with the new patch series.

Either way, the patch series should make it's way to OVS 2.6.

Regards,

  Jarno