[ovs-discuss] FW: Mempool issue for OVS 2.9

[Stokes, Ian]

Hi All,

Recently an issue was raised regarding the move from a single shared mempool model that was in place up to OVS 2.8, to a mempool per port model introduced in 2.9.

https://mail.openvswitch.org/pipermail/ovs-discuss/2018-January/046021.html

The per port mempool model was introduced in September 5th with commit d555d9bded to allow fine grain control on a per port case of memory usage.

In the 'common/shared mempool' model, ports sharing a similar MTU would all share the same buffer mempool (e.g. the most common example of this being that all ports are by default created with a 1500B MTU, and as such share the same mbuf mempool).

This approach had some drawbacks however. For example, with the shared memory pool model a user could exhaust the shared memory pool (for instance by requesting a large number of RXQs for a port), this would cause the vSwitch to crash as any remaining ports would not have the required memory to function. This bug was discovered and reported to the community in late 2016 https://mail.openvswitch.org/pipermail/ovs-discuss/2016-September/042560.html.

The per port mempool patch aimed to avoid such issues by allocating a separate buffer mempool to each port.

An issue has been flagged on ovs-discuss, whereby memory dimensions provided for a given number of ports on OvS 2.6-2.8 may be insufficient to support the same number of ports in OvS 2.9, on account of the per-port mempool model without re-dimensioning extra memory. The effect of this is use case dependent (number of ports, RXQs, MTU settings, number of PMDs etc.) The previous 'common-pool' model was rudimentary in estimating the mempool size and was marked as something that was to be improved upon. The memory allocation calculation for per port model was modified to take the possible configuration factors mentioned into account.

It's unfortunate that this came to light so close to the release code freeze - but better late than never as it is a valid problem to be resolved.

I wanted to highlight some options to the community as I don't think the next steps should be taken in isolation due to the impact this feature has.

There are a number of possibilities for the 2.9 release.

(i) Revert the mempool per port patches and return to the shared mempool model. There are a number of features and refactoring in place on top of the change so this will not be a simple revert. I'm investigating what exactly is involved with this currently.
(ii) Leave the per port mempool implementation as is, flag to users that memory requirements have increased. Extra memory may have to be provided on a per use case basis.
(iii) Reduce the amount of memory allocated per mempool per port. An RFC to this effect was submitted by Kevin but on follow up the feeling is that it does not resolve the issue adequately.
(iv) Introduce a feature to allow users to configure mempool as shared or on a per port basis: This would be the best of both worlds but given the proximity to the 2.9 freeze I don't think it's feasible by the end of January.

I'd appreciate peoples input on this ASAP so we can reach a consensus on the next steps.

Thanks
Ian