[ovs-dev] [PATCH v7 0/1] Support for non-contiguous numa nodes and core ids.

David Wilder dwilder at us.ibm.com
Tue Jun 22 18:53:07 UTC 2021


Ovs-numa currently makes the assumption that numa node ids and cpu core ids
will be numbered contiguously. Current Power systems don't always follow this
model. Furthermore, cpus on Power may be on/off lined based the setting of
Simultaneous multithreading (SMT). The result can be gaps in the numbering of
the cores. For example, a 2 socket system with 20 Core(s) per socket configured
with 4 thread per core (smt=4) has the following topology:

NUMA node0 CPU(s):   0-79
NUMA node8 CPU(s):   80-159

When set to smt=2 the following topology is created.

NUMA node0 CPU(s): 0,1,4,5,8,9,12,13,16,17,20,21,24,25,28,29,32,33,36,37,40,
                   41,44,45,48,49,52,53,56,57,60,61,64,65,68,69,72,73,76,77
NUMA node8 CPU(s): 80,81,84,85,88,89,92,93,96,97,100,101,104,105,108,109,112,
                   113,116,117,120,121,124,125,128,129,132,133,136,137,140,
                   141,144,145,148,149,152,153,156,157

This patch allows ovs-numa to work with non-contiguous node and cpu ids.
In addition lib/dpdk:construct_dpdk_socket_mem() is updated to correctly
build the EAL options: --socket-mem and --socket-limit on systems with
non-contiguous node ids. Pmd and dpif-netdev multi-node tests were updated
to validate a simulated numa topology with non-contiguous nodes.

v2 changes:
-0-day Robot suggested changes.
v3 changes:
-re-wrote cpu_detected() to address memory leak.
V4 changes:
-Rebased patches for:
 https://patchwork.ozlabs.org/project/openvswitch/list/?series=157389.
-Extended the automated tests to test both contiguous and non-contiguous
 configurations.
-changed the phrase "non-consecutive" to "non-contiguous" (since 0, 8 is
 consecutive but not contiguous).
V5 changes 
-Removed unused variable max_numa_id in discover_numa_and_core_dummy().
-Cleanup snprintf formatting in cpu_detected().
-Simplified test coverage of non-contiguous and contiguous nodes.
-Fixed bug in construct_dpdk_socket_mem().
V6 changes
-Sorted the list of numa nodes as hmap will not guarantee any order.
-Added a test to ensure that rxq assignment to pmds can be done when
  there are non-contiguous numa nodes (Provided by ktraynor at redhat.com).
-Reduced the number of non-contiguous numa nodes tests in dpif-netdev.at.
-Squashed the patch series to a single patch.
V7 changes
-Change numa_node_list and dependencies from size_t to int.
-Added dereference in sizeof in construct_dpdk_socket_mem().

David Wilder (1):
  ovs-numa: Support non-contiguous numa nodes and offline CPU cores

 lib/dpdk.c           | 57 +++++++++++++++++++++++++++++++++++------
 lib/ovs-numa.c       | 51 ++++++++++++++++++++++++------------
 lib/ovs-numa.h       |  2 ++
 tests/dpif-netdev.at |  2 +-
 tests/pmd.at         | 61 ++++++++++++++++++++++++++++++++++++++++----
 5 files changed, 142 insertions(+), 31 deletions(-)

-- 
2.27.0



More information about the dev mailing list