[ovs-discuss] Multiple PMD threads for DPDK ports

[Daniele Di Proietto]

2016-05-18 0:24 GMT-07:00 Bhanu Chander Gaddoju <bhanu.gaddoju at nxp.com>:

> Hi Mark,
>
>
>
>         Thank you for your response.
>
>         We are seeing single PMD thread even after setting pmd-cpu-mask
> to 3 as below after starting the vswitchd daemon.
>
>
>
> *          ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=3*
>
>
>
>         Our system is a 8 core ARM system.  Below are the steps we are
> running and the corresponding debug logs.
>
>
> -----------------------------------------------------------------------------------------------------------------------
>
> root at ls2085ardb:/tmp# *mkdir -p /dev/hugepages*
>
> root at ls2085ardb:/tmp# *mount -t hugetlbfs -o pagesize=1G none
> /dev/hugepages*
>
> root at ls2085ardb:/tmp# *mkdir -p /usr/local/etc/openvswitch*
>
> root at ls2085ardb:/tmp# *mkdir -p /usr/local/var/run/openvswitch*
>
> root at ls2085ardb:/tmp# *rm /usr/local/etc/openvswitch/conf.db*
>
> rm: cannot remove '/usr/local/etc/openvswitch/conf.db': No such file or
> directory
>
> root at ls2085ardb:/tmp#*/tmp/ovsdb/ovsdb-tool create
> /usr/local/etc/openvswitch/conf.db /tmp/vswitchd/vswitch.ovsschema*
>
> root at ls2085ardb:/tmp# */tmp/ovsdb/ovsdb-server
> --remote=punix:/usr/local/var/run/openvswitch/db.sock
> --remote=db:Open_vSwitch,Open_vSwitch,manager_options --pidfile --detach
> --log-file=/var/log/openvswitch/ovs-vswitchd.log*
>
> 2019-04-02T13:24:30Z|00001|vlog|WARN|failed to open
> /var/log/openvswitch/ovs-vswitchd.log for logging: No such file or directory
>
> root at ls2085ardb:/tmp# */tmp/utilities/ovs-vsctl --no-wait init*
>
> root at ls2085ardb:/tmp# *export
> DB_SOCK=/usr/local/var/run/openvswitch/db.sock*
>
> root at ls2085ardb:/tmp# *vswitchd/ovs-vswitchd --dpdk -c 0x1 -n 4
> --socket-mem 1024  -- unix:$DB_SOCK --pidfile
> --log-file=/var/log/openvswitch/ovs-vswitchd2.log*
>
> 2019-04-02T13[  157.753639] Bits 55-60 of /proc/PID/pagemap entries are
> about to stop being page-shift some time soon. See the
> linux/Documentation/vm/pagemap.txt for details.
>
> :24:43Z|00001|dpdk|INFO|No -vhost_sock_dir provided - defaulting to
> /usr/local/var/run/openvswitch
>
> EAL: VFIO support initialized
>
> EAL: cannot open /proc/self/numa_maps, consider that all memory is in
> socket_id 0
>
>         Processing Container = dprc.2
>
>         container device path = /sys/bus/fsl-mc/devices/dprc.2
>
> EAL: DPAA2-Unused container at index 0
>
> -->Initial SHM Virtual ADDR FFFD80000000
>
> -----> DMA size 0x40000000
>
> -----> dma_map.vaddr = 0xFFFD80000000
>
>
>
> Zone 0: name:<RG_MP_log_history>, phys:0x81bfffd640, len:0x2080,
> virt:0xfffdbfffd640, socket_id:0, flags:0
>
> Zone 1: name:<MP_log_history>, phys:0x81bfedd2c0, len:0x1202c0,
> virt:0xfffdbfedd2c0, socket_id:0, flags:0
>
> Zone 2: name:<rte_eth_dev_data>, phys:0x81bfeada80, len:0x2f800,
> virt:0xfffdbfeada80, socket_id:0, flags:0
>
> 2019-04-02T13:24:47Z|00002|vlog|WARN|failed to open
> /var/log/openvswitch/ovs-vswitchd2.log for logging: No such file or
> directory
>
> changing path******************
>
> 2019-04-02T13:24:47Z|00003|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
> connecting...
>
> 2019-04-02T13:24:47Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
> connected
>
> Discovered 1 CPU cores on NUMA node 0
>
>  Discovered 1 NUMA nodes and 1 CPU cores
>

It looks like OVS cannot properly discover CPUs.

OVS uses the Linux kernel interface described here:

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/ABI/stable/sysfs-devices-node?id=refs/tags/v4.6

If the kernel has been compiled without CONFIG_NUMA, we rely on this
interface:

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/ABI/stable/sysfs-devices-system-cpu?id=refs/tags/v4.6

OVS parses these info in discover_numa_and_core() (lib/ovs-numa.c)

Can you please post the output of

ls -d /sys/devices/system/cpu/*

and

ls -d /sys/devices/system/node/*/*

Thanks,

Daniele
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20160526/7600a437/attachment-0002.html>