[ovs-dev] [PATCH dpdk-latest v4] build: Add support for DPDK meson build.

[Tonghao Zhang]

On Wed, Dec 16, 2020 at 9:57 PM Richardson, Bruce
<bruce.richardson at intel.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Tonghao Zhang <xiangxia.m.yue at gmail.com>
> > Sent: Wednesday, December 16, 2020 1:49 PM
> > To: Richardson, Bruce <bruce.richardson at intel.com>
> > Cc: Ilya Maximets <i.maximets at ovn.org>; Pai G, Sunil
> > <sunil.pai.g at intel.com>; ovs dev <dev at openvswitch.org>; David Marchand
> > <david.marchand at redhat.com>; Stokes, Ian <ian.stokes at intel.com>
> > Subject: Re: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for DPDK
> > meson build.
> >
> > On Wed, Dec 16, 2020 at 9:41 PM Richardson, Bruce
> > <bruce.richardson at intel.com> wrote:
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Tonghao Zhang <xiangxia.m.yue at gmail.com>
> > > > Sent: Wednesday, December 16, 2020 1:58 AM
> > > > To: Richardson, Bruce <bruce.richardson at intel.com>
> > > > Cc: Ilya Maximets <i.maximets at ovn.org>; Pai G, Sunil
> > > > <sunil.pai.g at intel.com>; ovs dev <dev at openvswitch.org>; David Marchand
> > > > <david.marchand at redhat.com>; Stokes, Ian <ian.stokes at intel.com>
> > > > Subject: Re: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for
> > DPDK
> > > > meson build.
> > > >
> > > > On Tue, Dec 15, 2020 at 9:38 PM Richardson, Bruce
> > > > <bruce.richardson at intel.com> wrote:
> > > > >
> > > > > <snip>
> > > > >
> > > > > > > > Hi Ilya
> > > > > > > > I tested the patch
> > > > > > > >
> > https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825-
> > > > > > 30877-1-
> > > > > > > > git-send-email-ian.stokes at intel.com/
> > > > > > > > As I said if we update the pkg-conf to new version. and we
> > config
> > > > the
> > > > > > > > ovs --with-dpdk=static, ovs works fine.
> > > > > > > > but if we build ovs --with-dpdk=shared, ovs doesn't work,
> > there is
> > > > > > error
> > > > > > > > log:
> > > > > > > > 2020-12-14T20:15:07.940Z|00018|dpdk|INFO|EAL ARGS: ovs-
> > vswitchd -a
> > > > > > > >
> > > > > >
> > > >
> > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represen
> > > > > > > > tor=[0-6]
> > > > > > > > -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 --
> > socket-
> > > > limit
> > > > > > > > 1024,1024.
> > > > > > > > 2020-12-14T20:15:07.946Z|00019|dpdk|INFO|EAL: Detected 56
> > lcore(s)
> > > > > > > > 2020-12-14T20:15:07.946Z|00020|dpdk|INFO|EAL: Detected 2 NUMA
> > > > nodes
> > > > > > > > 2020-12-14T20:15:07.946Z|00021|dpdk|ERR|EAL: failed to parse
> > > > device
> > > > > > > > "0000:82:00.0"
> > > > > > > > 2020-12-14T20:15:07.946Z|00022|dpdk|ERR|EAL: Unable to parse
> > > > device
> > > > > > > >
> > > > > >
> > > >
> > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represe
> > > > > > > > ntor=[0-6]'
> > > > > > > > 2020-12-14T20:15:07.946Z|00023|dpdk|EMER|Unable to initialize
> > > > DPDK: No
> > > > > > > > such device
> > > > > > > >
> > > > > > > > because pci bus was not registered.
> > > > > > > >
> > > > > > > > In dpdk:
> > > > > > > > eal_plugins_init
> > > > > > > > rte_pci_scan
> > > > > > > > eal_option_device_parse -- no pci bus
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > Is DPDK installed systemwide so that the drivers - including pci
> > bus
> > > > > > > driver - can be picked up automatically on init? If not, you may
> > > > well
> > > > > > > need to pass the path to the drivers via the "-d" flag on
> > startup.
> > > > > > Hi
> > > > > > I build ovs, shown as below:
> > > > > >
> > > > > > export DPDK_TARGET=x86_64-native-linuxapp-gcc
> > > > > > meson -Dprefix=/root/local/dpdk-next-net -Dmax_ethports=256
> > > > > > -Dbuildtype=debug
> > > > > > -Ddisable_drivers=event/*,net/tap,net/nfb,net/mlx4,crypto/*
> > > > > > $DPDK_TARGET
> > > > > > ninja -C $DPDK_TARGET install
> > > > > > ldconfig
> > > > > >
> > > > > > export LD_LIBRARY_PATH=/root/local/dpdk-next-
> > > > net/lib64:$LD_LIBRARY_PATH
> > > > > > export PKG_CONFIG_PATH=/root/local/dpdk-next-
> > > > > > net/lib64/pkgconfig/:$PKG_CONFIG_PATH
> > > > > > ./configure --enable-Werror --prefix=/root/local/openvswitch-2.14
> > > > > > --disable-ssl --disable-libcapng --with-dpdk=shared 'CFLAGS=-g -O0
> > > > > > -Wno-deprecated-declarations'
> > > > > > make && make install
> > > > > >
> > > > > > One question if we export  LD_LIBRARY_PATH, we also use "-d" flag
> > ?
> > > > > >
> > > > > > > Regards,
> > > > > > > /Bruce
> > > > >
> > > > > In the case above, I think the drivers should all be loaded
> > > > automatically
> > > > > from "/root/local/dpdk-next-net/lib64/dpdk/pmds-21.0/". The
> > > > RTE_EAL_PMD_PATH
> > > > > should point to that location.
> > > > >
> > > > > To verify, can you check the "rte_build_config.h" file in your DPDK
> > > > build,
> > > > > and confirm that the path given there for RTE_EAL_PMD_PATH
> > corresponds
> > > > to
> > > > > the path where the actual driver .so files are installed?
> > > > In the rte_build_config.h:
> > > > #define RTE_EAL_PMD_PATH "/root/local/dpdk-next-net/lib64/dpdk/pmds-
> > 21.1"
> > > >
> > > That looks a correct path. Can you confirm that the actual pmd .so files
> > are
> > > in that directory on your system?
> > Yes,
> > librte_net_mlx5.so
> > librte_common_mlx5.so
> >
> > But note that there is not mlx5 so the ldd output
> > ldd /root/local/openvswitch-2.14/sbin/ovs-vswitchd | grep -i mlx5
> >
> > I debugged it and found that pci was not in bus_list, so pci device
> > was not attached to ovs.
>
> Does librte_bus_pci.so.21 exist in "/root/local/dpdk-next-net/lib64/dpdk/pmds-21.1"
> directory too?
> Also can you turn on debug logging for EAL, which should show the individual
> drivers getting loaded at runtime. The loading is done by function "eal_plugins_init"
> in DPDK.
Hi
I found the root cause, I changed the eal_plugins_init(but not commit
this change when I build ovs with dpdk 20.11 using my personal patch).
I changed this for a long time, and now I test the Pai patch still
using the my dpdk patch. Sorry for that.

> /Bruce



-- 
Best regards, Tonghao