[ovs-dev] [PATCH dpdk-latest v4] build: Add support for DPDK meson build.

Tue Dec 15 12:04:05 UTC 2020

> -----Original Message-----
> From: Pai G, Sunil <sunil.pai.g at intel.com>
> Sent: Tuesday, December 15, 2020 12:01 PM
> To: Stokes, Ian <ian.stokes at intel.com>; Ilya Maximets
> <i.maximets at ovn.org>; Richardson, Bruce <bruce.richardson at intel.com>;
> Tonghao Zhang <xiangxia.m.yue at gmail.com>
> Cc: ovs dev <dev at openvswitch.org>; David Marchand
> <david.marchand at redhat.com>
> Subject: RE: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for DPDK
> meson build.
> 
> Hi,
> 
> <snipped>
> 
> > > >>>> +   When DPDK is built using Meson, and OVS must consume DPDK
> > > >>>> + shared
> > > >> libraries
> > > >>>> +   (also equivalent to leaving --with-dpdk option empty)::
> > > >>>> +
> > > >>>> +       $ ./configure --with-dpdk=shared
> > > >>> Hi all
> > > >>> when I build ovs with --with-dpdk=shared, Compilation is fine, but
> > > >>> we I launch the ovs:
> > > >>>
> > > >>> 41 2020-12-14T10:42:22.964Z|00018|dpdk|INFO|EAL ARGS: ovs-
> > vswitchd
> > > >>> -
> > > a
> > > >>>
> > > >>
> > >
> > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r
> > epr
> > > e
> > > sen
> > > >> tor=[0-6]
> > > >>> -c 0xe --huge-dir /dev/hugepages --socket    -mem 1024,1024
> > > >>> --socket-limit 1024,1024.
> > > >>> 42 2020-12-14T10:42:22.969Z|00019|dpdk|INFO|EAL: Detected 56
> > > >>> lcore(s)
> > > >>> 43 2020-12-14T10:42:22.969Z|00020|dpdk|INFO|EAL: Detected 2
> > NUMA
> > > nodes
> > > >>> 44 2020-12-14T10:42:22.969Z|00021|dpdk|ERR|EAL: failed to parse
> > > >>> device "0000:82:00.0"
> > > >>> 45 2020-12-14T10:42:22.969Z|00022|dpdk|ERR|EAL: Unable to parse
> > > device
> > > >>>
> > > >>
> > >
> > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r
> > ep
> > > r
> > > ese
> > > >> ntor=[0-6]'
> > > >>> 46 2020-12-14T10:42:22.969Z|00023|dpdk|EMER|Unable to initialize
> > > DPDK:
> > > >>> No such device
> > > >>>
> > > >>> In dpdk:
> > > >>> eal_plugins_init
> > > >>> rte_pci_scan
> > > >>> rte_bus_register
> > > >>> eal_option_device_parse -- no pci bus
> > > >>>
> > > >>> pkg-config is pkgconf-1.4.2
> > > >>>> +   When DPDK is built using Meson, and OVS must consume DPDK
> > > >>>> + static
> > > >> libraries::
> > > >>>> +
> > > >>>> +       $ ./configure --with-dpdk=static
> > > >>> Only one issue, if we use the pkgconfig-0.27 The ovs log:
> > > >>> 2020-12-14T11:58:22.622Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd
> > -a
> > > >>>
> > > >>
> > >
> > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r
> > epr
> > > e
> > > sen
> > > >> tor=[0-6]
> > > >>> -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024
> > > >>> --socket-limit 1024,1024.
> > > >>> 2020-12-14T11:58:22.627Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s)
> > > >>> 2020-12-14T11:58:22.627Z|00020|dpdk|INFO|EAL: Detected 2 NUMA
> > > nodes
> > > >>> 2020-12-14T11:58:22.627Z|00021|dpdk|ERR|EAL: failed to parse
> > > >>> device "0000:82:00.0"
> > > >>> 2020-12-14T11:58:22.627Z|00022|dpdk|ERR|EAL: Unable to parse
> > > >>> device
> > > >>>
> > > >>
> > >
> > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r
> > ep
> > > r
> > > ese
> > > >> ntor=[0-6]'
> > > >>> 2020-12-14T11:58:22.627Z|00023|dpdk|EMER|Unable to initialize
> > DPDK:
> > > No
> > > >>> such device
> > > >>>
> > > >>> Because  /bin/pkg-config --static --libs libdpdk
> > > >>> pkg_cv_DPDK_LIBS='-Wl,--whole-archive -Wl,--no-whole-archive
> > > >>> -Wl,--as-needed -pthread -L/root/local/dpdk-next-net/lib64
> > > >>> -l:librte_common_cpt.a -l:librte_common_dpaax.a
> > > >>> -l:librte_common_iavf.a -l:librte_common_octeontx.a
> > > >>> -l:librte_common_octeontx2.a -l:librte_common_sfc_efx.a
> > > >>> -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a
> > > >>> -l:librte_bus_pci.a -l:librte_bus_vdev.a -l:librte_bus_vmbus.a
> > > >>> -l:librte_common_mlx5.a -l:librte_common_qat.a
> > > >>> -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a
> > > >>> -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a
> > > >>> -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a
> > > >>> -l:librte_mempool_stack.a -l:librte_net_af_packet.a
> > > >>> -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a
> > > >>> -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a
> > > >>> -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a
> > > >>> -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a
> > > >>> -l:librte_net_enic.a -l:librte_net_failsafe.a
> > > >>> -l:librte_net_fm10k.a -l:librte_net_i40e.a -l:librte_net_hinic.a
> > > >>> -l:librte_net_hns3.a -l:librte_net_iavf.a -l:librte_net_ice.a
> > > >>> -l:librte_net_igc.a -l:librte_net_ixgbe.a -l:librte_net_kni.a
> > > >>> -l:librte_net_liquidio.a -l:librte_net_memif.a
> > > >>> -l:librte_net_mlx5.a -l:librte_net_netvsc.a -l:librte_net_nfp.a
> > > >>> -l:librte_net_null.a -l:librte_net_octeontx.a
> > > >>> -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a
> > > >>> -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a
> > > >>> -l:librte_net_softnic.a -l:librte_net_thunderx.a
> > > >>> -l:librte_net_txgbe.a -l:librte_net_vdev_netvsc.a
> > > >>> -l:librte_net_vhost.a -l:librte_net_virtio.a
> > > >>> -l:librte_net_vmxnet3.a -l:librte_raw_dpaa2_cmdif.a
> > > >>> -l:librte_raw_dpaa2_qdma.a -l:librte_raw_ioat.a
> > > >>> -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a
> > > >>> -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a
> > > >>> -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a
> > > >>> -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a
> > > >>> -l:librte_vdpa_mlx5.a -l:librte_baseband_null.a
> > > >>> -l:librte_baseband_turbo_sw.a -l:librte_baseband_fpga_lte_fec.a
> > > >>> -l:librte_baseband_fpga_5gnr_fec.a
> > > >>> -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a
> > > >>> -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a
> > > >>> -l:librte_table.a -l:librte_port.a -l:librte_fib.a
> > > >>> -l:librte_ipsec.a -l:librte_vhost.a -l:librte_stack.a
> > > >>> -l:librte_security.a -l:librte_sched.a -l:librte_reorder.a
> > > >>> -l:librte_rib.a -l:librte_regexdev.a -l:librte_rawdev.a
> > > >>> -l:librte_pdump.a -l:librte_power.a -l:librte_member.a
> > > >>> -l:librte_lpm.a -l:librte_latencystats.a -l:librte_kni.a
> > > >>> -l:librte_jobstats.a -l:librte_ip_frag.a -l:librte_gso.a
> > > >>> -l:librte_gro.a -l:librte_eventdev.a -l:librte_efd.a
> > > >>> -l:librte_distributor.a -l:librte_cryptodev.a
> > > >>> -l:librte_compressdev.a -l:librte_cfgfile.a
> > > >>> -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a
> > > >>> -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a
> > > >>> -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a
> > > >>> -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a
> > > >>> -l:librte_mempool.a -l:librte_rcu.a -l:librte_ring.a
> > > >>> -l:librte_eal.a -l:librte_telemetry.a -l:librte_kvargs.a -lmlx5
> > > >>> -libverbs -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify
> > > >>> -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec
> > > >>> -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder
> > > >>> -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power
> > > >>> -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats
> > > >>> -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd
> > > >>> -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile
> > > >>> -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash
> > > >>> -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter -
> lrte_net
> > -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal -lrte_telemetry
> -
> > lrte_kvargs -lm -ldl -lnuma -lpcap  '
> > > >>>
> > > >>> Note that:
> > > >>> -Wl,--whole-archive -Wl,--no-whole-archive (RTE_INIT in dpdk
> > > >>> doesn't
> > > >> work.)
> > > >>> I guess we dont hope that, It should be:
> > > >>> -Wl,--whole-archive ...(other -lxxx)...  -Wl,--no-whole-archive
> > > >>>
> > > >>> update pkg-config to pkgconf-1.4.2 /bin/pkg-config --static --libs
> > > >>> libdpdk pkg_cv_DPDK_LIBS='-Wl,--whole-archive
> > > >>> -L/root/local/dpdk-next-net/lib64 -l:librte_common_cpt.a
> > > >>> -l:librte_common_dpaax.a -l:librte_common_iavf.a
> > > >>> -l:librte_common_octeontx.a -l:librte_common_octeontx2.a
> > > >>> -l:librte_common_sfc_efx.a -l:librte_bus_dpaa.a
> > > >>> -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a -l:librte_bus_pci.a
> > > >>> -l:librte_bus_vdev.a -l:librte_bus_vmbus.a -l:librte_common_mlx5.a
> > > >>> -l:librte_common_qat.a -l:librte_mempool_bucket.a
> > > >>> -l:librte_mempool_dpaa.a -l:librte_mempool_dpaa2.a
> > > >>> -l:librte_mempool_octeontx.a -l:librte_mempool_octeontx2.a
> > > >>> -l:librte_mempool_ring.a -l:librte_mempool_stack.a
> > > >>> -l:librte_net_af_packet.a -l:librte_net_ark.a
> > > >>> -l:librte_net_atlantic.a -l:librte_net_avp.a -l:librte_net_axgbe.a
> > > >>> -l:librte_net_bond.a -l:librte_net_bnxt.a -l:librte_net_cxgbe.a
> > > >>> -l:librte_net_dpaa.a -l:librte_net_dpaa2.a -l:librte_net_e1000.a
> > > >>> -l:librte_net_ena.a -l:librte_net_enetc.a -l:librte_net_enic.a
> > > >>> -l:librte_net_failsafe.a -l:librte_net_fm10k.a
> > > >>> -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a
> > > >>> -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a
> > > >>> -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a
> > > >>> -l:librte_net_memif.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a
> > > >>> -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a
> > > >>> -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a
> > > >>> -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a
> > > >>> -l:librte_net_softnic.a -l:librte_net_thunderx.a
> > > >>> -l:librte_net_txgbe.a -l:librte_net_vdev_netvsc.a
> > > >>> -l:librte_net_vhost.a -l:librte_net_virtio.a
> > > >>> -l:librte_net_vmxnet3.a -l:librte_raw_dpaa2_cmdif.a
> > > >>> -l:librte_raw_dpaa2_qdma.a -l:librte_raw_ioat.a
> > > >>> -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a
> > > >>> -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a
> > > >>> -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a
> > > >>> -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a
> > > >>> -l:librte_vdpa_mlx5.a -l:librte_baseband_null.a
> > > >>> -l:librte_baseband_turbo_sw.a -l:librte_baseband_fpga_lte_fec.a
> > > >>> -l:librte_baseband_fpga_5gnr_fec.a
> > > >>> -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a
> > > >>> -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a
> > > >>> -l:librte_table.a -l:librte_port.a -l:librte_fib.a
> > > >>> -l:librte_ipsec.a -l:librte_vhost.a -l:librte_stack.a
> > > >>> -l:librte_security.a -l:librte_sched.a -l:librte_reorder.a
> > > >>> -l:librte_rib.a -l:librte_regexdev.a -l:librte_rawdev.a
> > > >>> -l:librte_pdump.a -l:librte_power.a -l:librte_member.a
> > > >>> -l:librte_lpm.a -l:librte_latencystats.a -l:librte_kni.a
> > > >>> -l:librte_jobstats.a -l:librte_ip_frag.a -l:librte_gso.a
> > > >>> -l:librte_gro.a -l:librte_eventdev.a -l:librte_efd.a
> > > >>> -l:librte_distributor.a -l:librte_cryptodev.a
> > > >>> -l:librte_compressdev.a -l:librte_cfgfile.a
> > > >>> -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a
> > > >>> -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a
> > > >>> -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a
> > > >>> -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a
> > > >>> -l:librte_mempool.a -l:librte_rcu.a -l:librte_ring.a
> > > >>> -l:librte_eal.a -l:librte_telemetry.a -l:librte_kvargs.a
> > > >>> -Wl,--no-whole-archive -lpcap -lmlx5 -libverbs -Wl,--as-needed
> > > >>> -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify
> > > >>> -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec
> > > >>> -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder
> > > >>> -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power
> > > >>> -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats
> > > >>> -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd
> > > >>> -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile
> > > >>> -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash
> > > >>> -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter -
> lrte_net
> > -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal -lrte_telemetry
> -
> > lrte_kvargs -pthread -lm -ldl -lnuma -lpcap '
> > > >>>
> > > >>> ovs works fine.
> > > >>
> > > >> Hmm.  Thanks for the report.
> > > >>
> > > >> It'll be great if you can test this with new version of this patch:
> > > >> https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825-
> > > 30877-1-
> > > >> git-send-email-ian.stokes at intel.com/
> > > >>
> > > >> Bruce, David, it looks like not all versions of pkg-config could be
> used.
> > > >> Do you know what is the minimal required version or how to protect
> > > >> us from this kind of issues?
> > > >>
> > > >> Best regards, Ilya Maximets.
> > > >
> > > > This was just recently flagged to us and discussed on this email
> > > > thread on the dpdk dev list.
> > > >
> > > > http://inbox.dpdk.org/dev/20201029091638.26646-1-
> > getelson at nvidia.com
> > > > /
> > > >
> > > > At the end of the discussion the follow was noted in our docs as a
> > > > known issue for DPDK.
> > > >
> > > > "pkg-config v0.27 supplied with RHEL-7 does not process correctly
> > > > libdpdk.pc
> > > Libs.private section."
> > > >
> > > > If you have any suggestions as to how we could work around this
> > > > issue, I'd be very keen to hear them, though unfortunately it does
> > > > seem to be a bug in pkg-config that is outside of our control.
> > >
> > > Thanks for the information.  Ugh.
> > >
> > > It looks like we need to implement the check in our configure script
> > > and abort if pkg-config is bad to avoid weired runtime issues due to
> not
> > linked libs.
> > >
> > > Probably, a grep, similar to what is in above dpdk patch, over the
> > > DPDK_vswitchd_LDFLAGS with some meaningfull error message should do
> > > the trick.
> > >
> > > Ian, what do you think?
> >
> > Sounds good, Let myself and Sunil take a look at this and send a new v4
> with
> > the check.
> 
> Sure , I think this could be added.
> But , if there is a check already in place in DPDK, would we need one in
> OVS as well ?
> 

There is not a check in place in DPDK, because while we can check the pkg-config
version on the build machine, it's the version on the end-user machine (the OVS
build machine in this case) that matters, and we can't check that as part of a
DPDK build.

/Bruce