[ovs-dev] Fwd: [PATCH] build: check libibverbs deps before linking with dpdk

David Marchand david.marchand at redhat.com
Mon Nov 15 09:03:50 UTC 2021


Hello,

On Sun, Nov 14, 2021 at 10:27 AM Harold Huang <baymaxhuang at gmail.com> wrote:
> David Marchand <david.marchand at redhat.com> 于2021年11月12日周五 下午7:01写道:
> > OVS requests ibverbs only if mlx drivers are present in DPDK build (+
> > option RTE_IBVERBS_LINK_DLOPEN iirc).
> >
> > For your issue, things that come to mind are an outdated build
> > directory, maybe following a pull from OVS sources, or maybe you had
> > ibverbs installed in the past on this system, and now it has been
> > removed.
> >
> > Did you compile OVS from a scratch/clean environment?
>
> Hi, David,
>    I also started a clean centos 8.2 vm environment and init the vm
> environment with the following command:
> $dnf -y install 'dnf-command(config-manager)' && dnf config-manager
> --set-enabled PowerTools && dnf install -y gcc make numactl-devel
> meson autoconf automake libtool unbound unbound-devel
>
> I could also build dpdk and link ovs successfully. But after I
> installed libpcap-devel and rebuild dpdk, I could not link ovs
> anymore. And the error message is the same as what I posted before. I
> found libpcap.pc also has a dependency about libibverbs:
> #
> # pkg-config file for libpcap.
> #
> # These variables come from the configure script, so includedir and
> # libdir may be defined in terms of prefix and exec_prefix, so the
> # latter must be defined as well.
> #
> prefix="/usr"
> exec_prefix="/usr"
> includedir="/usr/include"
> libdir="/usr/lib64"
>
> Name: libpcap
> Description: Platform-independent network traffic capture library
> Version: 1.9.1
> Libs: -L${libdir} -lpcap
> Libs.private: -libverbs
> Cflags: -I${includedir}

I can reproduce the issue now.
As you describe, I installed libpcap-devel, rebuilt and installed DPDK, then:

[dpdk at centos8 build]$
PKG_CONFIG_PATH=/home/dpdk/dpdk/builddir/install/lib64/pkgconfig
../configure --with-dpdk
...
checking for struct tcf_t.firstuse... yes
checking whether dpdk is enabled... yes
checking for DPDK... yes
checking for faulty pkg-config version... no
checking for rte_config.h... yes
checking whether RTE_LIBRTE_VHOST_NUMA is declared... yes
checking for library containing get_mempolicy... -lnuma
checking whether RTE_EAL_NUMA_AWARE_HUGEPAGES is declared... yes
checking for library containing get_mempolicy... (cached) -lnuma
checking whether RTE_NET_PCAP is declared... yes
checking for library containing pcap_dump_close... -lpcap
checking whether RTE_NET_AF_XDP is declared... no
checking whether RTE_LIBRTE_VHOST_NUMA is declared... (cached) yes
checking whether RTE_NET_MLX5 is declared... no
checking whether RTE_NET_MLX4 is declared... no
checking for library containing dlopen... -ldl
checking whether linking with dpdk works... no
configure: error: Could not find DPDK library in default search path,
update PKG_CONFIG_PATH for pkg-config to find the .pc file in
non-standard location

(fwiw, linking against dpdk dynamically works).


The problem is either on dpdk side or (I would lean to) Centos
packaging, not OVS.
It can be reproduced with DPDK examples:

[dpdk at centos8 build]$ cd ~/dpdk/builddir/install/share/dpdk/examples/helloworld/
[dpdk at centos8 helloworld]$
PKG_CONFIG_PATH=/home/dpdk/dpdk/builddir/install/lib64/pkgconfig make
static
cc -O3 -I/home/dpdk/dpdk/builddir/install/include -include
rte_config.h -march=native -mno-avx512f  -DALLOW_EXPERIMENTAL_API
main.c -o build/helloworld-static  -Wl,--whole-archive
-L/home/dpdk/dpdk/builddir/install/lib64 -l:librte_common_cpt.a
-l:librte_common_dpaax.a -l:librte_common_iavf.a
-l:librte_common_octeontx.a -l:librte_common_octeontx2.a
-l:librte_common_sfc_efx.a -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a
-l:librte_bus_ifpga.a -l:librte_bus_pci.a -l:librte_bus_vdev.a
-l:librte_bus_vmbus.a -l:librte_common_qat.a
-l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a
-l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a
-l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a
-l:librte_mempool_stack.a -l:librte_net_af_packet.a
-l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a
-l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnx2x.a
-l:librte_net_bnxt.a -l:librte_net_cxgbe.a -l:librte_net_dpaa.a
-l:librte_net_dpaa2.a -l:librte_net_e1000.a -l:librte_net_ena.a
-l:librte_net_enetc.a -l:librte_net_enic.a -l:librte_net_failsafe.a
-l:librte_net_fm10k.a -l:librte_net_i40e.a -l:librte_net_hinic.a
-l:librte_net_hns3.a -l:librte_net_iavf.a -l:librte_net_ice.a
-l:librte_net_igc.a -l:librte_net_ixgbe.a -l:librte_net_kni.a
-l:librte_net_liquidio.a -l:librte_net_memif.a -l:librte_net_netvsc.a
-l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a
-l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a
-l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a
-l:librte_net_softnic.a -l:librte_net_tap.a -l:librte_net_thunderx.a
-l:librte_net_txgbe.a -l:librte_net_vdev_netvsc.a
-l:librte_net_vhost.a -l:librte_net_virtio.a -l:librte_net_vmxnet3.a
-l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a
-l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a
-l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a
-l:librte_compress_octeontx.a -l:librte_compress_zlib.a
-l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a
-l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a
-l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a
-l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a
-l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a
-l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a
-l:librte_vhost.a -l:librte_stack.a -l:librte_security.a
-l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a
-l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a
-l:librte_power.a -l:librte_member.a -l:librte_lpm.a
-l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a
-l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a
-l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a
-l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a
-l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a
-l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a
-l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a
-l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a
-l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a
-l:librte_kvargs.a -Wl,--no-whole-archive -Wl,--export-dynamic
-Wl,--as-needed -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify
-lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec
-lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder
-lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power
-lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats
-lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd
-lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile
-lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash
-lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter
-lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal
-lrte_telemetry -lrte_kvargs -pthread -lm -ldl -lnuma -lz -lpcap
-libverbs
/usr/bin/ld: cannot find -libverbs
collect2: error: ld returned 1 exit status
make: *** [Makefile:35: build/helloworld-static] Error 1

[dpdk at centos8 helloworld]$ ls -1 /usr/lib64/libpcap.so*
/usr/lib64/libibverbs.so*
/usr/lib64/libibverbs.so.1
/usr/lib64/libibverbs.so.1.11.32.0
/usr/lib64/libpcap.so
/usr/lib64/libpcap.so.1
/usr/lib64/libpcap.so.1.9.1

Absence of libibverbs.so is explained by the the absence of
libibverbs-devel package.
[dpdk at centos8 helloworld]$ rpm -q libpcap libpcap-devel libibverbs
libibverbs-devel
libpcap-1.9.1-5.el8.x86_64
libpcap-devel-1.9.1-5.el8.x86_64
libibverbs-32.0-4.el8.x86_64
package libibverbs-devel is not installed

Problem is solved for me, by installing libibverbs-devel.

Copying Bruce and Timothy who may have an opinion on this.


-- 
David Marchand



More information about the dev mailing list