[ovs-dev] [PATCH RFC dpdk-latest] build: Add support for DPDK meson build.

Ilya Maximets i.maximets at ovn.org
Thu Jul 2 16:24:25 UTC 2020


On 7/2/20 6:10 PM, Pai G, Sunil wrote:
>> -----Original Message-----
>> From: Richardson, Bruce <bruce.richardson at intel.com>
>> Sent: Thursday, July 2, 2020 9:32 PM
>> To: Pai G, Sunil <sunil.pai.g at intel.com>; Ilya Maximets
>> <i.maximets at ovn.org>; dev at openvswitch.org
>> Cc: Stokes, Ian <ian.stokes at intel.com>; david.marchand at redhat.com;
>> Tummala, Sivaprasad <sivaprasad.tummala at intel.com>
>> Subject: RE: [PATCH RFC dpdk-latest] build: Add support for DPDK meson
>> build.
>>
>>
>>
>>> -----Original Message-----
>>> From: Pai G, Sunil <sunil.pai.g at intel.com>
>>> Sent: Thursday, July 2, 2020 4:25 PM
>>> To: Ilya Maximets <i.maximets at ovn.org>; dev at openvswitch.org
>>> Cc: Stokes, Ian <ian.stokes at intel.com>; david.marchand at redhat.com;
>>> Richardson, Bruce <bruce.richardson at intel.com>; Tummala, Sivaprasad
>>> <sivaprasad.tummala at intel.com>
>>> Subject: RE: [PATCH RFC dpdk-latest] build: Add support for DPDK meson
>>> build.
>>>
>>>> -----Original Message-----
>>>> From: Ilya Maximets <i.maximets at ovn.org>
>>>> Sent: Thursday, July 2, 2020 7:26 PM
>>>> To: Pai G, Sunil <sunil.pai.g at intel.com>; dev at openvswitch.org
>>>> Cc: Stokes, Ian <ian.stokes at intel.com>; i.maximets at ovn.org;
>>>> david.marchand at redhat.com; Richardson, Bruce
>>>> <bruce.richardson at intel.com>; Tummala, Sivaprasad
>>>> <sivaprasad.tummala at intel.com>; i.maximets at ovn.org
>>>> Subject: Re: [PATCH RFC dpdk-latest] build: Add support for DPDK
>>>> meson
>>> build.
>>>>
>>>> On 7/2/20 3:13 PM, Sunil Pai G wrote:
>>>>> Make based build is deprecated in DPDK. Meson based build to be
>>>>> used for future DPDK releases.
>>>>>
>>>>> This updates travis, configure script and documentation for using
>>>>> DPDK Meson with OVS.
>>>>>
>>>>> Signed-off-by: Sunil Pai G <sunil.pai.g at intel.com>
>>>>
>>>> Thanks for working on this!
>>>> Not a full review, just a few quick bits.
>>>>
>>>> At first, why dpdk-latest?  Is there issue with meson build on 19.11?
>>>
>>> The linker always picked the shared DPDK libraries over static when
>>> built with Meson in DPDK-19.11. -Bstatic flag would get jumbled by
>>> libtool causing this.
>>> Thanks to Bruce, there was recently merged series which fixed a bunch
>>> of issues along with this :
>>> https://patches.dpdk.org/project/dpdk/list/?series=10690
>>> It is requested for a back port of this series to DPDK-19.11.
>>>
>>>>
>>>> Few more comments inline.
>>>>
>>>> Best regards, Ilya Maximets.
>>>>
>>>>> ---
>>>>>  .travis.yml                           |  3 ++
>>>>>  .travis/linux-build.sh                | 37 +++++++++-------
>>>>>  .travis/linux-prepare.sh              |  1 +
>>>>>  Documentation/intro/install/afxdp.rst |  2 +-
>>>>> Documentation/intro/install/dpdk.rst  | 56 ++++++++++++++++++++---
>> -
>>>>>  Makefile.am                           |  3 +-
>>>>>  acinclude.m4                          | 42 ++++++++++++------
>>>>>  parse_pkg_cfg.py                      | 62
>>> +++++++++++++++++++++++++++
>>>>>  8 files changed, 167 insertions(+), 39 deletions(-)  create mode
>>>>> 100644 parse_pkg_cfg.py
>>>>>
>>>>> diff --git a/.travis.yml b/.travis.yml index 97249c1ce..46d7ad9bb
>>>>> 100644
>>>>> --- a/.travis.yml
>>>>> +++ b/.travis.yml
>>>>> @@ -27,6 +27,9 @@ addons:
>>>>>        - selinux-policy-dev
>>>>>        - libunbound-dev
>>>>>        - libunwind-dev
>>>>> +      - python3-setuptools
>>>>> +      - python3-wheel
>>>>> +      - ninja-build
>>>>>
>>>>>  before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh
>>>>>
>>>>> diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh index
>>>>> 33b359a61..7fa7e738c 100755
>>>>> --- a/.travis/linux-build.sh
>>>>> +++ b/.travis/linux-build.sh
>>>>> @@ -85,17 +85,21 @@ function install_dpdk()  {
>>>>>      local DPDK_VER=$1
>>>>>      local VERSION_FILE="dpdk-dir/travis-dpdk-cache-version"
>>>>> +    local DPDK_OPTS=""
>>>>>
>>>>> -    if [ -z "$TRAVIS_ARCH" ] ||
>>>>> -       [ "$TRAVIS_ARCH" == "amd64" ]; then
>>>>> -        TARGET="x86_64-native-linuxapp-gcc"
>>>>> -    elif [ "$TRAVIS_ARCH" == "aarch64" ]; then
>>>>> -        TARGET="arm64-armv8a-linuxapp-gcc"
>>>>> -    else
>>>>> +    if [ "$TRAVIS_ARCH" == "aarch64" ]; then
>>>>> +        DPDK_OPTS="$DPDK_OPTS --cross-file
>>>> config/arm/arm64_armv8_linux_gcc"
>>>>
>>>> We're not cross compiling, we're actually building on aarch64 here.
>>>> This option should not be needed.  IIUC, meson should detect current
>>>> architecture and build with appropriate configuration.
>>>>
>>>> We should be able to just drop all the TRAVIS_ARCH checks for DPDK
>> here.
>>>
>>> Sure ,let me check on this . I see a similar approach taken in DPDK
>>> travis.
>>> May be Bruce can comment as well ?
>>>
>>>>
>>>>> +    elif [ "$TRAVIS_ARCH" != "amd64" ] && [ -n "$TRAVIS_ARCH" ];
>>>>> + then
>>>>>          echo "Target is unknown"
>>>>>          exit 1
>>>>>      fi
>>>>>
>>>>> +    if [ "$DPDK_SHARED" ]; then
>>>>> +        EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=shared"
>>>>> +    else
>>>>> +        EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=static"
>>>>> +    fi
>>>>
>>>> I gusee we could just change the env variable and not parse it here.
>>>> i.e. have DPDK_LIB=static defined by travis.yml by default and
>>>> redefine it for matrix entries where we need shared libs.
>>>
>>> Yes , this could be done. But if we install the libraries in a custom
>>> path using a prefix, we would have to export the LD_LIBRARY_PATH for
>>> which querying whether to build as shared/static will be
>>> required.(This in relation to the below comment as well :))
>>>
>>>
>>>>
>>>>> +
>>>>>      if [ "${DPDK_VER##refs/*/}" != "${DPDK_VER}" ]; then
>>>>>          # Avoid using cache for git tree build.
>>>>>          rm -rf dpdk-dir
>>>>> @@ -108,7 +112,8 @@ function install_dpdk()
>>>>>          if [ -f "${VERSION_FILE}" ]; then
>>>>>              VER=$(cat ${VERSION_FILE})
>>>>>              if [ "${VER}" = "${DPDK_VER}" ]; then
>>>>> -                EXTRA_OPTS="${EXTRA_OPTS} --with-dpdk=$(pwd)/dpdk-
>>> dir/build"
>>>>> +                sudo ninja -C $(pwd)/dpdk-dir/build install
>>>>> +                sudo ldconfig
>>>>
>>>> I think that installing right here inside the cached folder and just
>>>> adjusting environment variables should be a bit faster than re-
>>> installing DPDK every time.
>>>>
>>>> This script also will be a good example for people like me, who
>>>> really don't want to install development versions of DPDK globally
>>>> on a work laptop while testing OVS builds.
>>>
>>> Yes , Thanks for the suggestion. Although ,using an install to a
>>> directory with a prefix would require this patch from Bruce:
>>> https://patches.dpdk.org/patch/72271/
>>>  (which is not merged yet as of this writing) .without this , OVS
>>> would fail to run searching for few shared DPDK libraries even when
>>> built with static libraries.
>>>
>>>>
>>>>>                  echo "Found cached DPDK ${VER} build in
>>>>> $(pwd)/dpdk-
>>> dir"
>>>>>                  return
>>>>>              fi
>>>>> @@ -122,16 +127,18 @@ function install_dpdk()
>>>>>          pushd dpdk-dir
>>>>>      fi
>>>>>
>>>>> -    make config CC=gcc T=$TARGET
>>>>> +    # Disable building DPDK kernel modules. Not needed for OVS
>>>>> + build
>>> or tests.
>>>>> +    DPDK_OPTS="$DPDK_OPTS -Denable_kmods=false"
>>>>
>>>> We should also disable examples and tests at least.
>>>
>>> Sure , this will reduce the build time as well.
>>>  Will update in the next version.
>>>
>>
>> Examples will be disabled by default, but disabling tests indeed makes sense
>> here.
>>
>>>>
>>>>>
>>>>> -    if [ "$DPDK_SHARED" ]; then
>>>>> -        sed -i '/CONFIG_RTE_BUILD_SHARED_LIB=n/s/=n/=y/'
>>> build/.config
>>>>> -        export
>> LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/$TARGET/lib
>>>>> -    fi
>>>>> +    DPDK_OPTS="$DPDK_OPTS -Dc_args=-fPIC"
>>>>> +    CC=gcc meson $DPDK_OPTS build
>>>>> +    ninja -C build
>>>>> +    sudo ninja -C build install
>>>>> +
>>>>> +    # Update the library paths.
>>>>> +    sudo ldconfig
>>>>>
>>>>> -    make -j4 CC=gcc EXTRA_CFLAGS='-fPIC'
>>>>> -    EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=$(pwd)/build"
>>>>> -    echo "Installed DPDK source in $(pwd)"
>>>>> +    echo "Installed DPDK source"
>>>>>      popd
>>>>>      echo "${DPDK_VER}" > ${VERSION_FILE}  } diff --git
>>>>> a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh index
>>>>> 8cbbd5623..682f6234b 100755
>>>>> --- a/.travis/linux-prepare.sh
>>>>> +++ b/.travis/linux-prepare.sh
>>>>> @@ -16,6 +16,7 @@ cd ..
>>>>>
>>>>>  pip3 install --disable-pip-version-check --user flake8 hacking
>>>>>  pip3 install --user --upgrade docutils
>>>>> +pip3 install --user  'meson==0.47.1'
>>>>
>>>> I understand that that is the minimum required version, but why not
>>>> the most recent one or, at least, a bit more recent?
>>>>
>>>
>>> Yes , quoting David's commit message from DPDK travis :
>>> "
>>>     meson 0.53.0 has a compatibility issue [1] with the python 3.5.2
>>> that comes
>>>     in Ubuntu 16.04.
>>>     On the other hand, the minimal version supported in dpdk is 0.47.1.
>>>
>>>     Stick to this version to avoid getting hit by regressions in meson
>>> latest
>>>     shiny release.
>>>
>>>     1: https://github.com/mesonbuild/meson/issues/6427
>>> "
>>>
>>
>> Does your travis instance use 16.04 or 18.04? If possible can you standardize
>> on a new specific version to get some additional benefits. For example, with
>> meson
>> 0.54 there is support for "uninstalled" pkg-config files, which you can use for
>> linking against a DPDK instance which has not been installed on the system.
>> [https://mesonbuild.com/Release-notes-for-0-54-0.html#uninstalled-
>> pkgconfig-files]
>> I think that feature may be of interest or of use for the future.
> 
> Currently 16.04 is being used . I tried to moving to 18.04 as well. But seems like few other 
> test cases which require kernel 3.16 seem to fail. Probably Ilya/David/Ian could comment if those test cases
> can be removed and we could move to 18.04 travis? 

The incompatibility issue was fixed in v0.53.1 release and should work fine
with 0.54 on ubuntu 16.04 without upgrading the base image to 18.04.
So, it makes sense to standardize on 0.54 for now.  Could you, please, try it?


More information about the dev mailing list