[ovs-dev] [PATCH] github: Fix handling of python packages.

Aaron Conole aconole at redhat.com
Mon Mar 15 11:53:21 UTC 2021


Ilya Maximets <i.maximets at ovn.org> writes:

> GitHub Actions doesn't have python locations in PATH and different
> runners might have different configuration for default python
> location and versions.  For example, on some runners python2 might
> be installed or not.
>
> Missing PATH causes weird situations on older branches where during
> one run our scripts can locate just installed flake8 and can't do
> that on a different run.  But this might also create other
> unpredictable issues on all branches.

This is strange.  We do an install via the package manager - is it not
installing to the correct PATH location?

> It's required to use actions/setup-python at v2 in order to have
> predictable version of python installed and paths correctly configured.
> Due to some bugs in GHA itself it doesn't set $HOME/.local/bin into
> PATH, so we have to do that manually for now in order to use '--user'.
> This might be fixed later in actions/setup-python or in base runners.
> We already setting it for DPDK 20.11 (I think the issue was spotted
> but not fully investigated).  Moving PATH updates to a separate step
> to make them more explicit and avaialble for all steps of the job.

*available

Do you know if GHA team has a plan to fix this?

It's a strange syntax, but it looks like the manual requires that we do
PATH updates via 'echo "..." >> $GITHUB_PATH' so, LGTM.

> Unfortunately actions/setup-python at v2 also makes invisible python
> packages installed from Ubuntu repositories.  Switching them to
> 'pip3 install'.
>
> Fixes: 6cb2f5a630e3 ("github: Add GitHub Actions workflow.")
> Reported-by: Numan Siddique <numans at ovn.org>
> Signed-off-by: Ilya Maximets <i.maximets at ovn.org>
> ---
>  .ci/linux-prepare.sh                 |  3 ++-
>  .github/workflows/build-and-test.yml | 26 +++++++++++++++++++++-----
>  2 files changed, 23 insertions(+), 6 deletions(-)
>
> diff --git a/.ci/linux-prepare.sh b/.ci/linux-prepare.sh
> index 69a40011f..c55125cf7 100755
> --- a/.ci/linux-prepare.sh
> +++ b/.ci/linux-prepare.sh
> @@ -20,7 +20,8 @@ cd sparse
>  make -j4 HAVE_LLVM= HAVE_SQLITE= install
>  cd ..
>  
> -pip3 install --disable-pip-version-check --user flake8 hacking
> +pip3 install --disable-pip-version-check --user \
> +    flake8 hacking sphinx pyOpenSSL wheel setuptools
>  pip3 install --user --upgrade docutils
>  pip3 install --user  'meson==0.47.1'
>  
> diff --git a/.github/workflows/build-and-test.yml b/.github/workflows/build-and-test.yml
> index 1bb72bbb1..ce98a9f98 100644
> --- a/.github/workflows/build-and-test.yml
> +++ b/.github/workflows/build-and-test.yml
> @@ -8,9 +8,7 @@ jobs:
>        dependencies: |
>          automake libtool gcc bc libjemalloc1 libjemalloc-dev    \
>          libssl-dev llvm-dev libelf-dev libnuma-dev libpcap-dev  \
> -        ninja-build python3-openssl python3-pip                 \
> -        python3-setuptools python3-sphinx python3-wheel         \
> -        selinux-policy-dev
> +        ninja-build selinux-policy-dev
>        deb_dependencies: |
>          linux-headers-$(uname -r) build-essential fakeroot devscripts equivs
>        AFXDP:       ${{ matrix.afxdp }}
> @@ -115,6 +113,16 @@ jobs:
>      - name: checkout
>        uses: actions/checkout at v2
>  
> +    - name: update PATH
> +      run:  |
> +        echo "$HOME/bin"        >> $GITHUB_PATH
> +        echo "$HOME/.local/bin" >> $GITHUB_PATH
> +
> +    - name: set up python
> +      uses: actions/setup-python at v2
> +      with:
> +        python-version: '3.x'
> +
>      - name: create ci signature file for the dpdk cache key
>        if:   matrix.dpdk != '' || matrix.dpdk_shared != ''
>        # This will collect most of DPDK related lines, so hash will be different
> @@ -151,7 +159,7 @@ jobs:
>        run:  ./.ci/linux-prepare.sh
>  
>      - name: build
> -      run:  PATH="$PATH:$HOME/bin:$HOME/.local/bin" ./.ci/linux-build.sh
> +      run:  ./.ci/linux-build.sh
>  
>      - name: upload deb packages
>        if:   matrix.deb_package != ''
> @@ -194,12 +202,20 @@ jobs:
>      steps:
>      - name: checkout
>        uses: actions/checkout at v2
> +    - name: update PATH
> +      run:  |
> +        echo "$HOME/bin"        >> $GITHUB_PATH
> +        echo "$HOME/.local/bin" >> $GITHUB_PATH
> +    - name: set up python
> +      uses: actions/setup-python at v2
> +      with:
> +        python-version: '3.x'
>      - name: install dependencies
>        run:  brew install automake libtool
>      - name: prepare
>        run:  ./.ci/osx-prepare.sh
>      - name: build
> -      run:  PATH="$PATH:$HOME/bin" ./.ci/osx-build.sh
> +      run:  ./.ci/osx-build.sh
>      - name: upload logs on failure
>        if: failure()
>        uses: actions/upload-artifact at v2



More information about the dev mailing list