[ovs-dev] [PATCH] github: Fix handling of python packages.
Ilya Maximets
i.maximets at ovn.org
Mon Mar 15 12:04:41 UTC 2021
On 3/15/21 12:53 PM, Aaron Conole wrote:
> Ilya Maximets <i.maximets at ovn.org> writes:
>
>> GitHub Actions doesn't have python locations in PATH and different
>> runners might have different configuration for default python
>> location and versions. For example, on some runners python2 might
>> be installed or not.
>>
>> Missing PATH causes weird situations on older branches where during
>> one run our scripts can locate just installed flake8 and can't do
>> that on a different run. But this might also create other
>> unpredictable issues on all branches.
>
> This is strange. We do an install via the package manager - is it not
> installing to the correct PATH location?
We are installing 'flake8' and 'hacking' by pip and these are installed
to python locations that are not in PATH.
>
>> It's required to use actions/setup-python at v2 in order to have
>> predictable version of python installed and paths correctly configured.
>> Due to some bugs in GHA itself it doesn't set $HOME/.local/bin into
>> PATH, so we have to do that manually for now in order to use '--user'.
>> This might be fixed later in actions/setup-python or in base runners.
>> We already setting it for DPDK 20.11 (I think the issue was spotted
>> but not fully investigated). Moving PATH updates to a separate step
>> to make them more explicit and avaialble for all steps of the job.
>
> *available
>
> Do you know if GHA team has a plan to fix this?
There are 2 issues:
1. For general virtual-environments:
https://github.com/actions/virtual-environments/issues/2455
They tried and failed. :)
Promised to fix once they figure out what happened, but I'd not trust them.
2. For setup-python action:
https://github.com/actions/setup-python/issues/99
State is unclear to me.
>
> It's a strange syntax, but it looks like the manual requires that we do
> PATH updates via 'echo "..." >> $GITHUB_PATH' so, LGTM.
Yeah, never used this syntax before.
>
>> Unfortunately actions/setup-python at v2 also makes invisible python
>> packages installed from Ubuntu repositories. Switching them to
>> 'pip3 install'.
>>
>> Fixes: 6cb2f5a630e3 ("github: Add GitHub Actions workflow.")
>> Reported-by: Numan Siddique <numans at ovn.org>
>> Signed-off-by: Ilya Maximets <i.maximets at ovn.org>
>> ---
>> .ci/linux-prepare.sh | 3 ++-
>> .github/workflows/build-and-test.yml | 26 +++++++++++++++++++++-----
>> 2 files changed, 23 insertions(+), 6 deletions(-)
>>
>> diff --git a/.ci/linux-prepare.sh b/.ci/linux-prepare.sh
>> index 69a40011f..c55125cf7 100755
>> --- a/.ci/linux-prepare.sh
>> +++ b/.ci/linux-prepare.sh
>> @@ -20,7 +20,8 @@ cd sparse
>> make -j4 HAVE_LLVM= HAVE_SQLITE= install
>> cd ..
>>
>> -pip3 install --disable-pip-version-check --user flake8 hacking
>> +pip3 install --disable-pip-version-check --user \
>> + flake8 hacking sphinx pyOpenSSL wheel setuptools
>> pip3 install --user --upgrade docutils
>> pip3 install --user 'meson==0.47.1'
>>
>> diff --git a/.github/workflows/build-and-test.yml b/.github/workflows/build-and-test.yml
>> index 1bb72bbb1..ce98a9f98 100644
>> --- a/.github/workflows/build-and-test.yml
>> +++ b/.github/workflows/build-and-test.yml
>> @@ -8,9 +8,7 @@ jobs:
>> dependencies: |
>> automake libtool gcc bc libjemalloc1 libjemalloc-dev \
>> libssl-dev llvm-dev libelf-dev libnuma-dev libpcap-dev \
>> - ninja-build python3-openssl python3-pip \
>> - python3-setuptools python3-sphinx python3-wheel \
>> - selinux-policy-dev
>> + ninja-build selinux-policy-dev
>> deb_dependencies: |
>> linux-headers-$(uname -r) build-essential fakeroot devscripts equivs
>> AFXDP: ${{ matrix.afxdp }}
>> @@ -115,6 +113,16 @@ jobs:
>> - name: checkout
>> uses: actions/checkout at v2
>>
>> + - name: update PATH
>> + run: |
>> + echo "$HOME/bin" >> $GITHUB_PATH
>> + echo "$HOME/.local/bin" >> $GITHUB_PATH
>> +
>> + - name: set up python
>> + uses: actions/setup-python at v2
>> + with:
>> + python-version: '3.x'
>> +
>> - name: create ci signature file for the dpdk cache key
>> if: matrix.dpdk != '' || matrix.dpdk_shared != ''
>> # This will collect most of DPDK related lines, so hash will be different
>> @@ -151,7 +159,7 @@ jobs:
>> run: ./.ci/linux-prepare.sh
>>
>> - name: build
>> - run: PATH="$PATH:$HOME/bin:$HOME/.local/bin" ./.ci/linux-build.sh
>> + run: ./.ci/linux-build.sh
>>
>> - name: upload deb packages
>> if: matrix.deb_package != ''
>> @@ -194,12 +202,20 @@ jobs:
>> steps:
>> - name: checkout
>> uses: actions/checkout at v2
>> + - name: update PATH
>> + run: |
>> + echo "$HOME/bin" >> $GITHUB_PATH
>> + echo "$HOME/.local/bin" >> $GITHUB_PATH
>> + - name: set up python
>> + uses: actions/setup-python at v2
>> + with:
>> + python-version: '3.x'
>> - name: install dependencies
>> run: brew install automake libtool
>> - name: prepare
>> run: ./.ci/osx-prepare.sh
>> - name: build
>> - run: PATH="$PATH:$HOME/bin" ./.ci/osx-build.sh
>> + run: ./.ci/osx-build.sh
>> - name: upload logs on failure
>> if: failure()
>> uses: actions/upload-artifact at v2
>
More information about the dev
mailing list