[ovs-dev] [PATCH] github: Fix handling of python packages.

Ilya Maximets i.maximets at ovn.org
Fri Mar 12 18:28:25 UTC 2021

GitHub Actions doesn't have python locations in PATH and different
runners might have different configuration for default python
location and versions.  For example, on some runners python2 might
be installed or not.

Missing PATH causes weird situations on older branches where during
one run our scripts can locate just installed flake8 and can't do
that on a different run.  But this might also create other
unpredictable issues on all branches.

It's required to use actions/setup-python at v2 in order to have
predictable version of python installed and paths correctly configured.
Due to some bugs in GHA itself it doesn't set $HOME/.local/bin into
PATH, so we have to do that manually for now in order to use '--user'.
This might be fixed later in actions/setup-python or in base runners.
We already setting it for DPDK 20.11 (I think the issue was spotted
but not fully investigated).  Moving PATH updates to a separate step
to make them more explicit and avaialble for all steps of the job.

Unfortunately actions/setup-python at v2 also makes invisible python
packages installed from Ubuntu repositories.  Switching them to
'pip3 install'.

Fixes: 6cb2f5a630e3 ("github: Add GitHub Actions workflow.")
Reported-by: Numan Siddique <numans at ovn.org>
Signed-off-by: Ilya Maximets <i.maximets at ovn.org>
 .ci/linux-prepare.sh                 |  3 ++-
 .github/workflows/build-and-test.yml | 26 +++++++++++++++++++++-----
 2 files changed, 23 insertions(+), 6 deletions(-)

diff --git a/.ci/linux-prepare.sh b/.ci/linux-prepare.sh
index 69a40011f..c55125cf7 100755
--- a/.ci/linux-prepare.sh
+++ b/.ci/linux-prepare.sh
@@ -20,7 +20,8 @@ cd sparse
 make -j4 HAVE_LLVM= HAVE_SQLITE= install
 cd ..
-pip3 install --disable-pip-version-check --user flake8 hacking
+pip3 install --disable-pip-version-check --user \
+    flake8 hacking sphinx pyOpenSSL wheel setuptools
 pip3 install --user --upgrade docutils
 pip3 install --user  'meson==0.47.1'
diff --git a/.github/workflows/build-and-test.yml b/.github/workflows/build-and-test.yml
index 1bb72bbb1..ce98a9f98 100644
--- a/.github/workflows/build-and-test.yml
+++ b/.github/workflows/build-and-test.yml
@@ -8,9 +8,7 @@ jobs:
       dependencies: |
         automake libtool gcc bc libjemalloc1 libjemalloc-dev    \
         libssl-dev llvm-dev libelf-dev libnuma-dev libpcap-dev  \
-        ninja-build python3-openssl python3-pip                 \
-        python3-setuptools python3-sphinx python3-wheel         \
-        selinux-policy-dev
+        ninja-build selinux-policy-dev
       deb_dependencies: |
         linux-headers-$(uname -r) build-essential fakeroot devscripts equivs
       AFXDP:       ${{ matrix.afxdp }}
@@ -115,6 +113,16 @@ jobs:
     - name: checkout
       uses: actions/checkout at v2
+    - name: update PATH
+      run:  |
+        echo "$HOME/bin"        >> $GITHUB_PATH
+        echo "$HOME/.local/bin" >> $GITHUB_PATH
+    - name: set up python
+      uses: actions/setup-python at v2
+      with:
+        python-version: '3.x'
     - name: create ci signature file for the dpdk cache key
       if:   matrix.dpdk != '' || matrix.dpdk_shared != ''
       # This will collect most of DPDK related lines, so hash will be different
@@ -151,7 +159,7 @@ jobs:
       run:  ./.ci/linux-prepare.sh
     - name: build
-      run:  PATH="$PATH:$HOME/bin:$HOME/.local/bin" ./.ci/linux-build.sh
+      run:  ./.ci/linux-build.sh
     - name: upload deb packages
       if:   matrix.deb_package != ''
@@ -194,12 +202,20 @@ jobs:
     - name: checkout
       uses: actions/checkout at v2
+    - name: update PATH
+      run:  |
+        echo "$HOME/bin"        >> $GITHUB_PATH
+        echo "$HOME/.local/bin" >> $GITHUB_PATH
+    - name: set up python
+      uses: actions/setup-python at v2
+      with:
+        python-version: '3.x'
     - name: install dependencies
       run:  brew install automake libtool
     - name: prepare
       run:  ./.ci/osx-prepare.sh
     - name: build
-      run:  PATH="$PATH:$HOME/bin" ./.ci/osx-build.sh
+      run:  ./.ci/osx-build.sh
     - name: upload logs on failure
       if: failure()
       uses: actions/upload-artifact at v2

More information about the dev mailing list