[ovs-dev] [PATCH] python: Fixup python shebangs to python3

Flavio Leitner fbl at sysclose.org
Wed Aug 19 04:41:29 UTC 2020


On Tue, Aug 18, 2020 at 08:44:45PM -0700, Gregory Rose wrote:
> 
> On 8/18/2020 7:21 PM, Flavio Leitner wrote:
> > On Tue, Aug 18, 2020 at 01:29:33PM -0700, Gregory Rose wrote:
> > > On 8/18/2020 1:10 PM, Ilya Maximets wrote:
> > > > On 8/18/20 9:17 PM, Gregory Rose wrote:
> > > > > 
> > > > > On 8/18/2020 11:31 AM, Ilya Maximets wrote:
> > > > > > On 8/18/20 6:45 PM, Greg Rose wrote:
> > > > > > > Builds on RHEL 8.2 systems are failing due to this issue.
> > > > > > > 
> > > > > > > See [1] as to why this is necessary.
> > > > > > > 
> > > > > > > I used the following command to identify files that need this fix:
> > > > > > > find . -type f -executable | /usr/lib/rpm/redhat/brp-mangle-shebangs
> > > > > > > 
> > > > > > > I also updated the copyright notices as needed.
> > > > > > > 
> > > > > > > 1. https://fedoraproject.org/wiki/Changes/Make_ambiguous_python_shebangs_error
> > > > > > > 
> > > > > > > Signed-off-by: Greg Rose <gvrose8192 at gmail.com>
> > > > > > > ---
> > > > > > >     ipsec/ovs-monitor-ipsec.in                            |  4 ++--
> > > > > > >     ofproto/ipfix-gen-entities                            |  4 ++--
> > > > > > >     ovsdb/dot2pic                                         |  4 ++--
> > > > > > >     ovsdb/ovsdb-doc                                       |  4 ++--
> > > > > > >     ovsdb/ovsdb-dot.in                                    |  2 +-
> > > > > > >     ovsdb/ovsdb-idlc.in                                   |  2 +-
> > > > > > >     python/build/soutil.py                                |  4 ++--
> > > > > > >     tests/ovsdb-monitor-sort.py                           | 16 ++++++++++++++--
> > > > > > >     tests/sendpkt.py                                      |  4 ++--
> > > > > > >     tests/test-l7.py                                      |  4 ++--
> > > > > > >     tests/uuidfilt.py                                     | 18 +++++++++++++++++-
> > > > > > >     utilities/bugtool/ovs-bugtool.in                      |  4 ++--
> > > > > > >     utilities/ovs-check-dead-ifs.in                       |  2 +-
> > > > > > >     utilities/ovs-dev.py                                  |  4 ++--
> > > > > > >     utilities/ovs-dpctl-top.in                            |  4 ++--
> > > > > > >     utilities/ovs-l3ping.in                               |  2 +-
> > > > > > >     utilities/ovs-parse-backtrace.in                      |  4 ++--
> > > > > > >     utilities/ovs-pcap.in                                 |  4 ++--
> > > > > > >     utilities/ovs-pipegen.py                              |  4 ++--
> > > > > > >     utilities/ovs-tcpdump.in                              |  2 +-
> > > > > > >     utilities/ovs-tcpundump.in                            |  4 ++--
> > > > > > >     utilities/ovs-test.in                                 |  2 +-
> > > > > > >     utilities/ovs-vlan-test.in                            |  4 ++--
> > > > > > >     vtep/ovs-vtep.in                                      |  4 ++--
> > > > > > >     xenserver/etc_xapi.d_plugins_openvswitch-cfg-update   |  4 ++--
> > > > > > >     xenserver/opt_xensource_libexec_interface-reconfigure |  2 +-
> > > > > > >     xenserver/usr_share_openvswitch_scripts_ovs-xapi-sync |  4 ++--
> > > > > > >     27 files changed, 74 insertions(+), 46 deletions(-)
> > > > > > > 
> > > > > > > diff --git a/ipsec/ovs-monitor-ipsec.in b/ipsec/ovs-monitor-ipsec.in
> > > > > > > index 37e3703..32e8e90 100755
> > > > > > > --- a/ipsec/ovs-monitor-ipsec.in
> > > > > > > +++ b/ipsec/ovs-monitor-ipsec.in
> > > > > > > @@ -1,5 +1,5 @@
> > > > > > > -#! @PYTHON3@
> > > > > > > -# Copyright (c) 2017 Nicira, Inc.
> > > > > > > +#!/usr/bin/env python3
> > > > > > > +# Copyright (c) 2017, 2020 Nicira, Inc.
> > > > > > >     #
> > > > > > >     # Licensed under the Apache License, Version 2.0 (the "License");
> > > > > > >     # you may not use this file except in compliance with the License.
> > > > > > > diff --git a/ofproto/ipfix-gen-entities b/ofproto/ipfix-gen-entities
> > > > > > > index 0be7199..d5abe9c 100755
> > > > > > > --- a/ofproto/ipfix-gen-entities
> > > > > > > +++ b/ofproto/ipfix-gen-entities
> > > > > > > @@ -1,6 +1,6 @@
> > > > > > > -#! /usr/bin/env python
> > > > > > > +#!/usr/bin/env python3
> > > > > > >     #
> > > > > > > -# Copyright (C) 2012 Nicira, Inc.
> > > > > > > +# Copyright (C) 2012, 2020 Nicira, Inc.
> > > > > > >     #
> > > > > > >     # Copying and distribution of this file, with or without modification,
> > > > > > >     # are permitted in any medium without royalty provided the copyright
> > > > > > > diff --git a/ovsdb/dot2pic b/ovsdb/dot2pic
> > > > > > > index de67261..2f858e1 100755
> > > > > > > --- a/ovsdb/dot2pic
> > > > > > > +++ b/ovsdb/dot2pic
> > > > > > > @@ -1,6 +1,6 @@
> > > > > > > -#! /usr/bin/env python
> > > > > > > +#!/usr/bin/env python3
> > > > > > >     -# Copyright (c) 2009, 2010, 2011, 2013, 2017 Nicira, Inc.
> > > > > > > +# Copyright (c) 2009, 2010, 2011, 2013, 2017, 2020 Nicira, Inc.
> > > > > > >     #
> > > > > > >     # Licensed under the Apache License, Version 2.0 (the "License");
> > > > > > >     # you may not use this file except in compliance with the License.
> > > > > > > diff --git a/ovsdb/ovsdb-doc b/ovsdb/ovsdb-doc
> > > > > > > index 406c293..10d0c0c 100755
> > > > > > > --- a/ovsdb/ovsdb-doc
> > > > > > > +++ b/ovsdb/ovsdb-doc
> > > > > > > @@ -1,6 +1,6 @@
> > > > > > > -#! /usr/bin/python
> > > > > > > +#!/usr/bin/python3
> > > > > > >     -# Copyright (c) 2010, 2011, 2012, 2013, 2014, 2015 Nicira, Inc.
> > > > > > > +# Copyright (c) 2010, 2011, 2012, 2013, 2014, 2015, 2020 Nicira, Inc.
> > > > > > >     #
> > > > > > >     # Licensed under the Apache License, Version 2.0 (the "License");
> > > > > > >     # you may not use this file except in compliance with the License.
> > > > > > > diff --git a/ovsdb/ovsdb-dot.in b/ovsdb/ovsdb-dot.in
> > > > > > > index 41b986c..571601e 100755
> > > > > > > --- a/ovsdb/ovsdb-dot.in
> > > > > > > +++ b/ovsdb/ovsdb-dot.in
> > > > > > > @@ -1,4 +1,4 @@
> > > > > > > -#! @PYTHON3@
> > > > > > > +#!/usr/bin/env python3
> > > > > > 
> > > > > > Hmm.  This doesn't look right.
> > > > > > 
> > > > > > All the *.in files are used to generate actual scripts out of them
> > > > > > and '@PYTHON3@' is a placeholder for an actual python3 binary location
> > > > > > determind at the ./configure stage.  So, I don't think that we should
> > > > > > replace them.
> > > > > > 
> > > > > > Best regards, Ilya Maximets.
> > > > > > 
> > > > > 
> > > > > Hi Ilya,
> > > > > 
> > > > > Thanks for the review.
> > > > > 
> > > > > The shebang check occurs before the expansion of @PYTHON3@ and causes
> > > > > build failures.
> > > > 
> > > > This is strange.  I just tried to build rpms on rhel 8.2 and on fedora 31.
> > > > On both systems shebang check was triggered after the actual build and
> > > > only on resulted executable files (without .in).  I think, something
> > > > goes terribly wrong if shebang check invoked before the build and on
> > > > files that should not be part of a binary distribution.
> > > > 
> > > > Could you provide more information on how to reproduce?
> > > > 
> > > > > We have no control over when the shebang check runs
> > > > > because it is part of the RPM build environment supplied by Red Hat.
> > > > > Moreover, the shebang check also substitutes #!/usr/bin/env python3
> > > > > with someting like '/usr/libexec/platform-python'.
> > > > 
> > > > I see messages like this:
> > > > mangling shebang in /usr/bin/ovs-pcap from /usr/bin/python3 to #!/usr/libexec/platform-python
> > > > 
> > > > But these are not errors nor even warnings, so should not affect the build.
> > > > In fact /usr/bin/python3 is a symlink to /usr/libexec/platform-python.
> > > 
> > > Yes - but @PYTHON3@ is an error.  That's what I have to find some way
> > > to fix.
> > 
> > I agree with Ilya here that this is strange.
> > 
> > I just built ovs RPMs on RHEL-8.2 and it worked just fine,
> > same on Fedora 32.
> > 
> > Those below are the only message related to shebangs and as
> > you can see no python issues:
> > [...]
> > Bytecompiling .py files below /root/ovs/rpm/rpmbuild/BUILDROOT/openvswitch-2.14.90-1.el8.x86_64/usr/lib/python3.6 using /usr/libexec/platform-python
> > + /usr/lib/rpm/brp-python-hardlink
> > + PYTHON3=/usr/libexec/platform-python
> > + /usr/lib/rpm/redhat/brp-mangle-shebangs
> > *** WARNING: ./usr/lib64/libopenvswitch.la is executable but has empty or no shebang, removing executable bit
> > *** WARNING: ./usr/lib64/libsflow.la is executable but has empty or no shebang, removing executable bit
> > *** WARNING: ./usr/lib64/libofproto.la is executable but has empty or no shebang, removing executable bit
> > *** WARNING: ./usr/lib64/libovsdb.la is executable but has empty or no shebang, removing executable bit
> > *** WARNING: ./usr/lib64/libvtep.la is executable but has empty or no shebang, removing executable bit
> > *** WARNING: ./etc/bash_completion.d/ovs-appctl-bashcomp.bash is executable but has empty or no shebang, removing executable bit
> > *** WARNING: ./etc/bash_completion.d/ovs-vsctl-bashcomp.bash is executable but has empty or no shebang, removing executable bit
> > Executing(%check): /bin/sh -e /var/tmp/rpm-tmp.plv5lw
> > [...]
> > 
> > 
> > fbl
> > 
> 
> Hi Flavio,
> 
> This is truly curious because here's what we're getting:
> 
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
> ambiguous python shebang in
> /usr/src/openvswitch-2.13.1.rhel82.39060899/tests/sendpkt.py: #!/usr/bin/env
> python. Change it to python3 (or python2) explicitly.
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
> ambiguous python shebang in
> /usr/src/openvswitch-2.13.1.rhel82.39060899/tests/test-l7.py: #!/usr/bin/env
> python. Change it to python3 (or python2) explicitly.
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) ***
> WARNING: ./usr/src/openvswitch-2.13.1.rhel82.39060899/tests/checkpatch.at is
> executable but has empty or no shebang, removing executable bit
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
> ambiguous python shebang in
> /usr/src/openvswitch-2.13.1.rhel82.39060899/tests/ovsdb-monitor-sort.py:
> #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
> ambiguous python shebang in
> /usr/src/openvswitch-2.13.1.rhel82.39060899/tests/uuidfilt.py:
> #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) mangling
> shebang in /usr/src/openvswitch-2.13.1.rhel82.39060899/tests/flowgen.py from
> /usr/bin/env python3 to #!/usr/libexec/platform-python
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
> ambiguous python shebang in
> /usr/src/openvswitch-2.13.1.rhel82.39060899/ofproto/ipfix-gen-entities:
> #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
> ./usr/src/openvswitch-2.13.1.rhel82.39060899/ovsdb/ovsdb-dot.in has shebang
> which doesn't start with '/' (@PYTHON3@)
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
> ./usr/src/openvswitch-2.13.1.rhel82.39060899/ovsdb/ovsdb-idlc.in has shebang
> which doesn't start with '/' (@PYTHON3@)
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
> ambiguous python shebang in
> /usr/src/openvswitch-2.13.1.rhel82.39060899/ovsdb/ovsdb-doc:
> #!/usr/bin/python. Change it to python3 (or python2) explicitly.
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
> ambiguous python shebang in
> /usr/src/openvswitch-2.13.1.rhel82.39060899/ovsdb/dot2pic: #!/usr/bin/env
> python. Change it to python3 (or python2) explicitly.
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
> ./usr/src/openvswitch-2.13.1.rhel82.39060899/ipsec/ovs-monitor-ipsec.in has
> shebang which doesn't start with '/' (@PYTHON3@)
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
> ./usr/src/openvswitch-2.13.1.rhel82.39060899/utilities/ovs-vlan-test.in has
> shebang which doesn't start with '/' (@PYTHON3@)
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
> ./usr/src/openvswitch-2.13.1.rhel82.39060899/utilities/ovs-tcpdump.in has
> shebang which doesn't start with '/' (@PYTHON3@)
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
> ./usr/src/openvswitch-2.13.1.rhel82.39060899/utilities/ovs-tcpundump.in has
> shebang which doesn't start with '/' (@PYTHON3@)
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
> ./usr/src/openvswitch-2.13.1.rhel82.39060899/utilities/bugtool/ovs-bugtool.in
> has shebang which doesn't start with '/' (@PYTHON3@)
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
> ./usr/src/openvswitch-2.13.1.rhel82.39060899/utilities/ovs-pcap.in has
> shebang which doesn't start with '/' (@PYTHON3@)
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
> ./usr/src/openvswitch-2.13.1.rhel82.39060899/utilities/ovs-check-dead-ifs.in
> has shebang which doesn't start with '/' (@PYTHON3@)
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) mangling
> shebang in
> /usr/src/openvswitch-2.13.1.rhel82.39060899/utilities/checkpatch.py from
> /usr/bin/env python3 to #!/usr/libexec/platform-python
> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
> ./usr/src/openvswitch-2.13.1.rhel82.39060899/utilities/ovs-parse-backtrace.in
> has shebang which doesn't start with '/' (@PYTHON3@)
> 
> 
> Any idea what we could do to fix this?  Maybe it's a chroot thing?

Well, that seems to be a different spec since it's packaging files
that OVS normally does not install.

Are you packaging the OVS sources?

> Although I have no idea why it would have anything to do with chroot,
> but frankly I'm grasping at straws here.

Right, but there is no sense in having ovs-check-dead-ifs.in packaged
with execution permission. So, this is definitely a packaging issue.

Assuming that you want to package the sources, then you need to tell
RPM to ignore them, because they are not normal installable files.

The following line tells RPM to ignore files under your sources
directory:
%global __brp_mangle_shebangs_exclude_from /usr/src/openvswitch-2.13.1.rhel82.39060899

Most probably you need to replace those versions with RPM macros.

fbl

> And I want to say again, on a RHEL 8.2 system that I have, executing
> this, 'find . -type f -executable |
> /usr/lib/rpm/redhat/brp-mangle-shebangs', produces many errors.  So maybe my
> RHEL 8.2 system
> is different in some way?
> 
> Thanks,
> 
> - Greg
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev

-- 
fbl


More information about the dev mailing list