[ovs-dev] [PATCH] python: Fixup python shebangs to python3

Gregory Rose gvrose8192 at gmail.com
Wed Aug 19 15:00:10 UTC 2020



On 8/18/2020 9:41 PM, Flavio Leitner wrote:
> On Tue, Aug 18, 2020 at 08:44:45PM -0700, Gregory Rose wrote:
>>
>> On 8/18/2020 7:21 PM, Flavio Leitner wrote:
>>> On Tue, Aug 18, 2020 at 01:29:33PM -0700, Gregory Rose wrote:
>>>> On 8/18/2020 1:10 PM, Ilya Maximets wrote:
>>>>> On 8/18/20 9:17 PM, Gregory Rose wrote:
>>>>>>
>>>>>> On 8/18/2020 11:31 AM, Ilya Maximets wrote:
>>>>>>> On 8/18/20 6:45 PM, Greg Rose wrote:
>>>>>>>> Builds on RHEL 8.2 systems are failing due to this issue.
>>>>>>>>
>>>>>>>> See [1] as to why this is necessary.
>>>>>>>>
>>>>>>>> I used the following command to identify files that need this fix:
>>>>>>>> find . -type f -executable | /usr/lib/rpm/redhat/brp-mangle-shebangs
>>>>>>>>
>>>>>>>> I also updated the copyright notices as needed.
>>>>>>>>
>>>>>>>> 1. https://fedoraproject.org/wiki/Changes/Make_ambiguous_python_shebangs_error
>>>>>>>>
>>>>>>>> Signed-off-by: Greg Rose <gvrose8192 at gmail.com>
>>>>>>>> ---
>>>>>>>>      ipsec/ovs-monitor-ipsec.in                            |  4 ++--
>>>>>>>>      ofproto/ipfix-gen-entities                            |  4 ++--
>>>>>>>>      ovsdb/dot2pic                                         |  4 ++--
>>>>>>>>      ovsdb/ovsdb-doc                                       |  4 ++--
>>>>>>>>      ovsdb/ovsdb-dot.in                                    |  2 +-
>>>>>>>>      ovsdb/ovsdb-idlc.in                                   |  2 +-
>>>>>>>>      python/build/soutil.py                                |  4 ++--
>>>>>>>>      tests/ovsdb-monitor-sort.py                           | 16 ++++++++++++++--
>>>>>>>>      tests/sendpkt.py                                      |  4 ++--
>>>>>>>>      tests/test-l7.py                                      |  4 ++--
>>>>>>>>      tests/uuidfilt.py                                     | 18 +++++++++++++++++-
>>>>>>>>      utilities/bugtool/ovs-bugtool.in                      |  4 ++--
>>>>>>>>      utilities/ovs-check-dead-ifs.in                       |  2 +-
>>>>>>>>      utilities/ovs-dev.py                                  |  4 ++--
>>>>>>>>      utilities/ovs-dpctl-top.in                            |  4 ++--
>>>>>>>>      utilities/ovs-l3ping.in                               |  2 +-
>>>>>>>>      utilities/ovs-parse-backtrace.in                      |  4 ++--
>>>>>>>>      utilities/ovs-pcap.in                                 |  4 ++--
>>>>>>>>      utilities/ovs-pipegen.py                              |  4 ++--
>>>>>>>>      utilities/ovs-tcpdump.in                              |  2 +-
>>>>>>>>      utilities/ovs-tcpundump.in                            |  4 ++--
>>>>>>>>      utilities/ovs-test.in                                 |  2 +-
>>>>>>>>      utilities/ovs-vlan-test.in                            |  4 ++--
>>>>>>>>      vtep/ovs-vtep.in                                      |  4 ++--
>>>>>>>>      xenserver/etc_xapi.d_plugins_openvswitch-cfg-update   |  4 ++--
>>>>>>>>      xenserver/opt_xensource_libexec_interface-reconfigure |  2 +-
>>>>>>>>      xenserver/usr_share_openvswitch_scripts_ovs-xapi-sync |  4 ++--
>>>>>>>>      27 files changed, 74 insertions(+), 46 deletions(-)
>>>>>>>>
>>>>>>>> diff --git a/ipsec/ovs-monitor-ipsec.in b/ipsec/ovs-monitor-ipsec.in
>>>>>>>> index 37e3703..32e8e90 100755
>>>>>>>> --- a/ipsec/ovs-monitor-ipsec.in
>>>>>>>> +++ b/ipsec/ovs-monitor-ipsec.in
>>>>>>>> @@ -1,5 +1,5 @@
>>>>>>>> -#! @PYTHON3@
>>>>>>>> -# Copyright (c) 2017 Nicira, Inc.
>>>>>>>> +#!/usr/bin/env python3
>>>>>>>> +# Copyright (c) 2017, 2020 Nicira, Inc.
>>>>>>>>      #
>>>>>>>>      # Licensed under the Apache License, Version 2.0 (the "License");
>>>>>>>>      # you may not use this file except in compliance with the License.
>>>>>>>> diff --git a/ofproto/ipfix-gen-entities b/ofproto/ipfix-gen-entities
>>>>>>>> index 0be7199..d5abe9c 100755
>>>>>>>> --- a/ofproto/ipfix-gen-entities
>>>>>>>> +++ b/ofproto/ipfix-gen-entities
>>>>>>>> @@ -1,6 +1,6 @@
>>>>>>>> -#! /usr/bin/env python
>>>>>>>> +#!/usr/bin/env python3
>>>>>>>>      #
>>>>>>>> -# Copyright (C) 2012 Nicira, Inc.
>>>>>>>> +# Copyright (C) 2012, 2020 Nicira, Inc.
>>>>>>>>      #
>>>>>>>>      # Copying and distribution of this file, with or without modification,
>>>>>>>>      # are permitted in any medium without royalty provided the copyright
>>>>>>>> diff --git a/ovsdb/dot2pic b/ovsdb/dot2pic
>>>>>>>> index de67261..2f858e1 100755
>>>>>>>> --- a/ovsdb/dot2pic
>>>>>>>> +++ b/ovsdb/dot2pic
>>>>>>>> @@ -1,6 +1,6 @@
>>>>>>>> -#! /usr/bin/env python
>>>>>>>> +#!/usr/bin/env python3
>>>>>>>>      -# Copyright (c) 2009, 2010, 2011, 2013, 2017 Nicira, Inc.
>>>>>>>> +# Copyright (c) 2009, 2010, 2011, 2013, 2017, 2020 Nicira, Inc.
>>>>>>>>      #
>>>>>>>>      # Licensed under the Apache License, Version 2.0 (the "License");
>>>>>>>>      # you may not use this file except in compliance with the License.
>>>>>>>> diff --git a/ovsdb/ovsdb-doc b/ovsdb/ovsdb-doc
>>>>>>>> index 406c293..10d0c0c 100755
>>>>>>>> --- a/ovsdb/ovsdb-doc
>>>>>>>> +++ b/ovsdb/ovsdb-doc
>>>>>>>> @@ -1,6 +1,6 @@
>>>>>>>> -#! /usr/bin/python
>>>>>>>> +#!/usr/bin/python3
>>>>>>>>      -# Copyright (c) 2010, 2011, 2012, 2013, 2014, 2015 Nicira, Inc.
>>>>>>>> +# Copyright (c) 2010, 2011, 2012, 2013, 2014, 2015, 2020 Nicira, Inc.
>>>>>>>>      #
>>>>>>>>      # Licensed under the Apache License, Version 2.0 (the "License");
>>>>>>>>      # you may not use this file except in compliance with the License.
>>>>>>>> diff --git a/ovsdb/ovsdb-dot.in b/ovsdb/ovsdb-dot.in
>>>>>>>> index 41b986c..571601e 100755
>>>>>>>> --- a/ovsdb/ovsdb-dot.in
>>>>>>>> +++ b/ovsdb/ovsdb-dot.in
>>>>>>>> @@ -1,4 +1,4 @@
>>>>>>>> -#! @PYTHON3@
>>>>>>>> +#!/usr/bin/env python3
>>>>>>>
>>>>>>> Hmm.  This doesn't look right.
>>>>>>>
>>>>>>> All the *.in files are used to generate actual scripts out of them
>>>>>>> and '@PYTHON3@' is a placeholder for an actual python3 binary location
>>>>>>> determind at the ./configure stage.  So, I don't think that we should
>>>>>>> replace them.
>>>>>>>
>>>>>>> Best regards, Ilya Maximets.
>>>>>>>
>>>>>>
>>>>>> Hi Ilya,
>>>>>>
>>>>>> Thanks for the review.
>>>>>>
>>>>>> The shebang check occurs before the expansion of @PYTHON3@ and causes
>>>>>> build failures.
>>>>>
>>>>> This is strange.  I just tried to build rpms on rhel 8.2 and on fedora 31.
>>>>> On both systems shebang check was triggered after the actual build and
>>>>> only on resulted executable files (without .in).  I think, something
>>>>> goes terribly wrong if shebang check invoked before the build and on
>>>>> files that should not be part of a binary distribution.
>>>>>
>>>>> Could you provide more information on how to reproduce?
>>>>>
>>>>>> We have no control over when the shebang check runs
>>>>>> because it is part of the RPM build environment supplied by Red Hat.
>>>>>> Moreover, the shebang check also substitutes #!/usr/bin/env python3
>>>>>> with someting like '/usr/libexec/platform-python'.
>>>>>
>>>>> I see messages like this:
>>>>> mangling shebang in /usr/bin/ovs-pcap from /usr/bin/python3 to #!/usr/libexec/platform-python
>>>>>
>>>>> But these are not errors nor even warnings, so should not affect the build.
>>>>> In fact /usr/bin/python3 is a symlink to /usr/libexec/platform-python.
>>>>
>>>> Yes - but @PYTHON3@ is an error.  That's what I have to find some way
>>>> to fix.
>>>
>>> I agree with Ilya here that this is strange.
>>>
>>> I just built ovs RPMs on RHEL-8.2 and it worked just fine,
>>> same on Fedora 32.
>>>
>>> Those below are the only message related to shebangs and as
>>> you can see no python issues:
>>> [...]
>>> Bytecompiling .py files below /root/ovs/rpm/rpmbuild/BUILDROOT/openvswitch-2.14.90-1.el8.x86_64/usr/lib/python3.6 using /usr/libexec/platform-python
>>> + /usr/lib/rpm/brp-python-hardlink
>>> + PYTHON3=/usr/libexec/platform-python
>>> + /usr/lib/rpm/redhat/brp-mangle-shebangs
>>> *** WARNING: ./usr/lib64/libopenvswitch.la is executable but has empty or no shebang, removing executable bit
>>> *** WARNING: ./usr/lib64/libsflow.la is executable but has empty or no shebang, removing executable bit
>>> *** WARNING: ./usr/lib64/libofproto.la is executable but has empty or no shebang, removing executable bit
>>> *** WARNING: ./usr/lib64/libovsdb.la is executable but has empty or no shebang, removing executable bit
>>> *** WARNING: ./usr/lib64/libvtep.la is executable but has empty or no shebang, removing executable bit
>>> *** WARNING: ./etc/bash_completion.d/ovs-appctl-bashcomp.bash is executable but has empty or no shebang, removing executable bit
>>> *** WARNING: ./etc/bash_completion.d/ovs-vsctl-bashcomp.bash is executable but has empty or no shebang, removing executable bit
>>> Executing(%check): /bin/sh -e /var/tmp/rpm-tmp.plv5lw
>>> [...]
>>>
>>>
>>> fbl
>>>
>>
>> Hi Flavio,
>>
>> This is truly curious because here's what we're getting:
>>
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
>> ambiguous python shebang in
>> /usr/src/openvswitch-2.13.1.rhel82.39060899/tests/sendpkt.py: #!/usr/bin/env
>> python. Change it to python3 (or python2) explicitly.
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
>> ambiguous python shebang in
>> /usr/src/openvswitch-2.13.1.rhel82.39060899/tests/test-l7.py: #!/usr/bin/env
>> python. Change it to python3 (or python2) explicitly.
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) ***
>> WARNING: ./usr/src/openvswitch-2.13.1.rhel82.39060899/tests/checkpatch.at is
>> executable but has empty or no shebang, removing executable bit
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
>> ambiguous python shebang in
>> /usr/src/openvswitch-2.13.1.rhel82.39060899/tests/ovsdb-monitor-sort.py:
>> #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
>> ambiguous python shebang in
>> /usr/src/openvswitch-2.13.1.rhel82.39060899/tests/uuidfilt.py:
>> #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) mangling
>> shebang in /usr/src/openvswitch-2.13.1.rhel82.39060899/tests/flowgen.py from
>> /usr/bin/env python3 to #!/usr/libexec/platform-python
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
>> ambiguous python shebang in
>> /usr/src/openvswitch-2.13.1.rhel82.39060899/ofproto/ipfix-gen-entities:
>> #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
>> ./usr/src/openvswitch-2.13.1.rhel82.39060899/ovsdb/ovsdb-dot.in has shebang
>> which doesn't start with '/' (@PYTHON3@)
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
>> ./usr/src/openvswitch-2.13.1.rhel82.39060899/ovsdb/ovsdb-idlc.in has shebang
>> which doesn't start with '/' (@PYTHON3@)
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
>> ambiguous python shebang in
>> /usr/src/openvswitch-2.13.1.rhel82.39060899/ovsdb/ovsdb-doc:
>> #!/usr/bin/python. Change it to python3 (or python2) explicitly.
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
>> ambiguous python shebang in
>> /usr/src/openvswitch-2.13.1.rhel82.39060899/ovsdb/dot2pic: #!/usr/bin/env
>> python. Change it to python3 (or python2) explicitly.
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
>> ./usr/src/openvswitch-2.13.1.rhel82.39060899/ipsec/ovs-monitor-ipsec.in has
>> shebang which doesn't start with '/' (@PYTHON3@)
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
>> ./usr/src/openvswitch-2.13.1.rhel82.39060899/utilities/ovs-vlan-test.in has
>> shebang which doesn't start with '/' (@PYTHON3@)
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
>> ./usr/src/openvswitch-2.13.1.rhel82.39060899/utilities/ovs-tcpdump.in has
>> shebang which doesn't start with '/' (@PYTHON3@)
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
>> ./usr/src/openvswitch-2.13.1.rhel82.39060899/utilities/ovs-tcpundump.in has
>> shebang which doesn't start with '/' (@PYTHON3@)
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
>> ./usr/src/openvswitch-2.13.1.rhel82.39060899/utilities/bugtool/ovs-bugtool.in
>> has shebang which doesn't start with '/' (@PYTHON3@)
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
>> ./usr/src/openvswitch-2.13.1.rhel82.39060899/utilities/ovs-pcap.in has
>> shebang which doesn't start with '/' (@PYTHON3@)
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
>> ./usr/src/openvswitch-2.13.1.rhel82.39060899/utilities/ovs-check-dead-ifs.in
>> has shebang which doesn't start with '/' (@PYTHON3@)
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) mangling
>> shebang in
>> /usr/src/openvswitch-2.13.1.rhel82.39060899/utilities/checkpatch.py from
>> /usr/bin/env python3 to #!/usr/libexec/platform-python
>> 2020-08-13 23:12:44 | [23:12:44     INFO] (chroot: rhel82_x86_64) *** ERROR:
>> ./usr/src/openvswitch-2.13.1.rhel82.39060899/utilities/ovs-parse-backtrace.in
>> has shebang which doesn't start with '/' (@PYTHON3@)
>>
>>
>> Any idea what we could do to fix this?  Maybe it's a chroot thing?
> 
> Well, that seems to be a different spec since it's packaging files
> that OVS normally does not install.
> 
> Are you packaging the OVS sources?
> 
>> Although I have no idea why it would have anything to do with chroot,
>> but frankly I'm grasping at straws here.
> 
> Right, but there is no sense in having ovs-check-dead-ifs.in packaged
> with execution permission. So, this is definitely a packaging issue.
> 
> Assuming that you want to package the sources, then you need to tell
> RPM to ignore them, because they are not normal installable files.
> 
> The following line tells RPM to ignore files under your sources
> directory:
> %global __brp_mangle_shebangs_exclude_from /usr/src/openvswitch-2.13.1.rhel82.39060899
> 
> Most probably you need to replace those versions with RPM macros.

Let me work with our build team to see if that will fix the problem.
Thanks for the suggestion.

- Greg


> 
> fbl
> 
>> And I want to say again, on a RHEL 8.2 system that I have, executing
>> this, 'find . -type f -executable |
>> /usr/lib/rpm/redhat/brp-mangle-shebangs', produces many errors.  So maybe my
>> RHEL 8.2 system
>> is different in some way?
>>
>> Thanks,
>>
>> - Greg
>> _______________________________________________
>> dev mailing list
>> dev at openvswitch.org
>> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> 


More information about the dev mailing list