rpms/python-rpm-macros
3
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

brp-fix-pyc-reproducibility: automatically call on whole of /usr

Browse source 
Architecture-specific pyc files cause two problems:
- noarch packages may cause noarch packages to differ between different
  architectures in the case where we build an archful package which has some
  noarch subpackages,
- package rebuilds report a difference when build reproducibility is checked
  and the build happened on a different architecture.

Both problems can be resolved by using the existing %py_reproducible_pyc_path
macro, but this requires maintainers to opt-in. Let's just extend this operation
to all files under /usr, to make life easier for maintainers and for folks
working on reproducible builds.

"find /usr/ -name '*.pyc'" reveals that .pyc files are installed in quite a few
locations, not just the python site and arch directories, so just apply the
fixer to the whole /usr.

%py_fix_reproducibility can be unset to opt out.

%py_reproducible_pyc_path retains its function. We can probably drop all or
almost all of its uses in Fedora, but it might be useful for people doing some
special things.

Fixes https://pagure.io/fedora-reproducible-builds/project/issue/12,
https://bugzilla.redhat.com/show_bug.cgi?id=2266767.

To deal with bootstrapping, the script does nothing if marshalparser is
not installed. python-srpm-macros Require marshalparser, but not when built
with bootstrap.

The call to marshalparser takes a bit of time. I ran it on all .pyc files on my
system and that took a few minutes. But for each single package, the time is
usually < 1 s. There are some exceptions, for example libvirt.cpython-312.*.pyc
take ~15 s each. But I don't think this really matters: the files come from a
huge package which takes a long time to compile, and adding a few dozen seconds
at the end doesn't change much. I expect that marshalparser itself might need
some optimizations. We can do that at some later point if the slowdowns become
noticeable.
This commit is contained in:
Zbigniew Jędrzejewski-Szmek 2024-03-01 00:41:41 +01:00
commit c41028c137
3 changed files with 29 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

6
macros.python-srpm
View file

@ -77,6 +77,9 @@
## https://fedoraproject.org/wiki/Changes/ReproducibleBuildsClampMtimes#Python_bytecode
%__env_unset_source_date_epoch_if_not_clamp_mtime %[0%{?clamp_mtime_to_source_date_epoch} == 0 ? "env -u SOURCE_DATE_EPOCH" : "env"]
## Automatically apply __brp_fix_pyc_reproducibility
%py_fix_reproducibility 1
## The individual BRP scripts
%__brp_python_bytecompile %{__env_unset_source_date_epoch_if_not_clamp_mtime} %{_rpmconfigdir}/redhat/brp-python-bytecompile "" "%{?_python_bytecompile_errors_terminate_build}" "%{?_python_bytecompile_extra}" "%{?_smp_build_ncpus:-j%{_smp_build_ncpus}}"
%__brp_fix_pyc_reproducibility %{_rpmconfigdir}/redhat/brp-fix-pyc-reproducibility
@ -89,7 +92,8 @@
# 3. brp-python-hardlink de-duplicates identical pyc files
%__os_install_post_python \
%{?py_auto_byte_compile:%{?__brp_python_bytecompile}} \
%{?py_reproducible_pyc_path:%{?__brp_fix_pyc_reproducibility} "%{py_reproducible_pyc_path}"} \
%{?py_fix_reproducibility:%{__brp_fix_pyc_reproducibility} -a "%{buildroot}%{_prefix}"} \
%{?py_reproducible_pyc_path:%{__brp_fix_pyc_reproducibility} "%{py_reproducible_pyc_path}"} \
%{?__brp_python_hardlink} \
%{nil}