Compare commits

...
Sign in to create a new pull request.

72 commits

Author SHA1 Message Date
Lumir Balhar
a83a4480c0 Use reinstall and distrosync instead of downgrade to cover more use-cases 2026-07-13 19:35:24 +02:00
Lumir Balhar
3bc0535c27 Make sure the downgrade operation is always reverted 2026-07-07 08:32:41 +02:00
Lumir Balhar
533e84657e Include libpython .so files into scanned files 2026-07-01 14:43:13 +02:00
Lumir Balhar
b4af9ca880 Add new test that monitors symbols required from dynamic compiled libs 2026-07-01 12:17:03 +02:00
Miro Hrončok
dfd41a9acf When CPython tests fail, exit with 1 to signal failure (rather than error) 2026-05-13 17:02:36 +02:00
Miro Hrončok
018063867c INSTALL_OR_SKIP: Support python3.X-freethreading-devel installation
It assumed python3.Xt-devel package name and that is not correct,
this way, we go for files and we don't care what package they are in.
All our Pythons have the -config file, but not PyPys,
so we keep that installation optional (unchecked) and we keep the original -devel installation as well.
2025-10-24 13:01:55 +02:00
Miro Hrončok
5a345b59d6 Adjust the test cases here for rawhide 2025-07-10 13:16:45 +02:00
Miro Hrončok
223296d711 Allow skipping virtual environment tests
This is useful for tox end ensurepip tests for pypy 2.7.
2025-07-10 13:16:45 +02:00
Miro Hrončok
085416c781 Make venv.sh work with versions like 3.13t 2025-05-13 21:34:48 +02:00
Miro Hrončok
5c46fa4679 Inject $TOX_REQUIRES to tox config
E.g. use TOX_REQUIRES="virtualenv<20.22.0" to test tox provision for old Pythons.
2024-11-11 16:59:01 +01:00
Miro Hrončok
5ffff08833 Do not skip a flag without -O, if the flag is checked explicitly 2024-09-11 22:27:17 +02:00
Miro Hrončok
1ad7f0cc25 Skip Python 2.7 tests on rawhide 2024-09-11 20:29:46 +02:00
Miro Hrončok
a0a96e2721 Add --timeout to selftest/parallel.sh 2024-09-11 19:57:24 +02:00
Miro Hrončok
800608da2e Allow to check explicit flags in assertflags.py 2024-08-01 15:00:27 +02:00
Miro Hrončok
b8d9e52c94 Update the list of self-tested Pythons
python3.7 was retired in f40+
python3.5 is retired everywhere

I intentionally kept python3.4 so we have at least one test that is skipped.
2024-04-04 21:36:34 +02:00
Miro Hrončok
6f4fdf29c4 Allow testing virtualenv with --download 2024-04-04 17:50:39 +02:00
Miro Hrončok
548de9fc09 Run the CI tests of this repo with pypy3.9 instead of retired pypy3.8 2023-04-28 16:32:49 +02:00
Miro Hrončok
3ff6bdd8ba Don't execute setup.py directly, always use pip
Since Python 3.12, virtual environments don't have setuptools,
instead, pip will install it to an isolated build environment.
2023-04-28 13:28:29 +02:00
Tomáš Hrnčiar
b9eebf1c79 Enable Cython test for Python 3.12, using the git master 2023-02-10 07:10:29 +01:00
Lukas Zachar
dc4e662e56 parallel might run test_dtrace
Which obviously needs dtrace to be installed.
Added recommend as it is better to fail the test not whole plan if it is not possible to
install systemtap-sdt-devel by its /usr/bin/dtrace provide
2023-02-09 10:14:06 +01:00
Lukas Zachar
6be8056917 smoke/venv needs gcc
Which is not installed by default
2023-01-31 08:21:34 +01:00
Lukas Zachar
9a18358399 Extend duration of selftest/parallel
E.g. component python3.11 (from C9S) runs whole test suite which can't
finish within the default value of 5 minutes.
2023-01-27 13:22:52 +01:00
Lukas Zachar
6f4c0d0815 Make tmt lint pass for /smoke/venv 2023-01-17 20:33:40 +01:00
Lukas Zachar
f022884999 Convert selftest into tmt 2023-01-17 20:33:40 +01:00
Lumir Balhar
b9dc63ad15 Do not use Cython from master branch for Py 3.11 2023-01-10 13:37:41 +01:00
Miro Hrončok
51c24d51d1 dnf install cannot be run as non-root 2022-11-26 22:42:49 +01:00
Miro Hrončok
4c3ed6ec49 Try installing the devel package when INSTALL_OR_SKIP is set 2022-11-25 15:52:43 +01:00
Miro Hrončok
df6c0cef6e Don't set SETUPTOOLS_USE_DISTUTILS=stdlib when distutils does not exist
Python 3.12.0a2 removed the standard library distutils module,
so we can no longer use it.
2022-11-16 00:45:37 +01:00
Miro Hrončok
18de9ca2b3 Run tests as regular user
For this to work, we run the smoke test from a temporary directory,
on the CI /var/str/smoke is not writable by non-root.
2022-06-09 12:46:18 +02:00
Tomáš Hrnčiar
fd113d4cf8 Add gcc-c++ dependency to run Python 3.11 test 2022-05-09 12:27:26 +02:00
Karolina Surma
db335e1e44 Explicitly declare the usage of distutils from Python stdlib for tests
setuptools 60+ bundles distutils which then have an impact on the test
environment, causing test_with_pip to fail.
2022-03-16 17:52:34 +01:00
Miro Hrončok
19a49e8637 Drop no longer relevant conditionals for Python 2.6 2022-03-08 01:07:00 +01:00
Miro Hrončok
362f438cc6 Instead of checking for Jython, make it possible to opt-out from the Cython test
- we don't have Jython any more and even if we had, we could explicitly opt out from it
 - we sometimes need to opt out from Cython test in the development versions of Python
2022-03-08 01:05:03 +01:00
Miro Hrončok
bc948d6e50 Fixup the pypy3 test example 2022-03-08 01:03:37 +01:00
Miro Hrončok
c2466c2de5 Undo Python 3.11 py workarounds, py 1.11.0 is out with the fix 2021-11-04 18:26:12 +01:00
Miro Hrončok
beb705f0d0 Enable Cython test for Python 3.11, using the git master 2021-10-29 16:47:42 +02:00
Miro Hrončok
1adc399e3d Add Python 3.11 workarounds 2021-10-12 12:10:15 +02:00
Miro Hrončok
2fc6fc00bc Make python3.5 optional 2021-08-10 15:23:00 +02:00
Miro Hrončok
853a1f50a3 Correctly determine toxenv for versioned PyPy 2021-08-10 14:55:20 +02:00
Tomas Hrnciar
98ad8d0232 For debug build, assert -O0 instead of -Og
Python change: https://src.fedoraproject.org/rpms/python3.9/c/1d2d05b30506c5758da94f1cd3860f3d58a87db0
Rationale: https://bugzilla.redhat.com/show_bug.cgi?id=1818857
2021-05-05 10:52:42 +02:00
Miro Hrončok
1b6575d5a5 Avoid variable names with negative conotations 2020-11-05 13:32:26 +01:00
Miro Hrončok
59f2079f68 Support INSTALL_OR_SKIP for marshalparser test 2020-10-13 22:09:11 +02:00
Miro Hrončok
6cc91d52a7 No longer install Cython from master on Python 3.9
It also is not (yet) necessary on Python 3.10.
2020-10-12 11:18:00 +02:00
Lumir Balhar
86842acd1b Add marshalparser to Python tests
Marshalparser helps us to uniform byte-compiled Python files
so it has to be able to also parse pyc files from
the main python packages.
Moreover, it has its own database of magic numbers so this test
checks that the parser is compatible with the latest python before
we ship it.
2020-07-30 07:50:39 +02:00
Lumir Balhar
b4fd40e6ca Test virtualenv with pip for Py 2.7 (bundled wheels in ensurepip) and Py 3.9 2020-06-23 20:38:51 +02:00
Lumir Balhar
261dfe65e1 Add method for virtualenv with --seeder pip
--seeder pip uses pip (from wheel) to install seed packages (pip, setuptools
and wheel) from our own system wheels.
2020-06-23 13:35:27 +02:00
Miro Hrončok
c26b64823b Make it possible to run Python 3.4 tests optionally 2020-06-05 11:42:43 +02:00
Miro Hrončok
0ec11f1f64 Require Python packages with dots 2020-06-03 15:59:47 +02:00
Miro Hrončok
450364614f Also test plain ensurepip in smoke tests 2020-04-29 01:17:58 +02:00
Miro Hrončok
6a9ffe6cb2 Have a dynamic Python version by default if possible 2020-04-20 15:32:38 +02:00
Miro Hrončok
562c77a483 Smoke test self without Python 3.4
Python 3.4 is going to be removed in Fedora 33:

https://fedoraproject.org/wiki/Changes/RetirePython34
2020-02-12 15:14:50 +00:00
Miro Hrončok
38d9a441ce Test with Python 3.9 as well 2020-02-07 18:39:54 +01:00
Miro Hrončok
74239cac5d Assert that tox uses the appropriate Python version 2020-02-07 18:39:54 +01:00
Miro Hrončok
606e77a1ba Expand the OPENSSL_CONF=/non-existing-file comment
It's not just the minimum version now.
2019-12-02 18:24:15 +01:00
Miro Hrončok
754dc5a573 Python 3.9: Fetch Cython master 2019-11-26 01:10:14 +01:00
Miro Hrončok
96fbcc5c42 Test that the debug build has the proper -O flags set
Fixes https://src.fedoraproject.org/tests/python/issue/12
2019-10-21 15:17:34 +02:00
Miro Hrončok
11b7e874ef Test pypy3 tests with current pypy3, that is Python 3.6 2019-09-05 15:07:00 +02:00
Miro Hrončok
30e8067df5 Test debug tests with current rawhide python3-debug 2019-09-05 15:07:00 +02:00
Miro Hrončok
d37bcbfd9a Also enable testing virtualenv --no-download 2019-09-05 15:07:00 +02:00
Miro Hrončok
d72c62fe79 It is no longer needed to fetch Cython from git master 2019-09-02 15:50:46 +02:00
Miro Hrončok
f5948a878c Update the test image before running tests
See https://pagure.io/fedora-ci/general/issue/20
2019-06-27 17:40:09 +02:00
Miro Hrončok
8f563a8e11 Stop running findleaks tests, but run a test on debug Python
See https://src.fedoraproject.org/rpms/python3/pull-request/113

We keep findleaks script in the repo, until eliminated elsewhere.
2019-06-27 11:44:48 +02:00
Miro Hrončok
bd3ec9505c Workaround fedora-ci/general#31
https://pagure.io/fedora-ci/general/issue/31
2019-05-09 16:20:16 +02:00
Miro Hrončok
0ae2425874 Python 3.8: Install Cython from git master 2019-05-09 16:20:11 +02:00
Miro Hrončok
bb40c2a2a8 Smoke test Python 3.8 2019-02-09 11:55:22 +01:00
Miro Hrončok
ca464dcb60 Fix Cython installation for fresh CPythons
We try to fetch a wheel only and if that fails, we disable compilation.
Useful for fresh CPythons such as 3.8 now, where wheels are not yet ready,
but Cython defaults to compiling (because it detects CPython).

Note that --install-option="--no-cython-compile" makes pip use source package,
hence we don't want to use it right away. We still prefer wheel over
sdist.
2019-02-07 08:36:39 +01:00
Miro Hrončok
8dbc369a49 More test output 2018-12-20 11:31:22 +01:00
Miro Hrončok
ead01d15f0 Find leaks! 2018-11-30 10:12:31 +01:00
Miro Hrončok
97db1f97bb Run Python 3.7 selftest tests in selftest tests of these tests 2018-11-29 16:50:35 +01:00
Petr Šplíchal
81d0de13aa Enable smoke test in the tests namespace pipeline 2018-11-28 13:39:02 +01:00
Miro Hrončok
11bfd62cc1 Add selftest suite (untested) 2018-11-14 16:31:00 +01:00
Petr Šplíchal
8ddab88d43 Initial example of the Flexible Metadata Format 2018-09-19 16:03:10 +02:00
15 changed files with 600 additions and 30 deletions

1
.fmf/version Normal file
View file

@ -0,0 +1 @@
1

48
flags/assertflags.py Normal file
View file

@ -0,0 +1,48 @@
"""
This script asserts that Python config vars with compiler options including
one or more -O flags have the last specified -O flag equal to the script's
first argument.
We use it to check that the debug build (as well as extension modules) was
built with a desired optimization level (usually -Og or -O0).
About -O flags: https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html
"If you use multiple -O options, with or without level numbers,
the last such option is the one that is effective."
"""
import sys
import sysconfig
# We will check all flags if none were requested
KEYS_TO_CHECK = sys.argv[2:] or list(sysconfig.get_config_vars().keys())
# For backwards compatibility, if no flags were provided, we assume flags without -O are to be skipped
# But when we provide explicit list of flags, we assert they get the options,
# so we can assert things like "CFLAGS has -O3" vs. "CFLAGS has no -O at all"
NO_FLAG_FAILS = bool(sys.argv[2:])
# The flags that currently don't have the -Og flag on the debug build
# and we consider it OK, because we don't know any better :)
SKIP = [
'CONFIGURE_CFLAGS',
'CONFIGURE_CFLAGS_NODIST',
'CONFIG_ARGS',
'OPT',
]
print('Expecting that {} is the last -O flag:\n'.format(sys.argv[1]))
ret = 0
for key in KEYS_TO_CHECK:
if key in SKIP:
continue
flags = sysconfig.get_config_vars()[key]
if isinstance(flags, str):
oflags = [f for f in flags.split(' ') if f.startswith('-O')]
if (oflags and oflags[-1] != sys.argv[1]) or (not oflags and NO_FLAG_FAILS):
print('Problem in {} -O flags: {}'.format(key, ' '.join(oflags) or '<empty>'))
ret = 1
elif oflags:
print('{} are OK'.format(key))
sys.exit(ret)

4
main.fmf Normal file
View file

@ -0,0 +1,4 @@
contact:
- Petr Šplíchal <psplicha@redhat.com>
- Miro Hrončok <mhroncok@redhat.com>
component: [python]

View file

@ -0,0 +1,32 @@
#!/bin/sh -eux
# Marshalparser helps us to uniform byte-compiled Python files
# so it has to be able to also parse pyc files from
# the main python packages.
# Moreover, it has its own database of magic numbers so this test
# checks that the parser is compatible with the latest python before
# we ship it.
# Test the latest version by default because that's the place
# where magic number incompatibility might happen with
# the largest probability
VERSION=${VERSION:-3.9}
# How many randomly-selected pyc files to check?
SAMPLE=${SAMPLE:-50}
# This allows us to test marshalparser optionally: on Fedora 33+ only
INSTALL_OR_SKIP=${INSTALL_OR_SKIP:-false}
if [ "$INSTALL_OR_SKIP" == "true" ] && [ ! -f "/usr/bin/marshalparser" ]; then
dnf -y install marshalparser || :
if [ ! -f "/usr/bin/marshalparser" ]; then
echo "marshalparser not installable, skipping this test"
exit 0
fi
fi
path="/usr/lib*/python$VERSION/"
for file in `find $path -name "*.pyc" | shuf | head -n $SAMPLE`; do
marshalparser $file || exit 1
done

92
required-symbols/check.sh Executable file
View file

@ -0,0 +1,92 @@
#!/bin/bash
# Compare required external symbols of compiled Python extension modules between
# the PR build (pre-installed by Testing Farm) and the latest stable version
# available in the distribution repositories.
#
# The PR build is already installed before this test runs (via the Testing Farm
# artifact mechanism). We save its .so files, downgrade to the stable repo version,
# collect symbols from both, and compare.
#
# Output:
# + [module] symbol — PR build adds a new required symbol (potential issue possibly requiring update of Requires)
# - [module] symbol — PR build drops a previously required symbol (generally OK)
#
# Exit codes:
# 0 — no changes, or no stable version available to compare against (skipped)
# 1 — symbol differences detected between PR build and stable release
set -eo pipefail
PYVER=${VERSION}
SCRIPT_DIR="$(dirname "$(realpath "${BASH_SOURCE[0]}")")"
WORK_DIR=""
PKGS_MODIFIED=false
cleanup() {
if [[ "$PKGS_MODIFIED" == true ]]; then
echo ""
echo "=== Reverting package changes ==="
dnf history undo last -y 2>&1 || echo "WARNING: dnf history undo failed" >&2
fi
[[ -n "$WORK_DIR" ]] && rm -rf "$WORK_DIR"
}
trap cleanup EXIT
# Directories containing compiled extension modules
SCAN_DIRS=()
for d in "/usr/lib64/python${PYVER}" "/usr/lib64/python${PYVER}t"; do
[ -d "$d" ] && SCAN_DIRS+=("$d")
done
if [ ${#SCAN_DIRS[@]} -eq 0 ]; then
echo "ERROR: No Python extension directories found for python${PYVER}" >&2
exit 1
fi
SCAN_ARGS=("${SCAN_DIRS[@]}")
for f in /usr/lib64/libpython${PYVER}*.so.*; do
[[ -f "$f" && ! -L "$f" ]] && SCAN_ARGS+=("$f")
done
WORK_DIR=$(mktemp -d)
echo "=== Collecting PR build symbols ==="
python${PYVER} "$SCRIPT_DIR/collect_symbols.py" "${SCAN_ARGS[@]}" > "$WORK_DIR/new.json"
echo ""
echo "=== Available repos ==="
dnf repolist --all
# Discover packages dynamically from the installed source RPM so we don't need
# to hardcode subpackage names — works regardless of which subpackages were built.
readarray -t PKGS < <(
rpm -qa --qf '%{SOURCERPM} %{NAME}\n' |
awk -v src="python${PYVER}" '$1 ~ "^" src "-[0-9]" { print $2 }' |
sort -u
)
echo "Packages to install from stable repos: ${PKGS[*]}"
# Install the stable version from distribution repos, excluding the artifact repo
# that carries the PR build, so we always compare against what is publicly available.
#
# Strategy:
# 1. dnf reinstall — works when the PR did not bump NVR (same version in stable repos)
# 2. dnf distro-sync — fallback when the PR bumped NVR; installs the latest stable NVR
echo ""
echo "=== Installing stable version from repos ==="
if ! dnf reinstall -y --disablerepo=test-artifacts "${PKGS[@]}" 2>&1; then
echo "INFO: reinstall failed (NVR not in stable repos), falling back to distro-sync" >&2
if ! dnf distro-sync -y --disablerepo=test-artifacts "${PKGS[@]}" 2>&1; then
echo "INFO: distro-sync failed; no stable version available, skipping comparison." >&2
exit 0
fi
fi
PKGS_MODIFIED=true
echo ""
echo "=== Collecting stable version symbols ==="
python${PYVER} "$SCRIPT_DIR/collect_symbols.py" "${SCAN_ARGS[@]}" > "$WORK_DIR/old.json"
echo ""
echo "=== Comparing PR build against stable ==="
python${PYVER} "$SCRIPT_DIR/compare_symbols.py" "$WORK_DIR/old.json" "$WORK_DIR/new.json"

View file

@ -0,0 +1,67 @@
#!/usr/bin/env python3
"""
Collect undefined non-boring external symbols from compiled Python extension modules
and libpython shared libraries.
Usage: collect_symbols.py <path> [<path>...]
Each path may be a directory (scanned recursively for *.cpython-*.so) or a file
(scanned directly; intended for libpython*.so.* passed explicitly from the caller).
Symbols are unioned across all variants of each module (regular/debug/freethreading),
and the result is printed as JSON to stdout.
"""
import json
import re
import subprocess
import sys
from pathlib import Path
BORING = re.compile(
r"@GLIBC_" # glibc versioned symbols (stable by definition)
r"|@GCC_" # GCC built-ins
r"|^_?Py[A-Za-z_]" # Python C API (resolved from libpython at runtime)
r"|^__" # C runtime internals
r"|^_ITM_" # Intel transactional memory
)
def module_name(so: Path) -> str:
# _ssl.cpython-314td-x86_64-linux-gnu.so → _ssl
if m := re.match(r"(.+?)\.cpython-", so.name):
return m.group(1)
# libpython3.14.so.1.0 → libpython3.14
return re.sub(r"\.so.*$", "", so.name)
def external_symbols(so: Path) -> list[str]:
result = subprocess.run(["nm", "-D", str(so)], capture_output=True, text=True)
return sorted(
parts[-1]
for line in result.stdout.splitlines()
if len(parts := line.split()) >= 2
and parts[-2] == "U"
and not BORING.search(parts[-1])
)
modules: dict[str, set[str]] = {}
for path_arg in sys.argv[1:]:
p = Path(path_arg)
if p.is_file():
print(f"Scanning: {p.name}", file=sys.stderr)
if syms := external_symbols(p):
modules.setdefault(module_name(p), set()).update(syms)
else:
print(f"Scanning: {path_arg}", file=sys.stderr)
for so in sorted(p.rglob("*.cpython-*.so")):
if so.is_symlink():
continue
if syms := external_symbols(so):
modules.setdefault(module_name(so), set()).update(syms)
total_symbols = sum(len(v) for v in modules.values())
print(f"Found {len(modules)} modules, {total_symbols} tracked symbols", file=sys.stderr)
json.dump({k: sorted(v) for k, v in sorted(modules.items())}, sys.stdout, indent=2)
print()

View file

@ -0,0 +1,41 @@
#!/usr/bin/env python3
"""
Compare two symbol JSON files produced by collect_symbols.py.
Usage: compare_symbols.py <old.json> <new.json>
Output:
+ [module] symbol new build adds a required symbol the old build did not have
- [module] symbol new build drops a required symbol the old build had
Exit codes:
0 no differences
1 differences found
"""
import json
import sys
from pathlib import Path
old = json.loads(Path(sys.argv[1]).read_text())
new = json.loads(Path(sys.argv[2]).read_text())
all_mods = sorted(set(old) | set(new))
changes = []
for mod in all_mods:
added = sorted(set(new.get(mod, [])) - set(old.get(mod, [])))
removed = sorted(set(old.get(mod, [])) - set(new.get(mod, [])))
if added or removed:
changes.append((mod, added, removed))
if not changes:
print("OK: no symbol changes between stable and PR build")
sys.exit(0)
print("Symbol changes detected (+ new requirement in PR build, - dropped by PR build):")
for mod, added, removed in changes:
for sym in added:
print(f" + [{mod}] {sym}")
for sym in removed:
print(f" - [{mod}] {sym}")
sys.exit(1)

1
selftest/findleaks.fmf Normal file
View file

@ -0,0 +1 @@
test: ./findleaks.sh

21
selftest/findleaks.sh Executable file
View file

@ -0,0 +1,21 @@
#!/bin/sh -eux
# set python version
VERSION=${VERSION:-3.7}
PYTHON=${PYTHON:-python${VERSION}dm}
# what to skip
# test_socket swaps and kills the machine https://bugs.python.org/issue34587
X=${X:-"-x test_socket"}
# Fedora sets explicit minimum/maximum TLS versions.
# Python's test suite assumes that the minimum/maximum version is set to
# a magic marker. We workaround the test problem by setting:
export OPENSSL_CONF=/non-existing-file
# https://bugzilla.redhat.com/show_bug.cgi?id=1618753
# https://bugzilla.redhat.com/show_bug.cgi?id=1778357
# https://bugs.python.org/issue35045
# https://bugs.python.org/issue38815
$PYTHON -m test.pythoninfo
$PYTHON -m test -wW --findleaks $X

4
selftest/main.fmf Normal file
View file

@ -0,0 +1,4 @@
require:
- python3-test
- python3-rpm-macros
- python3-tkinter

4
selftest/parallel.fmf Normal file
View file

@ -0,0 +1,4 @@
test: ./parallel.sh
duration: 30m
recommend:
- /usr/bin/dtrace

41
selftest/parallel.sh Executable file
View file

@ -0,0 +1,41 @@
#!/bin/sh -eux
# set default version to %{python3_version} if available
VERSION=${VERSION:-$(rpm --eval '%{?python3_version}')} || true
# ...or 3.8 if that macro is not available or rpm fails for any reason
VERSION=${VERSION:-3.8}
PYTHON=${PYTHON:-python$VERSION}
# what to skip
# test_socket swaps and kills the machine https://bugs.python.org/issue34587
X=${X:-"-x test_socket"}
TIMEOUT=${TIMEOUT:-1800}
# parallel jobs, 0 lets Python decide what's best
JOBS=${JOBS:-0}
# Fedora sets explicit minimum/maximum TLS versions.
# Python's test suite assumes that the minimum/maximum version is set to
# a magic marker. We workaround the test problem by setting:
export OPENSSL_CONF=/non-existing-file
# https://bugzilla.redhat.com/show_bug.cgi?id=1618753
# https://bugzilla.redhat.com/show_bug.cgi?id=1778357
# https://bugs.python.org/issue35045
# https://bugs.python.org/issue38815
# setuptools 60+ uses its own copy of distutils by default
# this setting must be overriden with the environment variable for
# Python tests to use the standard library's distutils
# We only do this if stdlib distutils actually exists
$PYTHON -c 'import distutils' 2>/dev/null && export SETUPTOOLS_USE_DISTUTILS=stdlib || true
$PYTHON -m test.pythoninfo
$PYTHON -m test -wW -j$JOBS --timeout=$TIMEOUT $X && e=0 || e=$?
# https://tmt.readthedocs.io/en/stable/spec/tests.html#framework says only exit code 1 is failure, rest is error
# https://github.com/python/cpython/blob/v3.14.0/Lib/test/libregrtest/results.py#L18 says a bad test exits with 2
if [ $e -eq 2 ]; then
exit 1
fi
exit $e

54
smoke/venv.fmf Normal file
View file

@ -0,0 +1,54 @@
summary: Basic smoke test for virtualenv
description: |
Check basic functionality of Python virtual environments (venv
or virtualenv based). The test supports different python
versions & implementations including pypy and jython.
The following steps are performed:
* Create virtual environment
* Activate virtual environment
* Install pip, pytest and Cython
* Create and test a simple module
* Check we can do extension modules
* Ensure we can test with the tox tool
Use the following environment variables to adjust what
should be tested (default values listed first):
* VERSION ... python version (3.7, 3.6, 2.7, 2.6)
* PYTHON ... implementation (python${VERSION}, pypy, pypy3, jython)
* METHOD ... virtual environment creation method (venv, virtualenv)
* TOX ... enable or disable the tox test (true, false)
path: /smoke
test: ./venv.sh
tier: 1
tag: [venv]
require+:
- gcc
- gcc-c++
environment:
VERSION: 3.7
METHOD: venv
TOX: true
TOX_REQUIRES: ""
duration: 10m
component+:
- python2
- python26
- python3
- python34
- python35
- python36
- pypy
- pypy3
- jython
- python-pip
- python-wheel
- python-setuptools
- python-virtualenv
- python-tox

View file

@ -1,24 +1,56 @@
#!/bin/sh -eux #!/bin/sh -eux
# set python version # set default version to %{python3_version} if available
VERSION=${VERSION:-3.7} VERSION=${VERSION:-$(rpm --eval '%{?python3_version}')} || true
# ...or 3.8 if that macro is not available or rpm fails for any reason
VERSION=${VERSION:-3.8}
PYTHON=${PYTHON:-python$VERSION} PYTHON=${PYTHON:-python$VERSION}
METHOD=${METHOD:-venv} METHOD=${METHOD:-venv}
TOX=${TOX:-true} TOX=${TOX:-true}
TOX_REQUIRES=${TOX_REQUIRES:-}
CYTHON=${CYTHON:-true}
INSTALL_OR_SKIP=${INSTALL_OR_SKIP:-false}
# clean from possible older runs # Work from a custom directory
rm -rf venv .tox __pycache__ .pytest* test_*.py *.pyx *.c *.so || : workdir="$(mktemp --directory)"
cd "$workdir"
if [ "$INSTALL_OR_SKIP" == "true" ]; then
[ ! -f "/usr/bin/$PYTHON" ] && dnf -y install "/usr/bin/$PYTHON" || :
[ ! -f "/usr/bin/${PYTHON}-config" ] && dnf -y install "/usr/bin/${PYTHON}-config" || :
dnf -y install "$PYTHON-devel" || :
if [ ! -f "/usr/bin/$PYTHON" ]; then
echo "/usr/bin/$PYTHON not installable, skipping this test"
exit 0
fi
fi
# check the we can create the venv # check the we can create the venv
if [ "$METHOD" == "venv" ]; then if [ "$METHOD" == "venv" ]; then
$PYTHON -m venv venv $PYTHON -m venv venv
elif [ "$METHOD" == "virtualenv" ]; then elif [ "$METHOD" == "virtualenv" ]; then
virtualenv --python=$PYTHON venv virtualenv --python=$PYTHON venv
elif [ "$METHOD" == "virtualenv-download" ]; then
virtualenv --download --python=$PYTHON venv
elif [ "$METHOD" == "virtualenv-no-download" ]; then
virtualenv --no-download --python=$PYTHON venv
elif [ "$METHOD" == "virtualenv-seeder-pip" ]; then
virtualenv --seeder pip --python=$PYTHON venv
elif [ "$METHOD" == "none" ]; then
echo 'Skipping plain virtual environment tests' >&2
else else
echo 'Unsupported $METHOD' $METHOD >&2 echo 'Unsupported $METHOD' $METHOD >&2
exit 1 exit 1
fi fi
# create tests
cat > test_foo.py << EOF
def test_foo():
assert True
EOF
if [ "$METHOD" != "none" ]; then
# and activate it # and activate it
# unset variables on 3.5, it's known # unset variables on 3.5, it's known
@ -27,29 +59,15 @@ source venv/bin/activate
set -u set -u
# run python in it # run python in it
python -c 'import sys; print(sys.version)' | head -n1 | grep $VERSION python -c 'import sys; print(sys.version)' | head -n1 | grep ${VERSION/t}
# install packages with pip # install packages with pip
if [ "$VERSION" == "2.6" ]; then python -m pip install pytest
pip install pytest
pip install Cython --install-option="--no-cython-compile"
else
python -m pip install pytest
if [ "$PYTHON" != "jython" ]; then
python -m pip install Cython
fi
fi
# run tests # run tests
cat > test_foo.py << EOF
def test_foo():
assert True
EOF
python -m pytest -v test_foo.py python -m pytest -v test_foo.py
# check that we can do extension modules if [ "$CYTHON" == "true" ]; then
if [ "$PYTHON" != "jython" ]; then
cat > module.pyx << EOF cat > module.pyx << EOF
cdef int add(int a, int b): cdef int add(int a, int b):
return a + b return a + b
@ -60,6 +78,12 @@ def two():
return add(a, b) return add(a, b)
EOF EOF
cat > pyproject.toml << EOF
[build-system]
requires = ["setuptools", "cython"]
build-backend = "setuptools.build_meta"
EOF
cat > setup.py << EOF cat > setup.py << EOF
from setuptools import setup from setuptools import setup
from Cython.Build import cythonize from Cython.Build import cythonize
@ -67,7 +91,7 @@ from Cython.Build import cythonize
setup(ext_modules = cythonize('module.pyx')) setup(ext_modules = cythonize('module.pyx'))
EOF EOF
python setup.py build_ext --inplace python -m pip install .
python -c 'import module; print(module.two())' | grep '^2$' python -c 'import module; print(module.two())' | grep '^2$'
fi fi
@ -77,18 +101,50 @@ set +u
deactivate deactivate
set -u set -u
fi # METHOD != none
# ensurepip test (when testing virtualenv, this was not covered)
# with the main Python, this will say "Requirement already satisfied", but that's OK
$PYTHON -m ensurepip --root ensurepiptestroot
# use it with tox # use it with tox
[[ "$TOX" != "true" ]] && exit 0 [[ "$TOX" != "true" ]] && exit 0
cat > tox.ini << EOF if [[ $PYTHON == python* ]]; then
[tox] export TOXENV=py${VERSION/./}
elif [[ $PYTHON == pypy* ]]; then
export TOXENV=pypy${VERSION/./}
else
export TOXENV=$PYTHON
fi
echo '[tox]' > tox.ini
if [ -n "$TOX_REQUIRES" ]; then
echo 'requires = '"$TOX_REQUIRES" >> tox.ini
fi
cat >> tox.ini << EOF
skipsdist = True
[testenv]
commands = python -c 'import sys; print(sys.version)'
EOF
# Tests that the Python version used in tox is really the Python version we want
tox | tee toxlog
grep "^${VERSION/t}\.[0-9]" toxlog
echo '[tox]' > tox.ini
if [ -n "$TOX_REQUIRES" ]; then
echo 'requires = '"$TOX_REQUIRES" >> tox.ini
fi
cat >> tox.ini << EOF
skipsdist = True skipsdist = True
[testenv] [testenv]
deps = pytest deps = pytest
commands = python -m pytest -v test_foo.py commands = python -m pytest -v test_foo.py
EOF EOF
if [[ $PYTHON == python* ]]; then # A more complex example with pytest
tox -e py${VERSION/./} tox
else
tox -e $PYTHON # Clean up if everything else went well
fi cd -
rm -rf "$workdir"

104
tests.yml Normal file
View file

@ -0,0 +1,104 @@
---
- hosts: localhost
tags:
- classic
tasks:
- dnf:
name: "*"
state: latest
- ansible.builtin.user:
name: testuser
- hosts: localhost
roles:
- role: standard-test-basic
tags:
- classic
tests:
- smoke_default:
dir: smoke
run: runuser testuser -c './venv.sh'
- smoke27:
dir: smoke
run: VERSION=2.7 INSTALL_OR_SKIP=true METHOD=virtualenv TOX_REQUIRES="virtualenv<20.22.0" ./venv.sh
- smoke27_seeder_pip:
dir: smoke
run: VERSION=2.7 INSTALL_OR_SKIP=true METHOD=virtualenv-seeder-pip TOX=false ./venv.sh
- smoke36:
dir: smoke
run: runuser testuser -c 'VERSION=3.6 METHOD=venv TOX_REQUIRES="virtualenv<20.22.0" ./venv.sh'
- smoke37:
dir: smoke
run: VERSION=3.7 METHOD=venv INSTALL_OR_SKIP=true ./venv.sh
- smoke38:
dir: smoke
run: VERSION=3.8 METHOD=venv INSTALL_OR_SKIP=true ./venv.sh
- smoke39:
dir: smoke
run: runuser testuser -c 'VERSION=3.9 METHOD=venv ./venv.sh'
- smoke311:
dir: smoke
run: runuser testuser -c 'VERSION=3.11 METHOD=venv CYTHON=false ./venv.sh'
- smoke312:
dir: smoke
run: runuser testuser -c 'VERSION=3.12 METHOD=venv ./venv.sh'
- smoke314t_optional:
dir: smoke
run: VERSION=3.14t METHOD=venv INSTALL_OR_SKIP=true ./venv.sh
- smoke39_seeder_pip:
dir: smoke
run: runuser testuser -c 'VERSION=3.9 METHOD=virtualenv-seeder-pip ./venv.sh'
- smoke34_optional:
dir: smoke
run: VERSION=3.4 METHOD=venv INSTALL_OR_SKIP=true ./venv.sh
- smoke310_optional:
dir: smoke
run: VERSION=3.10 METHOD=venv TOX=false INSTALL_OR_SKIP=true ./venv.sh
- smoke311_virtualenv_download:
dir: smoke
run: VERSION=3.10 METHOD=virtualenv-download TOX=false ./venv.sh
- smokepypy:
dir: smoke
run: runuser testuser -c 'VERSION=2.7 METHOD=none PYTHON=pypy TOX_REQUIRES="virtualenv<20.22.0" ./venv.sh'
- smokepypy310:
dir: smoke
run: runuser testuser -c 'VERSION=3.10 METHOD=venv PYTHON=pypy3.10 ./venv.sh'
- selftest39:
dir: selftest
run: runuser testuser -c 'VERSION=3.9 X="-x test_socket -x test_asyncgen -x test_asyncio -x test_compile -x test_concurrent_futures -x test_itertools -x test_multiprocessing_fork -x test_multiprocessing_forkserver -x test_shutil -x test_time -x test_multiprocessing_spawn -x test_threading -x test_wsgiref" ./parallel.sh'
- debugtest310:
dir: selftest
run: runuser testuser -c 'VERSION=3.10 PYTHON="python3.10d" X="test_ssl" ./parallel.sh'
- debugflags:
dir: flags
run: runuser testuser -c 'python3-debug ./assertflags.py -O0'
- debugflags_some:
dir: flags
run: runuser testuser -c 'python3-debug ./assertflags.py -O0 PY_STDMODULE_CFLAGS'
- optflags_o3:
dir: flags
run: runuser testuser -c 'python3 ./assertflags.py -O3 CFLAGS'
- marshalparser_compatibility:
dir: marshalparser
run: runuser testuser -c 'SAMPLE=10 test_marshalparser_compatibility.sh'
required_packages:
- util-linux # for runuser
- dnf
- gcc
- gcc-c++
- virtualenv
- python3.6
- python3.9
- python3.11
- python3.12
- pypy
- pypy3.10-devel
- python3-devel
- pypy-devel
- python3-tox
- python3-tkinter
- python3-test
- python3-debug
- python3.10-debug
- python3-rpm-macros
- marshalparser