tests/llvm
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Comprehensive review of TMT metadata

Browse source 
* Test tag review: add fedora-only and multilib, removed spoils-installation
  for being a duplicate of not-in-default
* Remove outdiated architecture and distribution adjustment rules.
* Further simplification of adjust rules by using major version comparison (~<)
  (https://fmf.readthedocs.io/en/stable/context.html#major-version)
* Add bugzilla links and descriptions to some tests
* Improve README by documenting all context dimensions used and test tags.
This commit is contained in:
Jesus Checa Hidalgo 2025-09-01 14:34:02 +02:00
commit e4d1e99d30
27 changed files with 82 additions and 87 deletions
Download patch file Download diff file Expand all files Collapse all files

22
README.md
View file

@ -7,9 +7,13 @@ Tests are executed using TMT (Test Management Tool). The repository includes a
TMT plan that runs all the tests by default. All the tests support the following
context dimensions:
* `distro`
* `arch`
* `collection` (only applicable to RHEL 7)
* `distro`: The distribution version running the tests (e.g. `rhel-9.6`,
`fedora-42`, `centos-stream-10`)
* `arch`: The hardware architecture of the host running the tests (e.g. `aarch64`,
`s390x`, `x86_64`)
* `compat`: Indicates a run for fedora compat-versioned packages (e.g. `llvm18`)
* `snapshot`: Indicates a run for [llvm snapshots](
https://github.com/fedora-llvm-team/llvm-snapshots) copr build.
To run the tests:
```bash
@ -93,6 +97,18 @@ tier: 1
# - sometag
```
### Test Tags
The following tags are used to control which tests are included or excluded in
TMT plans. Please be cautious when adding or removing these tags, as they are
used by plans in multiple repositories.
* `fedora-only`: The test applies to Fedora distros only.
* `not-in-default`: Excludes the test from the default test plan. This is useful
for resource-intensive tests that are better suited for a dedicated test plan
or could potentially break the test system.
* `multilib`: Requires 32-bit (i686) package dependencies.
### Avoid common errors using pre-commit
This project uses [`pre-commit`](https://pre-commit.com/)

8
plans/build-gating-alternatives.fmf
View file

@ -26,10 +26,6 @@ adjust+:
- because: lld is not shipped in s390x on LLVM < 18
enabled: false
when: arch == s390x and distro < rhel-8.10, fedora-40, centos-stream-9
when: arch == s390x and distro ~< rhel-8.10
- enabled: false
when: arch == s390x and distro == rhel-9 and distro < rhel-9.5
- because: lld is not shipped for ppc64
enabled: false
when: arch == ppc64
when: arch == s390x and distro ~< rhel-9.5

2
plans/build-gating.fmf
View file

@ -38,7 +38,7 @@ adjust:
discover:
- name: llvm-tests
how: fmf
filter: "tag:-spoils-installation & tag:-not-in-default"
filter: "tag:-not-in-default"
execute:
how: tmt
provision:

3
tests/abi-test-suite/main.fmf
View file

@ -1,6 +1,9 @@
summary: Run ABI tests from llvm-test-suite package
duration: 1h
tag+:
- fedora-only
adjust+:
- because: "llvm-test-suite is available for Fedora, not RHEL/CentOS"
when: >-

3
tests/bolt-sanity/main.fmf
View file

@ -1,6 +1,9 @@
summary: Sanity check that llvm-bolt works
tier: 1
tag+:
- fedora-only
adjust+:
- because: "llvm-bolt is available for Fedora, not RHEL/CentOS"
when: distro != fedora

8
tests/broken-symlinks/main.fmf
View file

@ -14,10 +14,4 @@ adjust+:
# There are no plans to backport that, so we disable the test
# in distros where LLVM < 17.
- enabled: false
when: distro < rhel-8
- enabled: false
when: distro == rhel-8 and distro <= rhel-8.9
- enabled: false
when: distro == rhel-9 and distro <= rhel-9.3
- enabled: false
when: distro == fedora and distro <= fedora-38
when: distro ~<= rhel-9.3

3
tests/clang-format-diff/main.fmf
View file

@ -1,5 +1,8 @@
summary: clang-format-diff
tier: 1
link+:
verifies: https://bugzilla.redhat.com/show_bug.cgi?id=1939018
require+:
- clang-tools-extra

4
tests/cross-compile-i686/main.fmf
View file

@ -4,6 +4,10 @@ description: >
proper directory
tier: 1
tag+:
- multilib
- fedora-only
require+:
- compiler-rt.i686
- glibc-devel.i686

14
tests/fedora-flags/main.fmf
View file

@ -1,17 +1,5 @@
summary: fedora-flags
description: Build sample C/C++ code with default system flags from redhat-rpm-config
require+:
- redhat-rpm-config
tier: 1
adjust+:
- because: s390x does not have epel repo which is required to provide rpm macros
enabled: false
when: >-
distro == rhel-7
and arch == s390x
# Note: this needs epel-release repo to be available
- because: "To enable rpm macros not available by default (e.g. build_cflags), epel-rpm-macros is needed"
require+:
- epel-rpm-macros
when: distro == rhel-7

5
tests/find-package-llvm/main.fmf
View file

@ -20,3 +20,8 @@ require+:
- llvm-devel
- clang
- clang-devel
adjust+:
- enabled: false
when: deploymentmode == image
because: "/usr is readonly in imagemode systems"

8
tests/integration-test-suite/rhel.fmf
View file

@ -27,15 +27,15 @@ adjust+:
# a way to provide custom contexts, we need to filter out by distro versions.
- require-:
- lld
when: arch == s390x and distro < rhel-8.10, centos-stream-9
when: arch == s390x and distro < rhel-8.10
- require-:
- lld
when: arch == s390x and distro == rhel-9 and distro < rhel-9.5
when: arch == s390x and distro ~< rhel-9.5
# libomp is not supported in s390x in RHEL.
# libomp is not supported in s390x in RHEL... Until LLVM 19
- require-:
- libomp-devel
when: arch == s390x
when: arch == s390x and distro ~< rhel-9.6.0
# RHEL 8 use python3.12
- require-:

1
tests/kernel-ark-build/main.fmf
View file

@ -24,6 +24,7 @@ link+:
- verifies: https://issues.redhat.com/browse/LLVM-72
tag+:
- not-in-default
- fedora-only
/base:
summary+: " (base/non-LTO)"

12
tests/libomp/main.fmf
View file

@ -12,17 +12,9 @@ require+:
adjust+:
# Right now libomp in s390x is supported only in LLVM >= 18 in Fedora
# And in LLVM >= 19 in CentOS/RHEL
# TODO until tmt allows to use custom adjusted contexts we need to do this
# indirect adjustment using the distro version.
- when: arch == s390x and distro < fedora-40 and snapshot is not defined
- when: arch == s390x and distro ~< rhel-9.6.0
enabled: false
continue: false
- when: arch == s390x and distro != fedora
enabled: false
continue: false
# Dependencies in rhel-7 are handled differently: there are no recommends,
# only requires.
- when: distro <= rhel-7
enabled: false

5
tests/lld/gcc-compat-basic/main.fmf
View file

@ -2,8 +2,3 @@ summary: Test lld with gcc
require+:
- gcc
tier: 1
adjust+:
- because: system gcc is too old in rhel-7
enabled: false
when: distro <= rhel-7

4
tests/lld/ld-alternative/main.fmf
View file

@ -1,7 +1,7 @@
summary: Test linker switching with alternatives
# This test spoils the installation by removing lld, therefore it should be marked as such.
# This test spoils the installation by removing lld.
tag+:
- spoils-installation
- not-in-default
require+:
- binutils
tier: 1

8
tests/lld/main.fmf
View file

@ -3,12 +3,8 @@ require+:
- lld
adjust+:
- because: lld is not shipped for ppc64
enabled: false
when: arch == ppc64
- because: lld is not shipped in s390x on LLVM < 18
enabled: false
when: arch == s390x and distro < rhel-8.10, fedora-40, centos-stream-9
when: arch == s390x and distro < rhel-8.10
- enabled: false
when: arch == s390x and distro == rhel-9 and distro < rhel-9.5
when: arch == s390x and distro ~< rhel-9.5

4
tests/llvm-test-suite/main.fmf
View file

@ -1,6 +1,8 @@
# TODO: once llvm-test-suite is converted to TMT, we can just link the test from plan.
summary: Run tests from llvm-test-suite package
tag+:
- fedora-only
adjust+:
- because: "llvm-test-suite is available for Fedora, not RHEL/CentOS"
when: >-

3
tests/multilib-install/main.fmf
View file

@ -1,6 +1,9 @@
summary: Make sure that multilib installation works
tier: 1
tag+:
- multilib
require+:
- llvm-devel
- llvm-devel.i686

7
tests/openmp-rpm/main.fmf
View file

@ -8,12 +8,9 @@ component+:
adjust+:
# Right now libomp in s390x is supported only in LLVM >= 18 in Fedora
# And in LLVM >= 19 in CentOS/RHEL
# TODO until tmt allows to use custom adjusted contexts we need to do this
# indirect adjustment using the distro version.
- when: arch == s390x and distro < fedora-40 and snapshot is not defined
enabled: false
continue: false
- when: arch == s390x and distro != fedora
- when: arch == s390x and distro ~< rhel-9.6.0
enabled: false
continue: false

2
tests/rhbz_482491/main.fmf → tests/rhbz_1482491/main.fmf
View file

@ -1,4 +1,4 @@
summary: rhbz-482491
summary: rhbz-1482491
description: |
"Test that clang is able to find the right libgcc_s"
require+:

0
tests/rhbz_482491/test.sh → tests/rhbz_1482491/test.sh
View file

1
tests/rhbz_1647130/main.fmf
View file

@ -1,4 +1,5 @@
summary: RHBZ1647130
description: Test that scan-build and scan-view can run without aborting
tier: 1
require+:

5
tests/rhbz_1657544/main.fmf
View file

@ -1,2 +1,7 @@
summary: RHBZ1657544
description: |
Test that clang++ does not use the symbol __muloti4
tier: 1
link+:
resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1657544

3
tests/rhbz_2239619/main.fmf
View file

@ -1,2 +1,5 @@
summary: Test if debug information is disabled by default
tier: 1
link+:
verifies: https://bugzilla.redhat.com/show_bug.cgi?id=2239619

20
tests/toolchains/main.fmf
View file

@ -13,17 +13,7 @@ adjust+:
# requirements, remove it.
- require-:
- lld
when: >-
arch == s390x and distro < rhel-9, rhel-9.5, fedora-40, centos-stream-9
or arch == ppc64
# LLVM is getting updated to 18 in rhel 8.10 which adds lld for s390x, but we
# still have LLVM 17 in AppStream. Install lld if it's available, leave it out
# if not. The test will (correctly) fail if we lld is missing in LLVM 18.
# TODO once LLVM 18 is in the default module stream we can set this a require:
- recommend+:
- lld
when: arch == s390x and distro == rhel-8.10
when: arch == s390x and distro ~< rhel-9.5
- environment+:
CXXLIBS: "libc++"
@ -37,9 +27,7 @@ adjust+:
CXXLIBS: "libstdc++"
require+:
- libstdc++
when: >-
distro == rhel
or distro == centos
when: distro == rhel, centos
because: testing against libstdc++ package in RHEL as libcxx is not shipped with RHEL
# llvm-snapshots specific adjustments.
@ -58,7 +46,3 @@ adjust+:
- lld
when: snapshot is defined
because: llvm-snapshots adjustments
# TODO REVIEW: are these all requirements? test.sh seems to run quite a lot of stuff, looks like we
# need more packages from LLVM family.

10
tests/ucrt64-toolchain/main.fmf
View file

@ -2,7 +2,13 @@ summary: Test that ucrt64 toolchain is detected
require+:
- ucrt64-gcc-c++
tag+:
- fedora-only
link+:
- verifies: https://bugzilla.redhat.com/show_bug.cgi?id=2152220
adjust+:
- enabled: false
when: distro != fedora or distro < fedora-37
because: "The ucrt64 toolchain is only available since Fedora 37"
when: distro != fedora
because: "The ucrt64 toolchain is only available in Fedora"

4
tests/zstd-support/main.fmf
View file

@ -5,9 +5,7 @@ framework: beakerlib
require+:
- lld
adjust+:
- when: distro == fedora and distro < fedora-41 or
distro == rhel-8 and distro < rhel-8.10 or
distro == rhel-9 and distro < rhel-9.6
- when: distro ~< rhel-8.10, rhel-9.6
enabled: false
because: ZSTD is enabled only in LLVM 19 and newer
link+: