Update ucrt test for new file output

After a recent file update, the output is now:

> a.exe: PE32+ executable for MS Windows 5.02 (console), x86-64, 18 sections

Adjust the grep to match both the old and new output. I think the
key parts are that it's a "PE32+ executable" for x86-64.

.pre-commit-config.yaml

@@ -0,0 +1,21 @@
+# See https://pre-commit.com for more information
+# See https://pre-commit.com/hooks.html for more hooks
+repos:
+- repo: https://github.com/pre-commit/pre-commit-hooks
+  rev: v4.5.0
+  hooks:
+  -   id: trailing-whitespace
+  -   id: end-of-file-fixer
+
+# See https://tmt.readthedocs.io/en/latest/guide.html#checking-data-validity
+- repo: https://github.com/teemtee/tmt.git
+  rev: 1.32.2
+  hooks:
+  - id: tmt-lint
+
+- repo: https://github.com/koalaman/shellcheck-precommit
+  rev: v0.10.0
+  hooks:
+  - id: shellcheck
+    require_serial: true # Podman has trouble running concurrently
+    args: ["--exclude=SC1091"] # Ignore "Not following" sourced scripts

README.md

@@ -0,0 +1,145 @@
+# clang tests
+
+This repository contains tests for clang.
+
+## Usage
+Tests are executed using TMT (Test Management Tool). The repository includes a
+TMT plan that runs all the tests by default. All the tests support the following
+context dimensions:
+
+* `distro`
+* `arch`
+* `collection` (only applicable to RHEL 7)
+
+To run the tests:
+```bash
+# Run in the default virtual environment
+tmt -c distro=<distro> -c arch=<arch> run -avv
+
+# Run on a scheduled system
+tmt -c distro=<distro> -c arch=<arch> run -avv provision -h connect -g <ip_or_hostname>
+
+# Run specific tests
+tmt -c distro=<distro> -c arch=<arch> run -avv provision -h connect -g <ip_or_hostname> test -n <regex>
+```
+
+## Adding New Tests
+Please consider the following when adding new tests:
+
+* Test scripts must be named `runtest.sh`. If a different test is needed, you need
+to overwrite the `test:` key in the test `main.fmf`.
+* Tests must be placed under `tests/<test_name>`. Additionally they can be
+  placed into subdirectories if grouping make sense. E.g.
+  `tests/regression/<test_name>`.
+* This repository has a common configuration under `tests/main.fmf`, inherited
+  by all tests. When adding new ones, be careful not to overwrite existing keys
+  from the inherited `tests/main.fmf`.
+* When setting FMF metadata keys, do merge values instead overwriting them. E.g.
+use `require+:` instead `require:`. For advanced use and more examples check
+[FMF documentation](https://fmf.readthedocs.io/en/stable/features.html#merging).
+
+After creating the new test, consider running `tmt <context> tests show
+<test-name>` to review that the resulting metadata such as requires, enable or
+test keys are as expected. For example:
+```
+$ tmt -c distro=fedora-38 tests show ucrt64-toolchain
+/tests/ucrt64-toolchain
+                 summary Test that ucrt64 toolchain is detected
+                 contact Jesus Checa Hidalgo <jcheca@redhat.com>
+               component 'llvm-toolset' and 'clang'
+                    test ./runtest.sh
+                    path /tests/ucrt64-toolchain
+               framework shell
+                  manual false
+                     tty false
+                 require 'ucrt64-gcc-c++' and 'clang'
+                duration 5m
+                 enabled true
+                  result respect
+
+$ tmt -c distro=rhel-9.3 tests show ucrt64-toolchain
+/tests/ucrt64-toolchain
+                 summary Test that ucrt64 toolchain is detected
+                 contact Jesus Checa Hidalgo <jcheca@redhat.com>
+               component 'llvm-toolset' and 'clang'
+                    test ./runtest.sh
+                    path /tests/ucrt64-toolchain
+               framework shell
+                  manual false
+                     tty false
+                 require 'ucrt64-gcc-c++' and 'clang'
+                duration 5m
+                 enabled false
+                  result respect
+```
+
+### Templates
+Use the following templates when creating new tests.
+
+FMF template. Uncomment needed fields, remove the rest.
+```yaml
+summary: Descriptive summary for the test
+# Maximum duration for the test
+duration: 2m
+# Short tests should be tier <= 1 so they're used for gating.
+tier: 1
+# Add an additional contact(s) if applicable
+# contact+:
+#   - Somebody <somebody@redhat.com>
+# Additional requires
+# require+:
+#   - rpm-build
+# Adjustments based on context dimensions
+# adjust+:
+#   - when: collection is defined
+#     enabled: false
+#     because: Test not supported in collections
+# link+:
+#   - verifies: https://issues.redhat.com/browse/...
+# tag+:
+#   - sometag
+```
+
+### Avoid common errors using pre-commit
+
+This project uses [`pre-commit`](https://pre-commit.com/)
+
+* to [lint fmf files](https://tmt.readthedocs.io/en/latest/guide.html#lint)
+* to check that you don't have [trailing
+  whitespaces](https://github.com/pre-commit/pre-commit-hooks?tab=readme-ov-file#trailing-whitespace)
+* to check that [files end with a
+  newline](https://github.com/pre-commit/pre-commit-hooks?tab=readme-ov-file#end-of-file-fixer)
+
+Please install `pre-commit` using `pip install pre-commit` as described
+[here](https://pre-commit.com/#install). Then navigate to your clone of this
+project and install the git hook scripts using `pre-commit install`. This will
+run pre-commit on every `git commit` that you make in this repository from now
+on.
+
+## Test specific documentation
+
+### kernel-ark-rebuild
+The `kernel-ark-rebuild` test is a special test that we use to ensure that clang,
+llvm, and lld builds do not break [Always Ready Kernel](
+https://gitlab.com/cki-project/kernel-ark) builds, which are done in Fedora
+Rawhide. This is a virtual test comprised of 4 tests corresponding to different
+build configurations for the kernel:
+
+* base
+* base/lto
+* debug
+* debug/lto
+
+Due to this, this test is not regularly run on PRs opened in tests/clang, as
+running the 4 tests can take about 6 hours. Instead, if you modify the test
+you are expected to run it on your own and provide the results in the Pull
+Request. A specific tmt test plan named `kernel-ark-gating.fmf` is provided for
+that. It can be run using Testing Farm with this command:
+```
+testing-farm request -c initiator=human -c distro=fedora-rawhide -c arch=<your-arch> \
+    --compose Fedora-Rawhide --git-url <url-to-your-fork> --git-ref <your-pr-branch> \
+    --plan kernel-ark-gating
+```
+You need an [API key](https://docs.testing-farm.io/Testing%20Farm/0.1/onboarding.html)
+to be able to use Testing Farm. If you don't have on, please ask for help running
+the test when opening a Pull Request.

gcc-clang-compatibility/main.fmf

@@ -1,31 +0,0 @@
-summary: Test that both gcc/clang compile/link compatibility.
-description:
-  Build an object file with g++, link it with clang++ and viceversa, to ensure
-  that objects compiled with one can be linked with the other.
-test: "$WITH_SCL ./test.sh"
-framework: shell
-tier: 1
-component:
-  - llvm-toolset
-  - clang
-extra-summary: /tools/clang/gcc-clang-compatibility
-extra-task: /tools/clang/gcc-clang-compatibility
-
-require:
-  - gcc-c++
-
-adjust:
-  - require+:
-      - clang
-    when: collection is not defined
-
-  # Requirements for SCL-ized LLVM
-  - require+:
-      - llvm-toolset-13.0-clang
-    when: "collection == llvm-toolset-13.0"
-  - require+:
-      - llvm-toolset-14.0-clang
-    when: "collection == llvm-toolset-14.0"
-  - require+:
-      - llvm-toolset-15.0-clang
-    when: "collection == llvm-toolset-15.0"

libomp/runtest.sh

@@ -1,22 +0,0 @@
-#!/bin/bash
-
-set -exo pipefail
-
-CLANG_VERSION=$(rpm --queryformat="%{version}" -q clang.$(uname -m))
-LIBOMP_DEPENDENCIES="libomp libomp-devel"
-
-# Ensure clang depends on the correct clang-libs version
-rpm -q --requires clang | grep "clang-libs.* = ${CLANG_VERSION}"
-
-# Check that weak dependencies are correct. The versions of these should be the same
-# as clang's to guarantee the ABI compatibility, and that version should be actually
-# installed as well.
-for lomp_dep in $LIBOMP_DEPENDENCIES; do
-    rpm -q --recommends clang-libs | grep "${lomp_dep}.* = ${CLANG_VERSION}"
-    [[ "$(rpm --queryformat="%{version}" -q ${lomp_dep}.$(uname -m))" == "${CLANG_VERSION}" ]]
-done
-
-# Perform a sanity test to ensure everything works as expected
-clang -fopenmp openmp-compile-link-test.c
-
-./a.out | grep "Num Threads: 1"

long-double/main.fmf

@@ -1,33 +0,0 @@
-summary: Test that gcc and clang use the same long double format
-test: $WITH_SCL ./runtest.sh
-duration: 5m
-framework: shell
-tier: 1
-component:
-  - llvm-toolset
-  - clang
-extra-summary: /tools/clang/long-double
-extra-task: /tools/clang/long-double
-extra-nitrate: TC#0614593
-
-adjust:
-  # Common requirements when LLVM is not SCL-ized
-  - require+:
-      - clang
-    when: "collection is not defined"
-
-  - require+:
-    - gcc
-    when: distro != fedora
-
-  # Requirements for SCL-ized LLVM
-  - require+:
-      - llvm-toolset-13.0-clang
-    when: "collection == llvm-toolset-13.0"
-  - require+:
-      - llvm-toolset-14.0-clang
-    when: "collection == llvm-toolset-14.0"
-  - require+:
-      - llvm-toolset-15.0-clang
-    when: "collection == llvm-toolset-15.0"
-id: 4096a1bb-d2a5-4aeb-b14d-fc465c293e32

openmp-rpm/main.fmf

@@ -1,51 +0,0 @@
-summary: Test build a simple RPM package to ensure that -fopenmp works
-test: "$WITH_SCL ./runtest.sh"
-require:
-  - rpm-build
-framework: shell
-tier: 1
-component:
-  - llvm-toolset
-  - clang
-extra-summary: /tools/clang/openmp-rpm
-extra-task: /tools/clang/openmp-rpm
-extra-nitrate: TC#0614128
-
-adjust:
-  - because: "libomp not supported in s390x"
-    when: arch == s390x
-    enabled: false
-    continue: false
-
-  - because: "Use yum in RHEL < 8"
-    require+:
-      - yum-utils
-    environment+:
-      BUILDDEP_CMD: "yum-builddep"
-    when: distro == rhel and distro < rhel-8
-
-  - because: "Use dnf5 in Fedora 39+"
-    require+:
-      - dnf5-plugins
-    environment+:
-      BUILDDEP_CMD: "dnf builddep"
-    continue: false
-    when: distro == fedora and distro >= fedora-39
-
-  - because: "Use dnf in RHEL >= 8, Fedora and CentOS"
-    require+:
-      - dnf-plugins-core
-    environment+:
-      BUILDDEP_CMD: "dnf builddep"
-    when: distro >= rhel-8 or distro != rhel
-
-  # Requirements for SCL-ized LLVM
-  - require+:
-      - llvm-toolset-13.0-build
-    when: "collection == llvm-toolset-13.0"
-  - require+:
-      - llvm-toolset-14.0-build
-    when: "collection == llvm-toolset-14.0"
-  - require+:
-      - llvm-toolset-15.0-build
-    when: "collection == llvm-toolset-15.0"

openmp-rpm/runtest.sh

@@ -1,6 +0,0 @@
-#!/bin/bash
-
-set -ex
-
-${BUILDDEP_CMD} -y test.spec
-QA_RPATHS=$(( 0x0001 )) rpmbuild --define '_sourcedir .' --define '_builddir .' -bb test.spec

plans/build-gating.fmf

@@ -19,28 +19,14 @@ adjust:
       and trigger != build
     enabled: false
 
-  - because: "When testing SCL-ized LLVM, the collection must be enabled first"
-    environment+:
-      WITH_SCL: "scl enable llvm-toolset-13.0"
-    when: "collection == llvm-toolset-13.0"
-  - because: "When testing SCL-ized LLVM, the collection must be enabled first"
-    environment+:
-      WITH_SCL: "scl enable llvm-toolset-14.0"
-    when: "collection == llvm-toolset-14.0"
-  - because: "When testing SCL-ized LLVM, the collection must be enabled first"
-    environment+:
-      WITH_SCL: "scl enable llvm-toolset-15.0"
-    when: "collection == llvm-toolset-15.0"
-
   # Unfortunately, TMT does not support more declarative approach, we need to run commands on our own.
-  - because: "On RHEL, CRB must be enabled to provide rarer packages"
+  - because: "On RHEL-8+, CRB must be enabled to provide rarer packages"
     prepare+:
       - name: Enable CRB
         how: shell
-        script: dnf config-manager --set-enabled rhel-CRB
+        script: dnf config-manager --set-enabled *-CRB
     when: >-
-      distro == rhel-9
-      or distro == rhel-8
+      distro == rhel and distro >= rhel-8
 
   # Unfortunately, TMT does not support more declarative approach, we need to run commands on our own.
   - because: "On CentOS, CRB must be enabled to provide rarer packages"
@@ -55,6 +41,12 @@ discover:
     - name: clang-tests
       how: fmf
       filter: "tag:-not-in-default"
+    # TODO uncomment when https://src.fedoraproject.org/tests/llvm-test-suite/pull-request/10 is merged
+    # - name: llvm-test-suite
+    #   how: fmf
+    #   url: https://src.fedoraproject.org/tests/llvm-test-suite.git
+    #   ref: main
+    #   test: test-suite
     - name: upstream-llvm-integration-testsuite
       how: fmf
       url: https://src.fedoraproject.org/tests/llvm.git

plans/kernel-ark-gating.fmf

@@ -0,0 +1,18 @@
+summary: Build latest kernel-ark with clang using different build configurations
+discover:
+    how: fmf
+    test: kernel-ark-build
+execute:
+    how: tmt
+provision:
+    hardware:
+        memory: ">=16 GiB"
+        cpu:
+            cores: ">=8"
+
+# We do not want to run this test on every pull request done to tests/clang.
+# Only allow to run manually when the test is really modified to prevent
+# using too much time and resources in non-related pull requests.
+adjust:
+    - when: initiator is not defined or initiator != human
+      enabled: false

ppc64le-long-double/main.fmf

@@ -1,47 +0,0 @@
-summary: Test binary compatibility of the long double format on ppc64le
-test: $WITH_SCL ./runtest.sh
-duration: 1h
-framework: shell
-tier: 1
-component:
-  - llvm-toolset
-  - clang
-extra-summary: /tools/clang/ppc64le-long-double
-extra-task: /tools/clang/ppc64le-long-double
-extra-nitrate: TC#0614129
-
-adjust:
-  # Common requirements when LLVM is not SCL-ized
-  - require+:
-      - clang
-    when: "collection is not defined"
-
-  - because: "Fedora CI runs in x86_64 only, emulate with qemu and mock"
-    require+:
-    - qemu-user-static
-    - mock
-    test: ./runtest-fedora.sh
-    when: distro == fedora and arch == x86_64
-    continue: false
-
-  - enabled: false
-    when: arch != ppc64le
-
-  - because: "glibc in RHEL older than 9 don't support IEEE 128bit doubles"
-    enabled: false
-    when: distro < rhel-9
-
-  - require+:
-    - gcc
-    when: distro != fedora
-
-  # Requirements for SCL-ized LLVM
-  - require+:
-      - llvm-toolset-13.0-clang
-    when: "collection == llvm-toolset-13.0"
-  - require+:
-      - llvm-toolset-14.0-clang
-    when: "collection == llvm-toolset-14.0"
-  - require+:
-      - llvm-toolset-15.0-clang
-    when: "collection == llvm-toolset-15.0"

rhbz_1647130/runtest.sh

@@ -1,10 +0,0 @@
-#!/bin/sh
-set -e
-set -x
-
-tmp_cpp=`mktemp -t XXXXX.cpp`
-tmp_dir=`mktemp -d`
-echo 'int main(int argc, char*argv[]) { while(argc--) new int(); return 0; }' > $tmp_cpp
-scan-build -o $tmp_dir clang++ -c $tmp_cpp -o /dev/null
-(scan-view --no-browser $tmp_dir/* & WPID=$! && sleep 10s && kill $WPID)
-

rhbz_1657544/main.fmf

@@ -1,28 +0,0 @@
-summary: RHBZ1657544
-test: "$WITH_SCL ./runtest.sh"
-require: []
-framework: shell
-tier: 1
-component:
-  - llvm-toolset
-  - clang
-extra-summary: /tools/clang/rhbz_165744
-extra-task: /tools/clang/rhbz_165744
-extra-nitrate: TC#0614131
-
-adjust:
-  # Common requirements when LLVM is not SCL-ized
-  - require+:
-      - clang
-    when: "collection is not defined"
-
-  # Requirements for SCL-ized LLVM
-  - require+:
-      - llvm-toolset-13.0-clang
-    when: "collection == llvm-toolset-13.0"
-  - require+:
-      - llvm-toolset-14.0-clang
-    when: "collection == llvm-toolset-14.0"
-  - require+:
-      - llvm-toolset-15.0-clang
-    when: "collection == llvm-toolset-15.0"

rhbz_2239619/main.fmf

@@ -1,25 +0,0 @@
-summary: Test if debug information is disabled by default
-test: "$WITH_SCL ./runtest.sh"
-require: []
-framework: shell
-tier: 1
-component:
-  - llvm-toolset
-  - clang
-
-adjust:
-  # Common requirements when LLVM is not SCL-ized
-  - require+:
-      - clang
-    when: "collection is not defined"
-
-  # Requirements for SCL-ized LLVM
-  - require+:
-      - llvm-toolset-13.0-clang
-    when: "collection == llvm-toolset-13.0"
-  - require+:
-      - llvm-toolset-14.0-clang
-    when: "collection == llvm-toolset-14.0"
-  - require+:
-      - llvm-toolset-15.0-clang
-    when: "collection == llvm-toolset-15.0"

rhbz_482491/main.fmf

@@ -1,39 +0,0 @@
-summary: rhbz-482491
-description: |
-  "Test that clang is able to find the right libgcc_s"
-test: "$WITH_SCL ./test.sh"
-require:
-  - libgcc
-framework: shell
-tier: 1
-component:
-  - llvm-toolset
-  - clang
-extra-summary: /tools/clang/rhbz_482194
-extra-task: /tools/clang/rhbz_482194
-extra-nitrate: TC#0614132
-
-link:
-  - relates: https://bugzilla.redhat.com/show_bug.cgi?id=1482491
-
-adjust:
-  # gcc-x86_64-linux-gnu is not available in RHEL
-  - require+:
-      - gcc-x86_64-linux-gnu
-    when: "distro == fedora"
-
-  # Common requirements when LLVM is not SCL-ized
-  - require+:
-      - clang
-    when: "collection is not defined"
-
-  # Requirements for SCL-ized LLVM
-  - require+:
-      - llvm-toolset-13.0-clang
-    when: "collection == llvm-toolset-13.0"
-  - require+:
-      - llvm-toolset-14.0-clang
-    when: "collection == llvm-toolset-14.0"
-  - require+:
-      - llvm-toolset-15.0-clang
-    when: "collection == llvm-toolset-15.0"

rpmmacros/main.fmf

@@ -1,11 +0,0 @@
-summary: Test if RPM macros are properly set
-test: "./runtest.sh"
-framework: shell
-tier: 1
-component:
-  - clang
-require:
-  - clang
-  - clang-devel
-extra-summary: /tools/clang/rpmmacros
-extra-task: /tools/clang/rpmmacros

tests/clang-format-diff/main.fmf

@@ -1,27 +1,22 @@
 summary: clang-format-diff
-test: "$WITH_SCL ./test.sh"
-framework: shell
 tier: 1
-component:
-  - llvm-toolset
-  - clang
 extra-summary: /tools/clang/clang-format-diff
 extra-task: /tools/clang/clang-format-diff
 extra-nitrate: TC#0614126
 
-adjust:
+adjust+:
   # Common requirements when LLVM is not SCL-ized
-  - require:
+  - require+:
       - clang-tools-extra
     when: "collection is not defined"
 
   # Requirements for SCL-ized LLVM
-  - require:
+  - require+:
       - llvm-toolset-13.0-clang-tools-extra
     when: "collection == llvm-toolset-13.0"
-  - require:
+  - require+:
       - llvm-toolset-14.0-clang-tools-extra
     when: "collection == llvm-toolset-14.0"
-  - require:
+  - require+:
       - llvm-toolset-15.0-clang-tools-extra
     when: "collection == llvm-toolset-15.0"

tests/fedora-flags/main.fmf

@@ -1,34 +1,12 @@
 summary: fedora-flags
-test: "$WITH_SCL ./runtest.sh"
-require:
+require+:
   - redhat-rpm-config
-framework: shell
 tier: 1
-component:
-  - llvm-toolset
-  - clang
 extra-summary: /tools/clang/fedora-flags
 extra-task: /tools/clang/fedora-flags
 extra-nitrate: TC#0614127
 
-adjust:
-
-  # Common requirements when LLVM is not SCL-ized
-  - require+:
-      - clang
-    when: "collection is not defined"
-
-  # Requirements for SCL-ized LLVM
-  - require+:
-      - llvm-toolset-13.0-clang
-    when: "collection == llvm-toolset-13.0"
-  - require+:
-      - llvm-toolset-14.0-clang
-    when: "collection == llvm-toolset-14.0"
-  - require+:
-      - llvm-toolset-15.0-clang
-    when: "collection == llvm-toolset-15.0"
-
+adjust+:
   - because: s390x does not have epel repo which is required to provide rpm macros
     enabled: false
     when: >-

tests/fedora-flags/runtest.sh

@@ -1,10 +1,11 @@
 #!/bin/bash
+# shellcheck disable=SC2086
 
 set -ex pipefail
 
-cflags=`rpm -D '%toolchain clang' -E %{build_cflags}`
-cxxflags=`rpm -D '%toolchain clang' -E %{build_cxxflags}`
-ldflags=`rpm -D '%toolchain clang' -E %{build_ldflags}`
+cflags=$(rpm -D '%toolchain clang' -E '%{build_cflags}')
+cxxflags=$(rpm -D '%toolchain clang' -E '%{build_cxxflags}')
+ldflags=$(rpm -D '%toolchain clang' -E '%{build_ldflags}')
 
 
 # Test a c program

tests/gcc-clang-compatibility/main.fmf

@@ -0,0 +1,10 @@
+summary: Test that both gcc/clang compile/link compatibility.
+description:
+  Build an object file with g++, link it with clang++ and viceversa, to ensure
+  that objects compiled with one can be linked with the other.
+tier: 1
+extra-summary: /tools/clang/gcc-clang-compatibility
+extra-task: /tools/clang/gcc-clang-compatibility
+
+require+:
+  - gcc-c++

tests/gcc-clang-compatibility/runtest.sh

@@ -1,4 +1,4 @@
-#!/bin/sh -eux
+#!/bin/bash -eux
 
 tmp=$(mktemp -d)
 
@@ -6,7 +6,7 @@ tmp=$(mktemp -d)
 # gcc-toolset-XX, in such case we need to test the compatibility with that one.
 # We can get that from `clang -v` output.
 TOOLSET=$(clang -v |& grep "Selected GCC installation" | grep -P -o '(dev|gcc-)toolset-[0-9]*') ||:
-if [[ "x" = "x${TOOLSET}" ]]; then
+if [[ "" = "${TOOLSET}" ]]; then
     GCC="g++"
 else
     GCC="scl enable ${TOOLSET} -- g++"
@@ -14,13 +14,13 @@ else
 fi
 
 # Build the source with GCC, link it with clang
-${GCC} -c hello.cpp -o ${tmp}/hello.o
-clang++ -o ${tmp}/hello ${tmp}/hello.o
-${tmp}/hello | grep "Hello world"
-rm -rf ${tmp}/*
+${GCC} -c hello.cpp -o "${tmp}/hello.o"
+clang++ -o "${tmp}"/hello "${tmp}/hello.o"
+"${tmp}/hello" | grep "Hello world"
+rm -rf "${tmp:?}"/*
 
 # Build the source with clang, link it with GCC
-clang++ -c hello.cpp -o ${tmp}/hello.o
-${GCC} -o ${tmp}/hello ${tmp}/hello.o
-${tmp}/hello | grep "Hello world"
-rm -rf ${tmp}/*
+clang++ -c hello.cpp -o "${tmp}/hello.o"
+${GCC} -o "${tmp}"/hello "${tmp}/hello.o"
+"${tmp}/hello" | grep "Hello world"
+rm -rf "${tmp:?}"/*

tests/kernel-ark-build/main.fmf

@@ -0,0 +1,54 @@
+summary: Build kernel-ark with clang
+framework: beakerlib
+component+:
+  - llvm
+  - lld
+# CKI pipelines times out after 5h. 4h is safe for us.
+duration: 4h
+tier: 1
+require+:
+  - llvm-devel
+  - lld
+  - git
+  - make
+  - gcc
+  - flex
+  - bison
+  - bzip2
+  - rpm-build
+adjust+:
+  - when: distro is not defined or distro != fedora-rawhide
+    enabled: false
+    because: Only meaninful in Rawhide
+link+:
+  - verifies: https://issues.redhat.com/browse/LLVM-72
+tag+:
+  - not-in-default
+
+/base:
+  summary+: " (base/non-LTO)"
+
+/debug:
+  summary+: " (debug/non-LTO)"
+  environment+:
+    ENABLE_DEBUG: 1
+  adjust+:
+    - enabled: false
+      when: arch != x86_64 and arch != aarch64
+
+/base-lto:
+  summary+: " (base/LTO)"
+  environment+:
+    ENABLE_LTO: 1
+  adjust+:
+    - enabled: false
+      when: arch != x86_64 and arch != aarch64
+
+/debug-lto:
+  summary+: " (debug/LTO)"
+  environment+:
+    ENABLE_LTO: 1
+    ENABLE_DEBUG: 1
+  adjust+:
+    - enabled: false
+      when: arch != x86_64 and arch != aarch64

tests/kernel-ark-build/runtest.sh

@@ -0,0 +1,190 @@
+#!/bin/bash
+# vim: dict+=/usr/share/beakerlib/dictionary.vim cpt=.,w,b,u,t,i,k
+. /usr/share/beakerlib/beakerlib.sh || exit 1
+
+## General configuration variables. Can be overridden via environment
+# Kernel branch to build. Typically ark-latest or os-build.
+KERNEL_BRANCH=${KERNEL_BRANCH:-"ark-latest"}
+KERNEL_GIT_URL=${KERNEL_GIT_URL:-"https://gitlab.com/cki-project/kernel-ark.git"}
+
+## Helper variables to handle cached/generated SRPM
+CACHE_DIR_BASE="/var/tmp/kernel-"
+KERNEL_SRPM_PATTERN="kernel-*.src.rpm"
+# Path to the kernel SRPM to build.
+KERNEL_SRPM=""
+
+# Environment variables to configure the kernel build:
+#   * ENABLE_LTO: If set, it will build kernel-ark with clang_lto.
+#   * ENABLE_DEBUG: If set, it will build kernel-ark in debug mode.
+CLANG_MODE=${ENABLE_LTO:+"--with clang_lto"}
+CLANG_MODE=${CLANG_MODE:-"--with clang"}
+# Build mode, base or debug
+BUILD_MODE=${ENABLE_DEBUG:+"--with debug --without base"}
+BUILD_MODE=${BUILD_MODE:-"--with base --without debug"}
+
+# Log installed packages required by TMT
+logTmtRequiredPackages() {
+    if [[ ! -e $TMT_TEST_METADATA ]]; then
+        rlLogWarning "${FUNCNAME[0]}: No TMT_TEST_METADATA file found. Run the test from tmt"
+        return
+    fi
+    TMT_REQUIRES=$(rlGetYAMLdeps)
+    rlLog "# Installed packages:"
+    rlLog "#----------------------------#"
+    for pkg in $TMT_REQUIRES; do
+        rlAssertRpm "$pkg"
+    done
+    rlLog "#----------------------------#"
+}
+
+# Check if a previous test run created a kernel SRPM for the latest commit
+# If found, the abspath is stored in KERNEL_SRPM global variable
+# Returns 0 if srpm found, 1 otherwise.
+checkCache() {
+    local commit_hash
+    local cache_dir
+    local cached_srpm
+
+    commit_hash=$(git ls-remote "${KERNEL_GIT_URL}" "${KERNEL_BRANCH}" | awk '{print $1}')
+    rlLog "Latest commit for ${KERNEL_BRANCH} branch: $commit_hash"
+
+    cache_dir="${CACHE_DIR_BASE}${commit_hash}/"
+    if [[ -d "$cache_dir" ]]; then
+        cached_srpm=$(find "$cache_dir" -name "${KERNEL_SRPM_PATTERN}")
+        if [[ -n "$cached_srpm" ]]; then
+            rlLog "Found cached SRPM: $cached_srpm"
+            rlRun "dnf builddep -y $cached_srpm > install-buildreqs.log 2>&1"
+            rlFileSubmit install-buildreqs.log
+            KERNEL_SRPM="$cached_srpm"
+            return 0
+        fi
+    fi
+    rlLog "Cached kernel SRPM not found"
+    return 1
+}
+
+# Attempts to clone the kernel-ark tree in the CWD, at most 3 times.
+# Returns 0 on success, 1 otherwise
+cloneKernelTree() {
+    local retries=3
+    local delay=10
+
+    rlLog "Cloning kernel"
+    while [[ $retries -gt 0 ]]; do
+        rlRun "git clone -q --branch ${KERNEL_BRANCH} ${KERNEL_GIT_URL}" 0-255
+        retcode=$?
+        if [[ $retcode -eq 0 ]]; then
+            return 0
+        else
+            retries=$((retries - 1))
+            rlLog "Clone failed, wait ${delay}s and retry ($retries retries left)"
+            sleep $delay
+        fi
+    done
+    return "$retcode"
+}
+
+# In a kernel-ark repository, do all the operations needed to generate an SRPM
+# If the kernel is generated return the abspath in KERNEL_SRPM global variable
+# Returns 0 on success, 1 otherwise
+generateSRPM() {
+    local generated_srpm
+    local commit_hash
+    local cache_dir
+
+    if ! cloneKernelTree; then
+        rlFail "Failed to clone kernel tree"
+        return 1
+    fi
+
+    rlRun "pushd kernel-ark"
+    rlLog "Gathering and installing missing build requirements"
+    rlRun "make dist-get-buildreqs > make-buildreqs.log 2>&1"
+    rlFileSubmit make-buildreqs.log
+    if grep -q 'Missing dependencies:' make-buildreqs.log; then
+        # Getting the build requirements is quite tricky as it might contain
+        # not only package names but also "provides" perl(ExtUtils::Embed) which
+        # can break the dnf command if not escaped properly. The safest way is
+        # to create an array which contains each req as argument, then pass the
+        # array to dnf. Bash will later pass each element properly quoted.
+        # rlRun can also easily break the command due to special characters
+        # so we don't use it here.
+        read -ra KERNEL_BUILDREQS <<< "$(sed -n 's/Missing dependencies://p' make-buildreqs.log)"
+        rlLog "Installing dependencies: ${KERNEL_BUILDREQS[*]}"
+        dnf install -y "${KERNEL_BUILDREQS[@]}" > install-buildreqs.log 2>&1 || \
+            rlFail "$(cat install-buildreqs.log)"
+        rlFileSubmit install-buildreqs.log
+    elif grep 'PASS:' make-buildreqs.log; then
+        rlLog "All dependencies were already installed"
+    else
+        rlLogWarning "Error getting dependencies, the build might fail"
+        rlRun "cat make-buildreqs.log"
+    fi
+
+    rlLog "Generating SRPM"
+    rlRun "make dist-srpm > dist-srpm.log 2>&1"
+    rlFileSubmit dist-srpm.log
+
+    generated_srpm=$(find "$(pwd)" -name "$KERNEL_SRPM_PATTERN")
+    if [[ -e "$generated_srpm" ]]; then
+        rlLog "Caching generated SRPM"
+        commit_hash=$(git rev-parse HEAD)
+        cache_dir="${CACHE_DIR_BASE}${commit_hash}/"
+        rlRun "mkdir -p $cache_dir"
+        rlRun "cp $generated_srpm $cache_dir"
+        KERNEL_SRPM="$generated_srpm"
+        rlRun "popd"
+        return 0
+    else
+        rlLogWarning "Failed to generate SRPM"
+        rlRun "popd"
+        return 1
+    fi
+}
+
+rlJournalStart
+    rlPhaseStartSetup
+        rlRun "nproc"
+        rlRun "free -h"
+        logTmtRequiredPackages
+
+        declare tmp
+        rlRun "tmp=\$(mktemp -d)" 0 "Create tmp directory"
+        rlRun "pushd $tmp"
+        rlRun "set -o pipefail"
+        if rlIsFedora; then
+            rlLog "Disable updates-testing"
+            rlRun "dnf-3 config-manager --set-disabled updates-testing"
+        fi
+    rlPhaseEnd
+
+    rlPhaseStartTest
+        # Check if kernel package was generated by a previous test run
+        # and generate a new one if it wasn't
+        checkCache
+        if [[ -e "$KERNEL_SRPM" ]]; then
+            rlLog "Using cached kernel SRPM: $KERNEL_SRPM"
+        else
+            rlLog "No kernel SRPM found, generating new SRPM"
+            generateSRPM
+        fi
+
+        if [[ -e "$KERNEL_SRPM" ]]; then
+            BUILD_FLAGS="--target $(uname -m) --with up --with toolchain_clang "
+            BUILD_FLAGS+="--without trace --without arm64_16k --without arm64_64k "
+            BUILD_FLAGS+="--without realtime --without zfcpdump "
+            BUILD_FLAGS+="${CLANG_MODE} ${BUILD_MODE}"
+
+            rlLog "Building kernel (branch $KERNEL_BRANCH)"
+            rlRun "rpmbuild ${BUILD_FLAGS} --rebuild ${KERNEL_SRPM} > build.log 2>&1"
+            rlFileSubmit build.log
+        else
+            rlFail "Kernel SRPM not found, finishing test"
+        fi
+    rlPhaseEnd
+
+    rlPhaseStartCleanup
+        rlRun "popd"
+        rlRun "rm -r $tmp" 0 "Remove tmp directory"
+    rlPhaseEnd
+rlJournalEnd

tests/libomp/main.fmf

@@ -1,26 +1,29 @@
 # The point of this test is to ensure that clang/clang-libs demand the correct
 # libomp packages, and is able to use openmp headers and libraries
 summary: test clang can find libomp header and libraries
-test: "$WITH_SCL ./runtest.sh"
-framework: shell
 tier: 1
-component:
-  - llvm-toolset
-  - clang
 extra-summary: /tools/clang/libomp
 extra-task: /tools/clang/libomp
 extra-nitrate: TC#0614137
+component+:
+  - libomp
 
-require:
-  - clang
+require+:
   - clang-libs
   - libomp
   - libomp-devel
 
-adjust:
-  - because: "libomp is not supported in s390x"
-    when: arch == s390x
+adjust+:
+  # Right now libomp in s390x is supported only in LLVM >= 18 in Fedora
+  # TODO until tmt allows to use custom adjusted contexts we need to do this
+  # indirect adjustment using the distro version.
+  - when: arch == s390x and distro < fedora-40 and snapshot is not defined
     enabled: false
+    continue: false
+
+  - when: arch == s390x and distro != fedora
+    enabled: false
+    continue: false
 
   # Dependencies in rhel-7 are handled differently: there are no recommends,
   # only requires.

tests/libomp/runtest.sh

@@ -0,0 +1,27 @@
+#!/bin/bash
+
+set -exo pipefail
+
+CLANG_PKG=$(rpm -qf --queryformat '%{name}' "$(readlink -f "$(type -p clang)")")
+
+# For compat packages, we want to check if there's a package suffix: clang17 instead clang for example
+PKG_SUFFIX=${CLANG_PKG#clang}
+CLANG_NVR=$(rpm -q "$CLANG_PKG")
+CLANG_VERSION=$(rpm --queryformat="%{version}" -q "$CLANG_NVR")
+LIBOMP_DEPENDENCIES="libomp${PKG_SUFFIX} libomp${PKG_SUFFIX}-devel"
+
+# Ensure clang depends on the correct clang-libs version
+rpm -q --requires "$CLANG_NVR" | grep "${CLANG_PKG}-libs.* = ${CLANG_VERSION}"
+
+# Check that weak dependencies are correct. The versions of these should be the same
+# as clang's to guarantee the ABI compatibility, and that version should be actually
+# installed as well.
+for lomp_dep in $LIBOMP_DEPENDENCIES; do
+    rpm -q --recommends "clang${PKG_SUFFIX}-libs" | grep "${lomp_dep}.* = ${CLANG_VERSION}"
+    [[ "$(rpm --queryformat="%{version}" -q "${lomp_dep}"."$(uname -m)")" == "${CLANG_VERSION}" ]]
+done
+
+# Perform a sanity test to ensure everything works as expected
+clang -fopenmp openmp-compile-link-test.c
+
+./a.out | grep "Num Threads: 1"

tests/llvm-test-suite/main.fmf

@@ -1,16 +1,18 @@
 # TODO: once llvm-test-suite is converted to TMT, we can just link the test from plan.
 summary: Run tests from llvm-test-suite package
 
-adjust:
+adjust+:
   - because: "llvm-test-suite is available for Fedora, not RHEL/CentOS"
     when: >-
       distro == rhel
       or distro == centos
     enabled: false
 
-test: "$WITH_SCL ./test.sh"
+  - because: "llvm test suite not built for compat packages"
+    enabled: false
+    when: compat is defined
 
-require:
+require+:
   - git
   - clang
   - ninja-build

tests/long-double/main.fmf

@@ -0,0 +1,11 @@
+summary: Test that gcc and clang use the same long double format
+duration: 5m
+tier: 1
+extra-summary: /tools/clang/long-double
+extra-task: /tools/clang/long-double
+extra-nitrate: TC#0614593
+
+require+:
+  - gcc
+
+id: 4096a1bb-d2a5-4aeb-b14d-fc465c293e32

tests/long-double/runtest.sh

@@ -1,4 +1,4 @@
-set -e
+#!/bin/bash -e
 
 # Use __LDBL_MANT_DIG__ as a way to distinguish between long double formats.
 # While this is not guaranteed to change for all formats, it provides a

tests/main.fmf

@@ -0,0 +1,45 @@
+# Common configuration for all the tests in this repo
+# All the tmt tests under this directory inherit the settings from this file
+component:
+  - llvm-toolset
+  - clang
+
+# Default QA contact. If more are relevant use 'contact+:' in each test metadata.
+contact:
+  - Jesus Checa Hidalgo <jcheca@redhat.com>
+
+# All tests are shell by default. If a specific test uses another framework
+# such as beakerlib, define "framework: beakerlib" in the test's main.fmf
+framework: shell
+
+# Always define "test" key and override in adjust or in each specific test
+# metadata if needed.
+test: ./runtest.sh
+
+# Commonly used keys are initialized to empty.
+require: []
+link: []
+tag: []
+
+adjust+:
+  # Common requirements when LLVM is not SCL-ized
+  - require+:
+      - clang
+    when: collection is not defined
+
+  # Requirements for SCL-ized LLVM
+  # We need to override the test key to do the proper scl call
+  - test: scl enable llvm-toolset-13.0 -- ./runtest.sh
+    require+:
+      - llvm-toolset-13.0-clang
+    when: collection == llvm-toolset-13.0
+
+  - test: scl enable llvm-toolset-14.0 -- ./runtest.sh
+    require+:
+      - llvm-toolset-14.0-clang
+    when: collection == llvm-toolset-14.0
+
+  - test: scl enable llvm-toolset-15.0 -- ./runtest.sh
+    require+:
+      - llvm-toolset-15.0-clang
+    when: collection == llvm-toolset-15.0

tests/openmp-rpm/main.fmf

@@ -0,0 +1,39 @@
+summary: Test build a simple RPM package to ensure that -fopenmp works
+require+:
+  - rpm-build
+tier: 1
+extra-summary: /tools/clang/openmp-rpm
+extra-task: /tools/clang/openmp-rpm
+extra-nitrate: TC#0614128
+component+:
+  - libomp
+
+adjust+:
+  # Right now libomp in s390x is supported only in LLVM >= 18 in Fedora
+  # TODO until tmt allows to use custom adjusted contexts we need to do this
+  # indirect adjustment using the distro version.
+  - when: arch == s390x and distro < fedora-40 and snapshot is not defined
+    enabled: false
+    continue: false
+
+  - when: arch == s390x and distro != fedora
+    enabled: false
+    continue: false
+
+  - require+:
+      - libomp
+    when: collection is not defined
+
+  # Requirements for SCL-ized LLVM
+  - require+:
+      - llvm-toolset-13.0-build
+      - llvm-toolset-13.0-libomp
+    when: "collection == llvm-toolset-13.0"
+  - require+:
+      - llvm-toolset-14.0-build
+      - llvm-toolset-14.0-libomp
+    when: "collection == llvm-toolset-14.0"
+  - require+:
+      - llvm-toolset-15.0-build
+      - llvm-toolset-15.0-libomp
+    when: "collection == llvm-toolset-15.0"

tests/openmp-rpm/runtest.sh

@@ -0,0 +1,8 @@
+#!/bin/bash
+
+set -ex
+
+# Do not run dnf builddep here. TMT should take care of any packages needed to
+# run the test. If the rpmbuild fails due to missing packages, then fmf metadata
+# needs to be fixed.
+QA_RPATHS=$(( 0x0001 )) rpmbuild --define "_sourcedir $PWD" -bb test.spec

tests/openmp-rpm/test.spec

@@ -7,11 +7,15 @@ Release: 1
 Summary: Test package for checking that RPM packages using -fopenmp build correctly
 License: MIT
 
-BuildRequires: %{?scl_prefix}clang
-BuildRequires: %{?scl_prefix}libomp
+# Do not set buildrequires, tmt should cover any requirements to build.
+# If not, tmt test metadata must be updated.
+# BuildRequires: %{?scl_prefix}clang
+# BuildRequires: %{?scl_prefix}libomp
 
 Source0: test.c
 
+%global debug_package %{nil}
+
 %description
 clang was adding RUNPATH to binaries that use OpenMP, and since RUNPATH
 is prohibited in Fedora builds, this was causing packages using clang

tests/pie-rpm/.gitignore

@@ -0,0 +1,5 @@
+pie
+pie-static
+pie-shared
+pie-no-pie
+pie.o

tests/pie-rpm/main.fmf

@@ -0,0 +1,12 @@
+summary: Check that we build rpms with -pie
+tier: 1
+
+require+:
+  - clang-libs
+  - rpm-build
+  - glibc-static
+
+adjust+:
+  - enabled: false
+    when: distro != fedora or distro < fedora-40
+    because: "We only started doing this in Fedora 40"

tests/pie-rpm/pie.spec

@@ -0,0 +1,71 @@
+%global toolchain clang
+
+Name:           pie
+Version:        1
+Release:        %autorelease
+Summary:        Check that -pie is passed.
+
+License:        MIT
+
+BuildRequires:  clang glibc-static
+
+%description
+Make sure the standard configuration from redhat-rpm-config passes -pie when linking.
+
+%prep
+%setup -c -T
+cat << EOF > pie.c
+int main(int argc, char **argv) {
+  return 0;
+}
+EOF
+
+%build
+
+LDFLAGS="${LDFLAGS} -Werror"
+
+clang ${CFLAGS} -c pie.c -o %{NAME}.o
+
+# Regular
+clang ${LDFLAGS} %{NAME}.o -o %{NAME}
+
+# With -shared
+clang         ${LDFLAGS} -shared %{NAME}.o -o %{NAME}-shared-pre
+clang -shared ${LDFLAGS}         %{NAME}.o -o %{NAME}-shared-post
+
+# With -static
+clang         ${LDFLAGS} -static %{NAME}.o -o %{NAME}-static-pre
+clang -static ${LDFLAGS}         %{NAME}.o -o %{NAME}-static-post
+
+# With -no-pie
+clang         ${LDFLAGS} -no-pie %{NAME}.o -o %{NAME}-no-pie-pre
+clang -no-pie ${LDFLAGS}         %{NAME}.o -o %{NAME}-no-pie-post
+
+%check
+if [[ "$(file ./pie)" != *"pie executable"* ]]; then
+  exit 1
+fi
+
+if [[ "$(file ./pie-shared-pre)" == *"pie executable"* ]]; then
+  exit 1
+fi
+if [[ "$(file ./pie-shared-post)" == *"pie executable"* ]]; then
+  exit 1
+fi
+
+if [[ "$(file ./pie-static-pre)" == *"pie executable"* ]]; then
+  exit 1
+fi
+if [[ "$(file ./pie-static-post)" == *"pie executable"* ]]; then
+  exit 1
+fi
+
+if [[ "$(file ./pie-no-pie-pre)" == *"pie executable"* ]]; then
+  exit 1
+fi
+if [[ "$(file ./pie-no-pie-post)" == *"pie executable"* ]]; then
+  exit 1
+fi
+
+%changelog
+%autochangelog

tests/pie-rpm/runtest.sh

@@ -0,0 +1,9 @@
+#!/bin/bash
+
+set -ex
+set pipefail
+
+#${BUILDDEP_CMD} -y pie.spec
+
+QA_RPATHS=$(( 0x0001 )) rpmbuild -bb ./pie.spec
+

tests/ppc64le-long-double/main.fmf

@@ -0,0 +1,27 @@
+summary: Test binary compatibility of the long double format on ppc64le
+duration: 1h
+tier: 1
+extra-summary: /tools/clang/ppc64le-long-double
+extra-task: /tools/clang/ppc64le-long-double
+extra-nitrate: TC#0614129
+
+adjust+:
+  - because: "Fedora CI runs in x86_64 only, emulate with qemu and mock"
+    require+:
+    - qemu-user-static
+    - mock
+    test: ./runtest-fedora.sh
+    when: distro == fedora and arch == x86_64 and snapshot is not defined
+    continue: false
+
+  - enabled: false
+    when: arch != ppc64le
+
+  - because: "glibc in RHEL older than 9 don't support IEEE 128bit doubles"
+    enabled: false
+    when: distro < rhel-9 or distro < centos-stream-9
+
+  - because: "The test runs natively on RHEL so get gcc to build"
+    require+:
+    - gcc
+    when: distro != fedora or snapshot is defined

tests/ppc64le-long-double/runtest-fedora.sh

@@ -1,8 +1,7 @@
-set -e
+#!/bin/bash -e
 
-fedora_release=`rpm -E %{fedora}`
+fedora_release=$(rpm -E "%{fedora}")
 mock_root=fedora-$fedora_release-ppc64le
-triple=ppc64le-redhat-linux
 
 mock_cmd="mock -r $mock_root --isolation=simple"
 
@@ -12,10 +11,10 @@ run_test () {
 
     echo "Running $test_name"
     echo "Expected output: $expected"
-    actual=$($mock_cmd -q --shell ./$test_name)
+    actual=$($mock_cmd -q --shell "./$test_name")
     echo "Actual output:   $actual"
 
-    if [[ x$expected == x$actual ]]; then
+    if [[ "$expected" == "$actual" ]]; then
         return 0;
     else
         return 1;

tests/ppc64le-long-double/runtest.sh

@@ -1,7 +1,4 @@
-set -e
-
-triple=ppc64le-redhat-linux
-
+#!/bin/bash -e
 
 run_test () {
     test_name=$1
@@ -9,10 +6,10 @@ run_test () {
 
     echo "Running $test_name"
     echo "Expected output: $expected"
-    actual=$(./$test_name)
+    actual=$("./$test_name")
     echo "Actual output:   $actual"
 
-    if [[ x$expected == x$actual ]]; then
+    if [[ "$expected" == "$actual" ]]; then
         return 0;
     else
         return 1;

tests/rhbz_1647130/main.fmf

@@ -1,32 +1,22 @@
 summary: RHBZ1647130
-test: "$WITH_SCL ./runtest.sh"
-require: []
-framework: shell
 tier: 1
-component:
-  - llvm-toolset
-  - clang
 extra-summary: /tools/clang/rhbz_167130
 extra-task: /tools/clang/rhbz_167130
 extra-nitrate: TC#0614130
 
-adjust:
+adjust+:
   # Common requirements when LLVM is not SCL-ized
   - require+:
-      - clang
       - clang-analyzer
     when: "collection is not defined"
 
   # Requirements for SCL-ized LLVM
   - require+:
-      - llvm-toolset-13.0-clang
       - llvm-toolset-13.0-clang-analyzer
     when: "collection == llvm-toolset-13.0"
   - require+:
-      - llvm-toolset-14.0-clang
       - llvm-toolset-14.0-clang-analyzer
     when: "collection == llvm-toolset-14.0"
   - require+:
-      - llvm-toolset-15.0-clang
       - llvm-toolset-15.0-clang-analyzer
     when: "collection == llvm-toolset-15.0"

tests/rhbz_1647130/runtest.sh

@@ -0,0 +1,8 @@
+#!/bin/bash -ex
+
+tmp_cpp=$(mktemp -t XXXXX.cpp)
+tmp_dir=$(mktemp -d)
+echo 'int main(int argc, char*argv[]) { while(argc--) new int(); return 0; }' > "$tmp_cpp"
+scan-build -o "$tmp_dir" clang++ -c "$tmp_cpp" -o /dev/null
+(scan-view --no-browser "$tmp_dir"/* & WPID=$! && sleep 10s && kill $WPID)
+rm -rf "$tmp_cpp" "$tmp_dir"

tests/rhbz_1657544/main.fmf

@@ -0,0 +1,5 @@
+summary: RHBZ1657544
+tier: 1
+extra-summary: /tools/clang/rhbz_165744
+extra-task: /tools/clang/rhbz_165744
+extra-nitrate: TC#0614131

tests/rhbz_2239619/main.fmf

@@ -0,0 +1,2 @@
+summary: Test if debug information is disabled by default
+tier: 1

tests/rhbz_482491/main.fmf

@@ -0,0 +1,18 @@
+summary: rhbz-482491
+description: |
+  "Test that clang is able to find the right libgcc_s"
+require+:
+  - libgcc
+tier: 1
+extra-summary: /tools/clang/rhbz_482194
+extra-task: /tools/clang/rhbz_482194
+extra-nitrate: TC#0614132
+
+link:
+  - relates: https://bugzilla.redhat.com/show_bug.cgi?id=1482491
+
+adjust+:
+  # gcc-x86_64-linux-gnu is not available in RHEL
+  - require+:
+      - gcc-x86_64-linux-gnu
+    when: "distro == fedora"

tests/rpmmacros/main.fmf

@@ -0,0 +1,15 @@
+summary: Test if RPM macros are properly set
+tier: 1
+require+:
+  - clang-devel
+extra-summary: /tools/clang/rpmmacros
+extra-task: /tools/clang/rpmmacros
+
+adjust+:
+  # From spec file:
+  # File in the macros file for other packages to use.  We are not doing this
+  # in the compat package, because the version macros would conflict with
+  # eachother if both clang and the clang compat package were installed together.
+  - because: "macros.clang is not added to compat packages"
+    enabled: false
+    when: compat is defined

tests/rpmmacros/runtest.sh

@@ -1,11 +1,11 @@
-#!/bin/sh -eux
+#!/bin/bash -eux
 
-clang_pkg=${1:-"clang"}
+clang_pkg=${1:-"$(rpm -qf --queryformat '%{name}' "$(readlink -f "$(type -p clang)")")"}
 macros_path="/usr/lib/rpm/macros.d/macros.clang"
 
 set pipefail
 
-if ! rpm -q $clang_pkg > /dev/null; then
+if ! rpm -q "$clang_pkg" > /dev/null; then
     echo "Could not find package $clang_pkg"
     exit 1
 fi
@@ -23,7 +23,7 @@ echo
 # suffix ~rcN.  Meanwhile, the macro won't include it in order to allow packages
 # built with an RC package to be fully supported later.
 # In that case, we need to remove that prefix.
-rpm_version=$(rpm -q $clang_pkg --qf "%{version}" | sed 's/~.*//')
+rpm_version=$(rpm -q "$clang_pkg" --qf "%{version}" | sed 's/~.*//')
 macro_version=$(rpm --eval "%{clang_version}")
 
 if [[ "$rpm_version" != "$macro_version" ]]; then

tests/toolchains/main.fmf

@@ -1,24 +1,17 @@
-# TODO REVIEW: better summary
-summary: ""
-test: "$WITH_SCL ./runtest.sh"
-require:
+summary: "Test integration between clang, compiler-rt and lld"
+require+:
   - glibc-static
   - yum-utils
   # This require EPEL/CRB to be enabled on RHEL
   - libstdc++-static
-framework: shell
 tier: 1
-component:
-  - llvm-toolset
-  - clang
 extra-summary: /tools/clang/toolchains
 extra-task: /tools/clang/toolchains
 extra-nitrate: TC#0614133
 
-adjust:
+adjust+:
   # Common requirements when LLVM is not SCL-ized
   - require+:
-      - clang
       - compiler-rt
       - lld
     when: "collection is not defined"
@@ -40,14 +33,24 @@ adjust:
       - llvm-toolset-15.0-lld
     when: "collection == llvm-toolset-15.0"
 
-    # lld not supported in s390x or ppc64. If any lld package was added to
+    # lld not supported in s390x (LLVM<18) or ppc64. If any lld package was added to
     # requirements, remove it.
   - require-:
       - lld
       - llvm-toolset-13.0-lld
       - llvm-toolset-14.0-lld
       - llvm-toolset-15.0-lld
-    when: arch == s390x or arch == ppc64
+    when: >-
+      arch == s390x and distro < rhel-9, rhel-9.5, fedora-40, centos-stream-9
+      or arch == ppc64
+
+  # LLVM is getting updated to 18 in rhel 8.10 which adds lld for s390x, but we
+  # still have LLVM 17 in AppStream. Install lld if it's available, leave it out
+  # if not. The test will (correctly) fail if we lld is missing in LLVM 18.
+  # TODO once LLVM 18 is in the default module stream we can set this a require:
+  - recommend+:
+      - lld
+    when: arch == s390x and distro == rhel-8.10
 
   - environment+:
       CXXLIBS: "libc++"
@@ -78,7 +81,9 @@ adjust:
       - libcxx-static
     require+:
       - libstdc++
-    when: distro == fedora and snapshot is defined
+      # All archs support lld in snapshots, ensure it's required
+      - lld
+    when: snapshot is defined
     because: llvm-snapshots adjustments
 
 

tests/toolchains/runtest.sh

@@ -1,4 +1,4 @@
-#!/bin/sh -eux
+#!/bin/bash -eu
 
 set pipefail
 
@@ -12,41 +12,40 @@ status=0
 
 test_toolchain() {
 
-    toolchain=$@
+    toolchain=("$@")
     args=""
 
-    while [ $# -gt 0 ]; do
-        case $1 in
+    for arg in "${toolchain[@]}"; do
+        case "$arg" in
             clang)
-                compiler=$1
+                compiler=$arg
                 src=hello.c
                 ;;
             clang++)
-                compiler=$1
+                compiler=$arg
                 src=hello.cpp
                 ;;
             compiler-rt)
-                args="$args -rtlib=$1"
+                args="$args -rtlib=$arg"
                 ;;
             libc++)
-                args="$args -stdlib=$1"
+                args="$args -stdlib=$arg"
                 ;;
             libstdc++)
-                args="$args -stdlib=$1"
+                args="$args -stdlib=$arg"
                 ;;
             lld)
-                args="$args -fuse-ld=$1"
+                args="$args -fuse-ld=$arg"
                 ;;
             *)
-                args="$args $1"
+                args="$args $arg"
                 ;;
         esac
-    shift
     done
 
     cmd="$compiler $args $src"
     rm -f a.out
-    echo "* $toolchain"
+    echo "* ${toolchain[*]}"
     echo "  command: $cmd"
     if $cmd && ./a.out | grep -q 'Hello World'; then
         echo "  PASS"
@@ -57,15 +56,17 @@ test_toolchain() {
 }
 
 clang --version
+CLANG_PKG=$(rpm -qf --queryformat '%{name}' "$(readlink -f "$(type -p clang)")")
 # Repoquery is needed instead yum info for compatibility with RHEL-7
-repoquery -i --installed $(rpm -qf $(which clang)) | grep ^Source
+repoquery -i --installed "$CLANG_PKG" | grep ^Source
+clang_version=$(rpm -q --queryformat "%{version}" "$CLANG_PKG" | grep -ioP "^[0-9]+")
 echo ""
 
 for compiler in clang clang++; do
     for rtlib in "" compiler-rt; do
         for linker in "" lld; do
             for cxxlib in "" $CXXLIBS; do
-                if [ "$compiler" = "clang" -a -n "$cxxlib" ]; then
+                if [[ "$compiler" = "clang" && -n "$cxxlib" ]]; then
                     continue
                 fi
                 for args in "" -static; do
@@ -80,8 +81,14 @@ for compiler in clang clang++; do
                       args="$args -pthread"
                     fi
 
-                    # lld is not supported in s390x and ppc64
-                    if [[ "$(uname -m)" = "s390x" || "$(uname -m)" = "ppc64" ]] \
+                    # lld is not supported in ppc64
+                    if [[ "$(uname -m)" = "ppc64" ]] && [[ "$linker" = "lld" ]];
+                    then
+                      continue
+                    fi
+
+                    # lld is supported in s390x only in LLVM >= 18
+                    if [[ "$(uname -m)" = "s390x" ]] && [[ $clang_version -lt 18 ]] \
                         && [[ "$linker" = "lld" ]];
                     then
                       continue
@@ -92,7 +99,7 @@ for compiler in clang clang++; do
                       continue
                     fi
 
-                    test_toolchain $compiler $rtlib $linker $cxxlib $args
+                    test_toolchain "$compiler" "$rtlib" "$linker" "$cxxlib" "$args"
                 done
             done
         done

tests/ucrt64-toolchain/main.fmf

@@ -1,12 +1,8 @@
 summary: Test that ucrt64 toolchain is detected
-test: ./runtest.sh
-component:
-  - clang
-require:
-  - clang
+require+:
   - ucrt64-gcc-c++
 
-adjust:
+adjust+:
   - enabled: false
     when: distro != fedora or distro < fedora-37
     because: "The ucrt64 toolchain is only available since Fedora 37"

tests/ucrt64-toolchain/runtest.sh

@@ -0,0 +1,4 @@
+#!/bin/bash -eux
+
+clang++ --target=x86_64-windows-gnu test.cpp
+file a.exe | grep "PE32+ executable.*x86-64"

tests/use-correct-dwarf-default/.gitignore

@@ -0,0 +1,2 @@
+a.out
+build.log

tests/use-correct-dwarf-default/main.fmf

@@ -1,20 +1,14 @@
 summary: Test that clang uses DWARFv4 by default
-test: "$WITH_SCL ./test.sh"
-require:
+require+:
   - libgcc
-framework: shell
 tier: 1
-component:
-  - llvm-toolset
-  - clang
-extra-summary: /tools/clang/use-dwarf4-by-default
-extra-task: /tools/clang/use-dwarf4-by-default
+extra-summary: /tools/clang/use-correct-dwarf-default
+extra-task: /tools/clang/use-correct-dwarf-default
 extra-nitrate: TC#0614134
 
-adjust:
+adjust+:
   # Common requirements when LLVM is not SCL-ized
   - require+:
-      - clang
       - llvm
     when: "collection is not defined"
 
@@ -31,3 +25,7 @@ adjust:
       - llvm-toolset-15.0-clang
       - llvm-toolset-15.0-llvm
     when: "collection == llvm-toolset-15.0"
+
+  - environment+:
+      DWARF_VERSION: 4
+    when: distro < rhel-10,centos-stream-10 and snapshot is not defined

tests/use-correct-dwarf-default/runtest.sh

@@ -0,0 +1,20 @@
+#!/bin/bash -eux
+
+# Determine correct DWARF version to use. Defaults to version 5, but older
+# distros might need to use version 4, which can be configured using the
+# DWARF_VERSION env var
+required_dwarf_version=${DWARF_VERSION:-5}
+
+# Get clang version
+CLANG_PKG=$(rpm -qf --queryformat '%{name}' "$(readlink -f "$(type -p clang)")")
+clang_version=$(rpm -q --queryformat "%{version}" "$CLANG_PKG" | grep -ioP "^[0-9]+")
+if  [ "$clang_version" -lt 18 ]; then
+    >&2 echo "clang is older than version 18";
+    required_dwarf_version=4
+fi
+
+echo "int main(){ return 0; }" | clang -g -v -x c - 2> build.log
+# Make sure that clang is using the expected flag to use the required DWARF version
+grep -q "\-dwarf-version=$required_dwarf_version" build.log
+# Inspect the binary to double check expected DWARF version
+llvm-dwarfdump a.out | grep -ioP "version\s*=\s*0x000$required_dwarf_version"

ucrt64-toolchain/runtest.sh

@@ -1,4 +0,0 @@
-set -eux
-
-clang++ --target=x86_64-windows-gnu test.cpp
-file a.exe | grep "PE32+ executable (console) x86-64, for MS Windows"

use-dwarf4-by-default/test.sh

@@ -1,7 +0,0 @@
-#!/bin/sh -eux
-
-echo "int main(){ return 0; }" | clang -g -v -x c - 2> build.log
-# Make sure that clang is using the expected flag to use DWARF 4
-grep -q "\-dwarf-version=4" build.log
-# Inspect the binary to double check expected DWARF version
-llvm-dwarfdump a.out | grep -i version | grep 0x0004