tests/llvm
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
Tests for llvm
27 commits 3 branches 0 tags 345 KiB
  • Shell 81.1%
  • C 7.7%
  • CMake 4%
  • C++ 3.6%
  • Python 3.3%
  • Other 0.3%
Find a file
Jesus Checa Hidalgo 6ad9cfeb01 Convert integration-test-suite into a virtual test 
Split tmt metadata into several tests based on the package set
to be tested, and select one or another depending on the tmt context.
Created subtests for devtools, rhel, fedora and snapshots.
2024-04-26 07:24:38 +00:00
.fmf tests: Add llvm dist-git tests 2022-08-16 16:37:00 +02:00
plans Adapt build-gating.fmf plan to new metadata 2024-02-29 17:16:28 +01:00
tests Convert integration-test-suite into a virtual test 2024-04-26 07:24:38 +00:00
README.md Fix readme formatting 2024-04-02 08:58:31 +02:00

README.md

llvm Tests

This repository contains tests for llvm.

Usage

Tests are executed using TMT (Test Management Tool). The repository includes a TMT plan that runs all the tests by default. All the tests support the following context dimensions:

  • distro
  • arch
  • collection (only applicable to RHEL 7)

To run the tests:

# Run in the default virtual environment
tmt -c distro=<distro> -c arch=<arch> run -avv

# Run on a scheduled system
tmt -c distro=<distro> -c arch=<arch> run -avv provision -h connect -g <ip_or_hostname>

# Run specific tests
tmt -c distro=<distro> -c arch=<arch> run -avv provision -h connect -g <ip_or_hostname> test -n <regex>

Adding New Tests

Please consider the following when adding new tests:

  • Test scripts must be named test.sh. If a different test is needed, you need to overwrite the test: key in the test main.fmf.
  • Tests must be placed under tests/<test_name>. Additionally they can be placed into subdirectories if grouping make sense. E.g. tests/regression/<test_name>.
  • This repository has a common configuration under tests/main.fmf, inherited by all tests. When adding new ones, be careful not to overwrite existing keys from the inherited tests/main.fmf.
  • When setting FMF metadata keys, do merge values instead overwriting them. E.g. use require+: instead require:. For advanced use and more examples check FMF documentation.

After creating the new test, consider running tmt <context> tests show <test-name> to review that the resulting metadata such as requires, enable or test keys are as expected. For example:

$ tmt -c distro=fedora-38 -c arch=aarch64 tests show binutils-plugin-ar
/tests/binutils-plugin-ar
                 summary Make sure the the LLVM plugin for binutils' ar works
             description Test that ar picks the correct plugin when archiving
                         LLVM bitcode.
                 contact Jesus Checa Hidalgo <jcheca@redhat.com>
               component 'llvm-toolset' and 'llvm'
                    test ./test.sh
                    path /tests/binutils-plugin-ar
               framework shell
                  manual false
                     tty false
                 require binutils
                         llvm
                         llvm-libs
                         llvm-devel
                         clang
                duration 1m
                 enabled true
                  result respect
                    tier 1

Templates

Use the following templates when creating new tests.

FMF template. Uncomment needed fields, remove the rest.

summary: Descriptive summary for the test
# Maximum duration for the test
duration: 2m
# Short tests should be tier <= 1 so they're used for gating.
tier: 1
# Add an additional contact(s) if applicable
# contact+:
#   - Somebody <somebody@redhat.com>
# Additional requires
# require+:
#   - rpm-build
# Adjustments based on context dimensions
# adjust+:
#   - when: collection is defined
#     enabled: false
#     because: Test not supported in collections
# link+:
#   - verifies: https://issues.redhat.com/browse/...
# tag+:
#   - sometag