Add documentation to README.md

README.md

@@ -0,0 +1,99 @@
+# clang tests
+
+This repository contains tests for clang.
+
+## Usage
+Tests are executed using TMT (Test Management Tool). The repository includes a
+TMT plan that runs all the tests by default. All the tests support the following
+context dimensions:
+* `distro`
+* `arch`
+* `collection` (only applicable to RHEL 7)
+
+To run the tests:
+```bash
+# Run in the default virtual environment
+tmt -c distro=<distro> -c arch=<arch> run -avv
+
+# Run on a scheduled system
+tmt -c distro=<distro> -c arch=<arch> run -avv provision -h connect -g <ip_or_hostname>
+
+# Run specific tests
+tmt -c distro=<distro> -c arch=<arch> run -avv provision -h connect -g <ip_or_hostname> test -n <regex>
+```
+
+## Adding New Tests
+Please consider the following when adding new tests:
+* Test scripts must be named `runtest.sh`. If a different test is needed, you need
+to overwrite the `test:` key in the test `main.fmf`.
+* Tests must be placed under `tests/<test_name>`. Additionally they can be
+  placed into subdirectories if grouping make sense. E.g.
+  `tests/regression/<test_name>`.
+* This repository has a common configuration under `tests/main.fmf`, inherited
+  by all tests. When adding new ones, be careful not to overwrite existing keys
+  from the inherited `tests/main.fmf`.
+* When setting FMF metadata keys, do merge values instead overwriting them. E.g.
+use `require+:` instead `require:`. For advanced use and more examples check
+[FMF documentation](https://fmf.readthedocs.io/en/stable/features.html#merging).
+
+After creating the new test, consider running `tmt <context> tests show
+<test-name>` to review that the resulting metadata such as requires, enable or
+test keys are as expected. For example:
+```
+$ tmt -c distro=fedora-38 tests show ucrt64-toolchain
+/tests/ucrt64-toolchain
+                 summary Test that ucrt64 toolchain is detected
+                 contact Jesus Checa Hidalgo <jcheca@redhat.com>
+               component 'llvm-toolset' and 'clang'
+                    test ./runtest.sh
+                    path /tests/ucrt64-toolchain
+               framework shell
+                  manual false
+                     tty false
+                 require 'ucrt64-gcc-c++' and 'clang'
+                duration 5m
+                 enabled true
+                  result respect
+
+$ tmt -c distro=rhel-9.3 tests show ucrt64-toolchain
+/tests/ucrt64-toolchain
+                 summary Test that ucrt64 toolchain is detected
+                 contact Jesus Checa Hidalgo <jcheca@redhat.com>
+               component 'llvm-toolset' and 'clang'
+                    test ./runtest.sh
+                    path /tests/ucrt64-toolchain
+               framework shell
+                  manual false
+                     tty false
+                 require 'ucrt64-gcc-c++' and 'clang'
+                duration 5m
+                 enabled false
+                  result respect
+```
+
+### Templates
+Use the following templates when creating new tests.
+
+FMF template. Uncomment needed fields, remove the rest.
+```yaml
+summary: Descriptive summary for the test
+# Maximum duration for the test
+duration: 2m
+# Short tests should be tier <= 1 so they're used for gating.
+tier: 1
+# Add an additional contact(s) if applicable
+# contact+:
+#   - Somebody <somebody@redhat.com>
+# Additional requires
+# require+:
+#   - rpm-build
+# Adjustments based on context dimensions
+# adjust+:
+#   - when: collection is defined
+#     enabled: false
+#     because: Test not supported in collections
+# link+:
+#   - verifies: https://issues.redhat.com/browse/...
+# tag+:
+#   - sometag
+```