From 1eee8b0c2daa5d6370fe91d16a47566eadca135c Mon Sep 17 00:00:00 2001
From: Jesus Checa Hidalgo <jchecahi@redhat.com>
Date: Fri, 8 Mar 2024 14:43:00 +0100
Subject: [PATCH] Add documentation to README.md

---
 README.md | 93 +++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 91 insertions(+), 2 deletions(-)

diff --git a/README.md b/README.md
index 5129148..0001fca 100644
--- a/README.md
+++ b/README.md
@@ -1,3 +1,92 @@
-# llvm
+# llvm Tests
 
-Tests for llvm
\ No newline at end of file
+This repository contains tests for llvm.
+
+## Usage
+Tests are executed using TMT (Test Management Tool). The repository includes a
+TMT plan that runs all the tests by default. All the tests support the following
+context dimensions:
+* `distro`
+* `arch`
+* `collection` (only applicable to RHEL 7)
+
+To run the tests:
+```bash
+# Run in the default virtual environment
+tmt -c distro=<distro> -c arch=<arch> run -avv
+
+# Run on a scheduled system
+tmt -c distro=<distro> -c arch=<arch> run -avv provision -h connect -g <ip_or_hostname>
+
+# Run specific tests
+tmt -c distro=<distro> -c arch=<arch> run -avv provision -h connect -g <ip_or_hostname> test -n <regex>
+```
+
+## Adding New Tests
+Please consider the following when adding new tests:
+* Test scripts must be named `test.sh`. If a different test is needed, you need
+to overwrite the `test:` key in the test `main.fmf`.
+* Tests must be placed under `tests/<test_name>`. Additionally they can be
+  placed into subdirectories if grouping make sense. E.g.
+  `tests/regression/<test_name>`.
+* This repository has a common configuration under `tests/main.fmf`, inherited
+  by all tests. When adding new ones, be careful not to overwrite existing keys
+  from the inherited `tests/main.fmf`.
+* When setting FMF metadata keys, do merge values instead overwriting them. E.g.
+use `require+:` instead `require:`. For advanced use and more examples check
+[FMF documentation](https://fmf.readthedocs.io/en/stable/features.html#merging).
+
+After creating the new test, consider running `tmt <context> tests show
+<test-name>` to review that the resulting metadata such as requires, enable or
+test keys are as expected. For example:
+```
+$ tmt -c distro=fedora-38 -c arch=aarch64 tests show binutils-plugin-ar
+/tests/binutils-plugin-ar
+                 summary Make sure the the LLVM plugin for binutils' ar works
+             description Test that ar picks the correct plugin when archiving
+                         LLVM bitcode.
+                 contact Jesus Checa Hidalgo <jcheca@redhat.com>
+               component 'llvm-toolset' and 'llvm'
+                    test ./test.sh
+                    path /tests/binutils-plugin-ar
+               framework shell
+                  manual false
+                     tty false
+                 require binutils
+                         llvm
+                         llvm-libs
+                         llvm-devel
+                         clang
+                duration 1m
+                 enabled true
+                  result respect
+                    tier 1
+
+```
+
+### Templates
+Use the following templates when creating new tests.
+
+FMF template. Uncomment needed fields, remove the rest.
+```yaml
+summary: Descriptive summary for the test
+# Maximum duration for the test
+duration: 2m
+# Short tests should be tier <= 1 so they're used for gating.
+tier: 1
+# Add an additional contact(s) if applicable
+# contact+:
+#   - Somebody <somebody@redhat.com>
+# Additional requires
+# require+:
+#   - rpm-build
+# Adjustments based on context dimensions
+# adjust+:
+#   - when: collection is defined
+#     enabled: false
+#     because: Test not supported in collections
+# link+:
+#   - verifies: https://issues.redhat.com/browse/...
+# tag+:
+#   - sometag
+```