Add a plan to run all the tests.

* Update the plan /plans/modified to run in CI. See the following link for the
  condition `trigger == commit`.
  https://tmt.readthedocs.io/en/stable/spec/context.html#trigger
* Add the plan /plans/all to run all the tests. The plan is not executed in CI
  as a default. Don't trigger this plan in CI to save the infra resource and
  the running time.
  Note we don't describe the triggered environment such as local or CI in the
  `summary` element in the plan /plans/all of the `plans.fmf`. Because this
  plan is executed as a default, and users can override the triggered
  environment by running the `tmt` with `trigger` option, such as
  `tmt -c trigger=commit`. We don't want to give users the impression that the
  plan only works in a specific running environment.
* Update the `README.md` removing the parts of the `plans --default` option in
  the `tmt run` commands that are not needed due to the plan /plans/all.

Co-authored-by: Lukáš Zachar <lzachar@redhat.com>
This commit is contained in:
Lukáš Zachar 2024-09-24 15:21:39 +02:00 committed by Jun Aruga
commit ec8759b052
2 changed files with 18 additions and 3 deletions

View file

@ -23,7 +23,7 @@ $ sudo dnf install tmt+all
Run the following command to run the tests for Fedora rawhide:
```
$ tmt -c distro=fedora-rawhide run --all provision -h virtual -i fedora-rawhide plans --default
$ tmt -c distro=fedora-rawhide run --all provision -h virtual -i fedora-rawhide
/var/tmp/tmt/run-NNN
...
```
@ -33,7 +33,7 @@ The tmt's provisioning method virtual (`tmt run provision -h virtual`) must be u
Run the following command to see the report, setting the value of the above run ID to the `--id` option. As this repository doesn't have tmt's plans to run the tests, we need to set the `plans --default` to use the default plans. See [this tmt's document](https://tmt.readthedocs.io/en/stable/stories/cli.html#report) about the `tmt run report` options.
```
$ tmt run --id run-NNN report -vvv plans --default
$ tmt run --id run-NNN report -vvv
```
#### Running a specific test
@ -57,5 +57,5 @@ $ tmt test ls
Run the following command to run a specific test with `-n` option to specify a regular expression to match the test name for Fedora rawhide:
```
$ tmt -c distro=fedora-rawhide run --all provision -h virtual -i fedora-rawhide plans --default tests -n '^/systemtap-static-probes-in-ruby$'
$ tmt -c distro=fedora-rawhide run --all provision -h virtual -i fedora-rawhide tests -n '^/systemtap-static-probes-in-ruby$'
```

View file

@ -5,8 +5,23 @@ execute:
how: tmt
/modified:
summary: Run only modified tests in CI.
enabled: false
adjust:
# https://tmt.readthedocs.io/en/stable/spec/context.html#trigger
- when: trigger == commit
enabled: true
discover:
how: fmf
modified-only: true
modified-ref: reference/main
modified-url: https://src.fedoraproject.org/tests/ruby
/all:
summary: Run all the tests.
adjust:
# Don't trigger this plan in CI.
- when: trigger == commit
enabled: false
discover:
how: fmf