The plan "modified" selects only the tests that were edited relative to the
reference branch. However, when a pull-request has only changed files such as
`README.md`, the tmt's modified plan's discover process doesn't detect any tests,
showing "summary: 0 tests selected" and tmt exits as exit status 3. And Fedora
CI fails marking failure.
Below is the tmt command to emulate the case of "0 tests selected" before this
change.
tmt trigger:
https://tmt.readthedocs.io/en/stable/spec/context.html#trigger
```
$ rpm -q tmt
tmt-1.67.0-1.fc43.noarch
$ tmt -c distro=fedora-rawhide -c trigger=commit run -a \
provision -h virtual -i fedora-rawhide
/var/tmp/tmt/run-065
/plans/modified
discover
directory: /home/jaruga/var/pkg/fed/tests/ruby
how: fmf
modified-url: https://src.fedoraproject.org/tests/ruby
modified-ref: reference/main
summary: 0 tests selected
warning: No tests found, finishing plan.
total: no results found
$ echo $?
3
```
Here is the case.
https://src.fedoraproject.org/tests/ruby/pull-request/35#comment-304383
This commit is to add a dummy plan and test which always runs a test in
addition to the plan "modified" to make CI pass in the case of the 0 tests
selected in the fmf tests.
Note that the plan "modified" is still selected in CI which defines the context
variable `trigger` with value `commit` after this change.
Note there are the following tickets to solve this issue, to pass CI on no
tests selected in the upstream tmt and Fedora Forge which is a coming successor
of Fedora Package Sources powered by Pagure.
So, this commit is a temporary workaround until this issue is solved by
these projects.
* Upstream tmt: https://github.com/teemtee/tmt/issues/4550
* Fedora Forge: https://forge.fedoraproject.org/ci/tickets/issues/542
* Update the plan /plans/modified to run in CI. See the following link for the
condition `trigger == commit`.
https://tmt.readthedocs.io/en/stable/spec/context.html#trigger
* Add the plan /plans/all to run all the tests. The plan is not executed in CI
as a default. Don't trigger this plan in CI to save the infra resource and
the running time.
Note we don't describe the triggered environment such as local or CI in the
`summary` element in the plan /plans/all of the `plans.fmf`. Because this
plan is executed as a default, and users can override the triggered
environment by running the `tmt` with `trigger` option, such as
`tmt -c trigger=commit`. We don't want to give users the impression that the
plan only works in a specific running environment.
* Update the `README.md` removing the parts of the `plans --default` option in
the `tmt run` commands that are not needed due to the plan /plans/all.
Co-authored-by: Lukáš Zachar <lzachar@redhat.com>
