80 lines
2.8 KiB
Markdown
80 lines
2.8 KiB
Markdown
# rubygem-cucumber stack test suite
|
|
|
|
Test suite for th rubygem-cucumber stack to ensure functional Ruby cucumber test runner.
|
|
|
|
## Using test plans for rubygem packages
|
|
|
|
Simply use the [plan import](https://tmt.readthedocs.io/en/stable/spec/plans.html#import-plans)
|
|
TMT feature.
|
|
|
|
Let's say we want to test `rubygem-cucumber-gherkin` package, we will use the following TMT plan:
|
|
```yaml
|
|
# plans/all.fmf
|
|
plan:
|
|
import:
|
|
url: https://src.fedoraproject.org/tests/rubygem-cucumber
|
|
name: /plans/full/all
|
|
```
|
|
This will use the generic plan that runs all the tests described in the Layout section.
|
|
|
|
Next, after the plan is comitted, we run the in the command-line:
|
|
```console
|
|
$ tmt run --environment KOJI_TASK_ID=120746946 --all
|
|
```
|
|
This will run all steps with global defaults for the build [`120746946`](https://koji.fedoraproject.org/koji/taskinfo?taskID=120746946)
|
|
which is a regular Koji build where we can source required package for reverse dependency build and
|
|
isntall tests.
|
|
|
|
## Layout
|
|
|
|
#### The `BUILD_PACKAGE` variable
|
|
|
|
Some of the tests use `BUILD_PACKAGE` variable.
|
|
|
|
This variable is mainly used in tests to define which package to build/install as part of the test
|
|
to verify the behavior.
|
|
|
|
Nearly all metadata is inherited via [tmt inheritance](https://tmt.readthedocs.io/en/latest/guide.html#inheritance)
|
|
so individual tests only need to define the `BUILD_PACKAGE` variable in the `main.fmf` test plan.
|
|
|
|
Example:
|
|
```yaml
|
|
# file rubygem-rspec.fmf
|
|
environment:
|
|
BUILD_PACKAGE: rubygem-rspec
|
|
```
|
|
|
|
### Basic Test Suites
|
|
|
|
Basic high-level tests of ensuring functional rubygem-cucumber. Tests include running a test suite of
|
|
an upstream project or running a test suite containing Aruba test cases.
|
|
|
|
### Reverse Dependencies Build
|
|
|
|
Set of tests ensuring that the main users of rubygem-cucumber (such as rubygem-rspec) continue
|
|
building after a rubygem-cucumber stack's dependency update.
|
|
|
|
A failure in this class means we broke build of reverse dependencies.
|
|
|
|
### Reverse Dependencies Install
|
|
|
|
Ensure that the main users of the rubygem-cucumber dependencies at runtime (rubygem-cucumber and rubygem-aruba)
|
|
still install with the updated dependency.
|
|
|
|
A failure in this class means we broke ability to use cucumber in any context with the update.
|
|
|
|
## Running
|
|
|
|
Most tests, especially `reverse_dependencies*` tests rely on an environment variable:
|
|
|
|
* `KOJI_TASK_ID` -- build result (built RPMs) from koji build. Usually provided by Testing Farm in
|
|
Fedora CI
|
|
|
|
### The `KOJI_TASK_ID` variable
|
|
|
|
This variable is expected to be provided via Testing Farm executed in a Fedora PR after a passing
|
|
scratch build. It has to be a koji task ID, so for local runs, any Task ID is enough.
|
|
|
|
To execute tests locally we will either need to have a scratch build task ID can utilize production builds.
|
|
|
|
See section above *Using test plans for rubygem packages* for practical example.
|