# rubygem-cucumber stack test suite

Test suite for th rubygem-cucumber stack to ensure functional Ruby cucumber test runner.

## Using test plans for rubygem packages

Simply use the [plan import](https://tmt.readthedocs.io/en/stable/spec/plans.html#import-plans)
TMT feature.

Let's say we want to test `rubygem-cucumber-gherkin` package, we will use the following TMT plan:
```yaml
# plans/all.fmf
plan:
  import:
    url: https://src.fedoraproject.org/tests/rubygem-cucumber
    name: /plans/full/all
```
This will use the generic plan that runs all the tests described in the Layout section.

Next, after the plan is comitted, we run the in the command-line:
```console
$ tmt run --environment KOJI_TASK_ID=120746946 --all
```
This will run all steps with global defaults for the build [`120746946`](https://koji.fedoraproject.org/koji/taskinfo?taskID=120746946)
which is a regular Koji build where we can source required package for reverse dependency build and
isntall tests.

## Layout

#### The `BUILD_PACKAGE` variable

Some of the tests use `BUILD_PACKAGE` variable.

This variable is mainly used in tests to define which package to build/install as part of the test
to verify the behavior.

Nearly all metadata is inherited via [tmt inheritance](https://tmt.readthedocs.io/en/latest/guide.html#inheritance)
so individual tests only need to define the `BUILD_PACKAGE` variable in the `main.fmf` test plan.

Example:
```yaml
# file rubygem-rspec.fmf
environment:
  BUILD_PACKAGE: rubygem-rspec
```

### Basic Test Suites

Basic high-level tests of ensuring functional rubygem-cucumber. Tests include running a test suite of
an upstream project or running a test suite containing Aruba test cases.

### Reverse Dependencies Build

Set of tests ensuring that the main users of rubygem-cucumber (such as rubygem-rspec) continue
building after a rubygem-cucumber stack's dependency update.

A failure in this class means we broke build of reverse dependencies.

### Reverse Dependencies Install

Ensure that the main users of the rubygem-cucumber dependencies at runtime (rubygem-cucumber and rubygem-aruba)
still install with the updated dependency.

A failure in this class means we broke ability to use cucumber in any context with the update.

## Running

Most tests, especially `reverse_dependencies*` tests rely on an environment variable:

* `KOJI_TASK_ID` -- build result (built RPMs) from koji build. Usually provided by Testing Farm in
  Fedora CI

### The `KOJI_TASK_ID` variable

This variable is expected to be provided via Testing Farm executed in a Fedora PR after a passing
scratch build. It has to be a koji task ID, so for local runs, any Task ID is enough.

To execute tests locally we will either need to have a scratch build task ID can utilize production builds.

See section above *Using test plans for rubygem packages* for practical example.