Commit graph

9 commits

Author SHA1 Message Date
Petr Lautrbach
1d45ca7dec checkmodule/checkmodule: Grep for error messages rather than debug logs
checkpolicy was updated in 2.9 in order not to write debug messages on ouput. It
doesn't output "loading policy ..." messages anymore.

At the same time, it should be enough to check the checkmodule exit status and
whether the OUTPUT_FILE exists

See 854fdc1ac4

Fixes:
[   PASS   ] :: Command 'checkmodule >& /tmp/tmp.GUkhm09UGI' (Expected 1, got 1)
[   FAIL   ] :: File '/tmp/tmp.GUkhm09UGI' should contain 'loading policy configuration from policy.conf'
[   PASS   ] :: Command 'checkmodule -b >& /tmp/tmp.GUkhm09UGI' (Expected 1, got 1)
[   FAIL   ] :: File '/tmp/tmp.GUkhm09UGI' should contain 'loading policy configuration from policy'
[  BEGIN   ] :: Running 'checkmodule -m -o mypolicy.output mypolicy.te 2>&1 | grep "checkmodule.*loading policy configuration from mypolicy.te"'
[   FAIL   ] :: Command 'checkmodule -m -o mypolicy.output mypolicy.te 2>&1 | grep "checkmodule.*loading policy configuration from mypolicy.te"' (Expected 0, got 1)
[  BEGIN   ] :: Running 'checkmodule -m -o mypolicy.output mypolicy.te 2>&1 | grep "checkmodule.*writing binary representation.*to mypolicy.output"'
[   FAIL   ] :: Command 'checkmodule -m -o mypolicy.output mypolicy.te 2>&1 | grep "checkmodule.*writing binary representation.*to mypolicy.output"' (Expected 0, got 1)
[  BEGIN   ] :: Running 'checkmodule -m -C -o mypolicy.output mypolicy.te 2>&1 | grep "checkmodule.*loading policy configuration from mypolicy.te"'
[   FAIL   ] :: Command 'checkmodule -m -C -o mypolicy.output mypolicy.te 2>&1 | grep "checkmodule.*loading policy configuration from mypolicy.te"' (Expected 0, got 1)
[  BEGIN   ] :: Running 'checkmodule -m -C -o mypolicy.output mypolicy.te 2>&1 | grep "checkmodule.*writing CIL to mypolicy.output"'
[   FAIL   ] :: Command 'checkmodule -m -C -o mypolicy.output mypolicy.te 2>&1 | grep "checkmodule.*writing CIL to mypolicy.output"' (Expected 0, got 1)
2019-04-09 15:32:29 +02:00
Petr Lautrbach
a207b0ec49 checkpolicy/checkpolicy: Grep for error messages rather than debug logs
checkpolicy was updated in 2.9 in order not to write debug messages on ouput. It
doesn't output "loading policy ..." messages anymore.

See 854fdc1ac4

Fixes:
[   PASS   ] :: Command 'checkpolicy >& /tmp/tmp.NMYnXbEmez' (Expected 1, got 1)
[   FAIL   ] :: File '/tmp/tmp.NMYnXbEmez' should contain 'loading policy configuration from policy.conf'
[   PASS   ] :: Command 'checkpolicy -b >& /tmp/tmp.NMYnXbEmez' (Expected 1, got 1)
[   FAIL   ] :: File '/tmp/tmp.NMYnXbEmez' should contain 'loading policy configuration from policy'
2019-04-09 15:23:30 +02:00
Petr Lautrbach
b41ff2e4c1 Fix path meta data
Commit 769880f7 added path meta data into .fmf files. As path's didn't have /
at the beginning test were not created correctly.

Fixes:
fatal: [fedora29]: FAILED! => {
    "msg": [
        "Tests failed: True",
        "Tests msg: FAIL selinuxselinux-policy/policy-rpm-macros",
        "FAIL selinuxcheckpolicy/checkpolicy",
        "FAIL selinuxlibsepol/sepol_check_context",
        "FAIL selinuxlibsemanage/verify-options-in-semanage-conf",
...

$ cat artifacts/FAIL_selinuxcheckpolicy-checkpolicy.log
FAIL test selinuxcheckpolicy/checkpolicy does not appear to be a file or directory
2018-10-14 21:52:42 +02:00
Petr Lautrbach
66a56a3e9b Revert "Fix path meta data"
This reverts commit 4cf575fc0b.

Some of paths were not correctly updated

Fixes:
        "FAIL selinux/olicycoreutils/booleans",
2018-10-14 21:48:49 +02:00
Petr Lautrbach
4cf575fc0b Fix path meta data
Commit 769880f7 added path meta data into .fmf files. As path's didn't have /
at the beginning test were not created correctly.

Fixes:
fatal: [fedora29]: FAILED! => {
    "msg": [
        "Tests failed: True",
        "Tests msg: FAIL selinuxselinux-policy/policy-rpm-macros",
        "FAIL selinuxcheckpolicy/checkpolicy",
        "FAIL selinuxlibsepol/sepol_check_context",
        "FAIL selinuxlibsemanage/verify-options-in-semanage-conf",
...

$ cat artifacts/FAIL_selinuxcheckpolicy-checkpolicy.log
FAIL test selinuxcheckpolicy/checkpolicy does not appear to be a file or directory
2018-10-14 21:31:10 +02:00
Petr Šplíchal
769880f739 Define path metadata for each test
Adding attribute "path" to each test according to the CI metadata
specification [1]. This also resolves problem with autodetecting
empty directories [2] which will be fixed in the next fmf release.

[1] https://fedoraproject.org/wiki/CI/Metadata#Path
[2] https://github.com/psss/fmf/pull/48
2018-10-11 10:45:08 +02:00
Petr Šplíchal
40894f79d3 Update tier metadata to the latest specification
There is now a dedicated field to store the tier info:
https://fedoraproject.org/wiki/CI/Metadata#Tier
Also added some missing component data.
2018-09-20 17:08:10 +02:00
Petr Šplíchal
3b2fd40114 Add initial test metadata using the FMF format
This adds an initial set of test metadata using the Flexible Metadata
Format for experimenting. This should allow us to more easily execute
relevant test coverage for all selinux-related components. Example:

    fmf --key test --filter 'tags:Tier1'
    fmf --key test --filter 'component:libselinux'
    fmf --key test --filter 'component:libselinux | tags:Tier1'

See https://github.com/psss/fmf for more details about the FMF format.
2018-02-19 15:05:29 +01:00
Petr Lautrbach
d731f78c63 Add tests from https://src.fedoraproject.org/rpms/checkpolicy/blob/master/f/tests 2018-02-14 13:39:35 +01:00