Commit 20a71e4
committed
runtimetest: Make TAP output more granular
Granular TAP output makes it easier to see what's being tested (and
what's going wrong). You'll want a TAP harness looking for test
failures, because runtimetest now returns a lot of information, most
of which is (hopefully ;) about passing tests. In order to accomplish
this change, I've setup a complianceTester structure to make it easy
to pass the TAP harness down into the helper functions.
complianceTester also holds the target compliance level, so it can
decide whether a spec violation is sufficiently serious to be a failed
test violoation or is minor enough to be a warning. I've used skips
for the warnings, since they seemed like the closest fit from the TAP
spec.
And I've made a number of other improvements while I was working
through this, but haven't written them all up in this commit message.
One change is the new support for testing file write access;
previously we only tested directory write access. Its possible that
some WriteFile errors should kill the test suite, but for now I'm
counting all of them as successful "cannot write" determinations.
Signed-off-by: W. Trevor King <wking@tremily.us>1 parent fcd5b23 commit 20a71e4
2 files changed
Lines changed: 666 additions & 353 deletions
0 commit comments