APITest
Test
Organizing and running tests.
A suite is a tree: describe groups tests, test names a body that
states Expect expectations, and skip, only, and
todo mark work that is not run. A program tests itself with
main() = Test.run suite:
import Expect
import String
import Test exposing (Test, describe, test)
suite : Test {}
suite =
describe "String.split"
[ test "splits on the separator" {
Expect.equal [ "a", "b" ] (String.split "," "a,b")
Expect.equal [ "" ] (String.split "," "")
}
, test "keeps the text without a separator" {
String.split "" "abc" |> Expect.equal [ "abc" ]
}
]
True
execute runs without IO and reports structured progress through the
Reporter effect. Handle it with silent to get only the summary,
console for the default output, or a custom handler for other formats.
run selects a format from command-line arguments, so it performs
Process.Environment, and applies the process exit policy; without options
it uses console. Optional JSON reports perform File.FileSystem; a
caller can capture them with File.runMemory.
By default, run prints its report to standard output. Each failure is
printed in full, the heading naming the path through the groups, and a
summary ends the report:
FAIL String.split > splits on the separator
StringTests.fango:12
Expected: ["a", "b"]
Actual: ["a"]
TEST RUN FAILED
Passed: 3
Failed: 1
Skipped: 1
Todo: 1
The summary line reads TEST RUN PASSED when every test passed,
TEST RUN FAILED when one failed, and TEST RUN INCOMPLETE when none
failed but the suite contains skipped leaves, only, or todo. Only a
passed and complete run exits with status 0; the others exit with status 1,
so a placeholder cannot make a build green.
A group fails, without stopping its tests, when it has no tests, an empty name, a test with an empty name, or two tests with the same name. A runtime crash inside a test body ends the whole run: there is no Fango level catch for it.
CaseOutcome
type CaseOutcome = CasePassed | CaseFailed Error | CaseSkipped | CaseTodoSource
Re-exported from Test.Reporter.
The outcome of a leaf in the suite.
Constructors
CaseFailed
CaseFailed : Error -> CaseOutcomeSource
Re-exported from Test.Reporter.
The body’s first failing expectation.
CasePassed
CasePassed : CaseOutcomeSource
Re-exported from Test.Reporter.
The body completed without a failing expectation.
CaseSkipped
CaseSkipped : CaseOutcomeSource
Re-exported from Test.Reporter.
The body or placeholder was skipped explicitly or by focus.
CaseTodo
CaseTodo : CaseOutcomeSource
Re-exported from Test.Reporter.
A placeholder with no body, not explicitly skipped.
Instances:
Eq CaseOutcome
Show CaseOutcome
Event
type Event = SuiteStarted Int | CaseStarted (List String) | CaseFinished (List String) CaseOutcome | GroupFailed (List String) String | SuiteFinished SummarySource
Re-exported from Test.Reporter.
A synchronous notification in depth-first tree order. Paths contain group and leaf names from outermost to innermost, without formatting.
Constructors
CaseFinished
CaseFinished : List String -> CaseOutcome -> EventSource
Re-exported from Test.Reporter.
A leaf completed, failed, was skipped, or is a todo. Skipped leaves
and todos have no preceding CaseStarted.
CaseStarted
CaseStarted : List String -> EventSource
Re-exported from Test.Reporter.
An active body is about to run.
GroupFailed
GroupFailed : List String -> String -> EventSource
Re-exported from Test.Reporter.
One group validation problem, reported before visiting its children.
SuiteStarted
SuiteStarted : Int -> EventSource
Re-exported from Test.Reporter.
The run begins, with its total number of leaves, including todos and skipped leaves. Group validation problems are not included.
SuiteFinished
SuiteFinished : Summary -> EventSource
Re-exported from Test.Reporter.
The completed summary, identical to the value returned by execute.
Instances:
Eq Event
Show Event
Summary
type Summary = { passed : Int , failed : Int , skipped : Int , todo : Int , verdict : Verdict }Source
Re-exported from Test.Reporter.
Counts and verdict of a completed suite. Group validation problems each count as a failure, so the counts can exceed the number of leaves.
Fields
failed
failed : IntSource
Re-exported from Test.Reporter.
Failed bodies plus individual group validation problems.
skipped
skipped : IntSource
Re-exported from Test.Reporter.
Leaves not run because of skip or focus.
todo
todo : IntSource
Re-exported from Test.Reporter.
Placeholders not explicitly skipped; focus does not suppress todos.
verdict
verdict : VerdictSource
Re-exported from Test.Reporter.
Whether the run passed, failed, or was incomplete.
Instances:
Eq Summary
Show Summary
Test
type Test eSource
A test or a group of tests. The bodies may perform Expect and the
effects in e besides: Test {} holds pure tests, and
Test File.FileSystem tests that use whole-file services. The constructors
are private so the tree can grow.
Verdict
type Verdict = RunPassed | RunFailed | RunIncompleteSource
Re-exported from Test.Reporter.
The verdict of a completed suite. Failures take precedence over incompleteness.
Constructors
RunPassed
RunPassed : VerdictSource
Re-exported from Test.Reporter.
Every test passed and the suite was complete.
RunFailed
RunFailed : VerdictSource
Re-exported from Test.Reporter.
A test or group validation failed.
RunIncomplete
RunIncomplete : VerdictSource
Re-exported from Test.Reporter.
No failures, but the suite contains only, skipped tests, or todos.
Instances:
Eq Verdict
Show Verdict
Reporter
effect ReporterSource
Re-exported from Test.Reporter.
Progress notifications from execute. There is no default handler;
console, silent, or a custom handler chooses what to do with them.
Operations
report
report : Event ->{Reporter} ()Source
Re-exported from Test.Reporter.
Reports one event synchronously before execution continues.
console
console : (() ->{Reporter, e} a) ->{Console, e} aSource
Re-exported from Test.Reporter.
Handles progress by printing failures and the final summary to standard
output, preserving run’s report format. Start events and non-failing
outcomes are silent. Returns the action’s result without exiting;
effects besides Reporter pass through.
describe
describe : String -> List (Test e) -> Test eSource
A named group of tests. Group names form the path a failure is reported under.
documentation
documentation : (() ->{Reporter, e} a) ->{Console, e} aSource
Re-exported from Test.Reporter.
Prints group headings and indented case names with their outcomes as leaves complete. Full failure messages and the summary follow. Group names shared by consecutive paths are printed once. Returns the action’s result without exiting.
execute
execute : Test e ->{Reporter, e} SummarySource
Executes the suite without printing or exiting, returning its summary
and reporting progress through Reporter. Other body effects pass
through to the caller. The first failing expectation ends only its body.
Runtime crashes and unhandled effects propagate without a synthetic
completion event.
import Expect
import Test
result : Test.Summary
result = Test.silent { Test.execute (Test.test "pure" { Expect.equal 1 1 }) }
result.passed == 1
result.verdict == Test.RunPassed
A custom handler can collect progress in memory:
import List
import Test
import Tuple
collect action =
handle action() with events = [] on
Test.report event -> resume () with [event | events]
return value -> (value, List.reverse events)
collected = collect { Test.execute (Test.test "one" {}) }
summary = Tuple.first collected
events = Tuple.second collected
summary.verdict == Test.RunPassed
events == [ Test.SuiteStarted 1, Test.CaseStarted ["one"], Test.CaseFinished ["one"] Test.CasePassed, Test.SuiteFinished summary ]
only
only : Test e -> Test eSource
Focuses the run: when a suite contains only, just the focused tests run
and other test bodies count as skipped. Todos remain todos unless
explicitly skipped. The run is incomplete.
progress
progress : (() ->{Reporter, e} a) ->{Console, e} aSource
Re-exported from Test.Reporter.
Prints one progress character for each completed leaf: . for a pass,
F for a failure, and * for a skipped leaf or todo. Group validation
failures also print F. Full failure messages and the summary follow
the progress line. Returns the action’s result without exiting.
run
run : Test e ->{Environment, Console, FileSystem, e} StatusSource
Runs the suite using command-line output options and answers the status
for main to return: failure unless every test passed and the run was
complete. Other effects the tests perform pass through to the caller,
which handles them around the run. Use execute with a reporter handler
to choose output and exit policy yourself. Command-line options select
progress or documentation output and an optional JSON file; see
Test.Reporter.run.
silent
silent : (() ->{Reporter, e} a) ->{e} aSource
Re-exported from Test.Reporter.
Discards progress events and returns the action’s result. Effects besides
Reporter pass through, so pure suites can execute without IO.
skip
skip : Test e -> Test eSource
A test or group that is not run. Its leaves, including todos, count as skipped and make the run incomplete; an empty subtree adds no skipped leaves.
test
test : String -> (() ->{Expect, e} ()) -> Test eSource
A named test. The body’s expectations run as statements; the first that fails ends the body and fails the test.
todo
todo : String -> Test eSource
A test that is not written yet. It is counted and makes the run incomplete.
