APITest

Test

Organizing and running tests.

A suite is a tree: describe groups tests, test names a body that states Expect expectations, and skip, only, and todo mark work that is not run. A program tests itself with main() = Test.run suite:

import Expect
import String
import Test exposing (Test, describe, test)

suite : Test {}
suite =
    describe "String.split"
        [ test "splits on the separator" {
            Expect.equal [ "a", "b" ] (String.split "," "a,b")
            Expect.equal [ "" ] (String.split "," "")
          }
        , test "keeps the text without a separator" {
            String.split "" "abc" |> Expect.equal [ "abc" ]
          }
        ]

True

execute runs without IO and reports structured progress through the Reporter effect. Handle it with silent to get only the summary, console for the default output, or a custom handler for other formats. run selects a format from command-line arguments, so it performs Process.Environment, and applies the process exit policy; without options it uses console. Optional JSON reports perform File.FileSystem; a caller can capture them with File.runMemory.

By default, run prints its report to standard output. Each failure is printed in full, the heading naming the path through the groups, and a summary ends the report:

FAIL  String.split > splits on the separator

    StringTests.fango:12
    Expected: ["a", "b"]
    Actual:   ["a"]

TEST RUN FAILED

Passed:  3
Failed:  1
Skipped: 1
Todo:    1

The summary line reads TEST RUN PASSED when every test passed, TEST RUN FAILED when one failed, and TEST RUN INCOMPLETE when none failed but the suite contains skipped leaves, only, or todo. Only a passed and complete run exits with status 0; the others exit with status 1, so a placeholder cannot make a build green.

A group fails, without stopping its tests, when it has no tests, an empty name, a test with an empty name, or two tests with the same name. A runtime crash inside a test body ends the whole run: there is no Fango level catch for it.

CaseOutcome

type CaseOutcome    = CasePassed    | CaseFailed Error    | CaseSkipped    | CaseTodoSource

Re-exported from Test.Reporter.

The outcome of a leaf in the suite.

Constructors

CaseFailed

CaseFailed : Error -> CaseOutcomeSource

Re-exported from Test.Reporter.

The body’s first failing expectation.

CasePassed

CasePassed : CaseOutcomeSource

Re-exported from Test.Reporter.

The body completed without a failing expectation.

CaseSkipped

CaseSkipped : CaseOutcomeSource

Re-exported from Test.Reporter.

The body or placeholder was skipped explicitly or by focus.

CaseTodo

CaseTodo : CaseOutcomeSource

Re-exported from Test.Reporter.

A placeholder with no body, not explicitly skipped.

Instances:

Eq CaseOutcome
Show CaseOutcome

Event

type Event    = SuiteStarted Int    | CaseStarted (List String)    | CaseFinished (List String) CaseOutcome    | GroupFailed (List String) String    | SuiteFinished SummarySource

Re-exported from Test.Reporter.

A synchronous notification in depth-first tree order. Paths contain group and leaf names from outermost to innermost, without formatting.

Constructors

CaseFinished

CaseFinished : List String -> CaseOutcome -> EventSource

Re-exported from Test.Reporter.

A leaf completed, failed, was skipped, or is a todo. Skipped leaves and todos have no preceding CaseStarted.

CaseStarted

CaseStarted : List String -> EventSource

Re-exported from Test.Reporter.

An active body is about to run.

GroupFailed

GroupFailed : List String -> String -> EventSource

Re-exported from Test.Reporter.

One group validation problem, reported before visiting its children.

SuiteStarted

SuiteStarted : Int -> EventSource

Re-exported from Test.Reporter.

The run begins, with its total number of leaves, including todos and skipped leaves. Group validation problems are not included.

SuiteFinished

SuiteFinished : Summary -> EventSource

Re-exported from Test.Reporter.

The completed summary, identical to the value returned by execute.

Instances:

Eq Event
Show Event

Summary

type Summary =    { passed : Int    , failed : Int    , skipped : Int    , todo : Int    , verdict : Verdict    }Source

Re-exported from Test.Reporter.

Counts and verdict of a completed suite. Group validation problems each count as a failure, so the counts can exceed the number of leaves.

Fields

passed

passed : IntSource

Re-exported from Test.Reporter.

Bodies that completed successfully.

failed

failed : IntSource

Re-exported from Test.Reporter.

Failed bodies plus individual group validation problems.

skipped

skipped : IntSource

Re-exported from Test.Reporter.

Leaves not run because of skip or focus.

todo

todo : IntSource

Re-exported from Test.Reporter.

Placeholders not explicitly skipped; focus does not suppress todos.

verdict

verdict : VerdictSource

Re-exported from Test.Reporter.

Whether the run passed, failed, or was incomplete.

Instances:

Eq Summary
Show Summary

Test

type Test eSource

A test or a group of tests. The bodies may perform Expect and the effects in e besides: Test {} holds pure tests, and Test File.FileSystem tests that use whole-file services. The constructors are private so the tree can grow.

Verdict

type Verdict    = RunPassed    | RunFailed    | RunIncompleteSource

Re-exported from Test.Reporter.

The verdict of a completed suite. Failures take precedence over incompleteness.

Constructors

RunPassed

RunPassed : VerdictSource

Re-exported from Test.Reporter.

Every test passed and the suite was complete.

RunFailed

RunFailed : VerdictSource

Re-exported from Test.Reporter.

A test or group validation failed.

RunIncomplete

RunIncomplete : VerdictSource

Re-exported from Test.Reporter.

No failures, but the suite contains only, skipped tests, or todos.

Instances:

Eq Verdict
Show Verdict

Reporter

effect ReporterSource

Re-exported from Test.Reporter.

Progress notifications from execute. There is no default handler; console, silent, or a custom handler chooses what to do with them.

Operations

report

report : Event ->{Reporter} ()Source

Re-exported from Test.Reporter.

Reports one event synchronously before execution continues.

console

console : (() ->{Reporter, e} a) ->{Console, e} aSource

Re-exported from Test.Reporter.

Handles progress by printing failures and the final summary to standard output, preserving run’s report format. Start events and non-failing outcomes are silent. Returns the action’s result without exiting; effects besides Reporter pass through.

describe

describe : String -> List (Test e) -> Test eSource

A named group of tests. Group names form the path a failure is reported under.

documentation

documentation : (() ->{Reporter, e} a) ->{Console, e} aSource

Re-exported from Test.Reporter.

Prints group headings and indented case names with their outcomes as leaves complete. Full failure messages and the summary follow. Group names shared by consecutive paths are printed once. Returns the action’s result without exiting.

execute

execute : Test e ->{Reporter, e} SummarySource

Executes the suite without printing or exiting, returning its summary and reporting progress through Reporter. Other body effects pass through to the caller. The first failing expectation ends only its body. Runtime crashes and unhandled effects propagate without a synthetic completion event.

import Expect
import Test

result : Test.Summary
result = Test.silent { Test.execute (Test.test "pure" { Expect.equal 1 1 }) }

result.passed == 1
result.verdict == Test.RunPassed

A custom handler can collect progress in memory:

import List
import Test
import Tuple

collect action =
    handle action() with events = [] on
        Test.report event -> resume () with [event | events]
        return value -> (value, List.reverse events)

collected = collect { Test.execute (Test.test "one" {}) }
summary = Tuple.first collected
events = Tuple.second collected

summary.verdict == Test.RunPassed
events == [ Test.SuiteStarted 1, Test.CaseStarted ["one"], Test.CaseFinished ["one"] Test.CasePassed, Test.SuiteFinished summary ]

only

only : Test e -> Test eSource

Focuses the run: when a suite contains only, just the focused tests run and other test bodies count as skipped. Todos remain todos unless explicitly skipped. The run is incomplete.

progress

progress : (() ->{Reporter, e} a) ->{Console, e} aSource

Re-exported from Test.Reporter.

Prints one progress character for each completed leaf: . for a pass, F for a failure, and * for a skipped leaf or todo. Group validation failures also print F. Full failure messages and the summary follow the progress line. Returns the action’s result without exiting.

run

run : Test e ->{Environment, Console, FileSystem, e} StatusSource

Runs the suite using command-line output options and answers the status for main to return: failure unless every test passed and the run was complete. Other effects the tests perform pass through to the caller, which handles them around the run. Use execute with a reporter handler to choose output and exit policy yourself. Command-line options select progress or documentation output and an optional JSON file; see Test.Reporter.run.

silent

silent : (() ->{Reporter, e} a) ->{e} aSource

Re-exported from Test.Reporter.

Discards progress events and returns the action’s result. Effects besides Reporter pass through, so pure suites can execute without IO.

skip

skip : Test e -> Test eSource

A test or group that is not run. Its leaves, including todos, count as skipped and make the run incomplete; an empty subtree adds no skipped leaves.

test

test : String -> (() ->{Expect, e} ()) -> Test eSource

A named test. The body’s expectations run as statements; the first that fails ends the body and fails the test.

todo

todo : String -> Test eSource

A test that is not written yet. It is counted and makes the run incomplete.