APIEncoding
Encoding
Character encodings: converting between text and bytes.
UTF-8 and Latin-1 are supported. Neither encoding normalizes text, detects
an encoding, removes a byte order mark, or converts newlines; a UTF-8 byte
order mark decodes to U+FEFF. Byte offsets are zero-based and relative to
the bytes given. Bytes.toStringLossy decodes UTF-8 with replacement
instead of failing.
AsciiSet
type AsciiSetSource
A set of ASCII characters, built once by asciiSet and used by
spanUntil to find where a run of text stops.
Encoding
type EncodingSource
A character encoding: utf8 or latin1.
Error
type Error = InvalidSequence Int | IncompleteSequence Int | Unrepresentable CharSource
Why text could not be decoded or encoded.
Constructors
InvalidSequence
InvalidSequence : Int -> ErrorSource
A malformed sequence starts at this byte offset. For UTF-8 that includes overlong forms, surrogates, and values above U+10FFFF.
IncompleteSequence
IncompleteSequence : Int -> ErrorSource
A valid but unfinished sequence starts at this byte offset at the end of the input; more bytes may complete it.
Unrepresentable
Unrepresentable : Char -> ErrorSource
This character has no encoding in the selected encoding.
Instances:
Eq Error
Show Error
asciiSet
asciiSet : (Char -> Bool) -> AsciiSetSource
The ASCII characters that satisfy member. It calls member once per
ASCII code while building the set and never afterwards, so build a set
outside a scanning loop. Characters at or above U+0080 are never members.
decode
decode : Encoding -> Bytes -> Result Error StringSource
The text the bytes encode, or the first error in them.
import Bytes
import Encoding exposing (Error(..))
import Result exposing (Result(..))
Encoding.decode Encoding.utf8 (Bytes.fromList [104, 105]) == Ok "hi"
Encoding.decode Encoding.utf8 (Bytes.fromList [104, 255]) == Err (InvalidSequence 1)
Encoding.decode Encoding.utf8 (Bytes.fromList [104, 226, 130]) == Err (IncompleteSequence 1)
Encoding.decode Encoding.latin1 (Bytes.fromList [233]) == Ok "é"
decodeAt
decodeAt : Encoding -> Bytes -> Int -> Result Error (Maybe (Char, Int))Source
The character encoded at byte offset and its width in bytes, without
consuming anything; Ok Nothing when the offset is outside the bytes. An
offset inside a UTF-8 sequence is invalid. An IncompleteSequence may
decode once more bytes are appended.
import Bytes
import Encoding exposing (Error(..))
import Result exposing (Result(..))
Encoding.decodeAt Encoding.utf8 (Bytes.fromString "é!") 0 == Ok (Just ('é', 2))
Encoding.decodeAt Encoding.utf8 (Bytes.fromString "é!") 1 == Err (InvalidSequence 1)
Encoding.decodeAt Encoding.utf8 (Bytes.fromString "a") 1 == Ok Nothing
encode
encode : Encoding -> String -> Result Error BytesSource
The bytes encoding the text, or Unrepresentable with the first
character the encoding cannot represent.
import Bytes
import Encoding exposing (Error(..))
import Result exposing (Result(..))
Encoding.encode Encoding.latin1 "é" == Ok (Bytes.fromList [233])
Encoding.encode Encoding.latin1 "a€" == Err (Unrepresentable '€')
latin1
latin1 : EncodingSource
Latin-1 (ISO 8859-1): each byte is the character with the same code. Every byte sequence decodes; only U+0000 to U+00FF encode.
matchAt
matchAt : Encoding -> Bytes -> Int -> String -> Maybe IntSource
The encoded width of text when its encoding occurs in bytes at
offset, or Nothing. Text the encoding cannot represent, and an offset
outside the bytes, never match; empty text matches at any offset from zero
through the length. Bytes after the match are not inspected.
import Bytes
import Encoding exposing (Error(..))
import Result exposing (Result(..))
Encoding.matchAt Encoding.utf8 (Bytes.fromString "key: v") 0 "key" == Just 3
Encoding.matchAt Encoding.utf8 (Bytes.fromString "key: v") 1 "key" == Nothing
message
message : Error -> StringSource
A sentence describing the error, for reporting.
import Bytes
import Encoding exposing (Error(..))
import Result exposing (Result(..))
Encoding.message (InvalidSequence 3) == "Invalid encoded sequence at byte 3"
spanUntil
spanUntil : Encoding -> AsciiSet -> Bytes -> Int -> IntSource
The end of the longest run of complete, valid characters from offset
that contains no member of stops. A malformed or incomplete sequence
ends the run, as does the end of the bytes; an offset outside the bytes
is returned unchanged.
import Bytes
import Encoding
stops = Encoding.asciiSet { c -> c == ',' }
Encoding.spanUntil Encoding.utf8 stops (Bytes.fromString "ab,c") 0 == 2
Encoding.spanUntil Encoding.utf8 stops (Bytes.fromString "ab,c") 3 == 4
utf8
utf8 : EncodingSource
UTF-8, the encoding adapters use when none is given. Decoding rejects
malformed sequences; encoding always succeeds, since a String is valid
text.
