%unimsg 0 spec/unimsg/v0 -- The unimsg v0 specification, written in unimsg. -- -- Self-hosting is not a stunt here. The format claims that its own rules are -- expressible as data, and a specification that cannot be written in the thing -- it specifies has not earned that claim. -- -- This document is sufficient to implement a parser and a canonical encoder -- without any other material. Where it is not, that is a defect in this file. spec @unimsg/v0 { title "unimsg v0 — text syntax and canonical encoding" status :provisional release "v0.1 — 2026-08-23. A release label for this document and the nine implementations that agree on it, NOT a format version: the header `%unimsg 0` is the format version and stays 0 until something in the bytes changes. v1 is the goal and not the next step. See docs/11, which reaffirmed `never label it v1` on the day the case for v1 was strongest — five defects were found that day, every one invisible to every check that existed that morning, and a version number is a promise about what has stopped changing." base "RFC 8949 (CBOR)" doc "A human text syntax for CBOR with degradable extensions. The text form is a lossless projection of the binary form: text -> CBOR -> text -> CBOR is byte-identical." -- =================================================================== -- KERNEL -- =================================================================== kernel normative { doc commentary "Everything else is a composition over these. A value is one of:" types [ | name cbor-major notes | :"null" 7 "simple value 22" | :bool 7 "simple values 20 and 21" | :int 0 "unbounded; beyond 64 bits use tag 2 or 3" | :float 7 "IEEE-754; NaN and infinities are values" | :bytes 2 "opaque octets, never text" | :text 3 "UTF-8, not normalised" | :seq 4 "ordered" | :map 5 "keys unique by canonical bytes" | :tagged 6 "a value qualified by a tag" | :array 6 "a homogeneous numeric array: an RFC 8746 tag over a byte string. See arrays" ] a-key-is-answered-at-three-layers "AND THE THREE ANSWERS DIFFER, which is not a defect in any of them. Asking `what may a key be` without saying WHERE produced a specification, a profile and six implementations that each answered a different question and all sounded like they were answering the same one. THE KERNEL admits any canonical value as a key, and D1 settles their equality: byte equality of the canonical encoding. So 0.0 and -0.0 are two keys, because f90000 and f98000 are two byte strings, and 1 and 1.0 are two keys for the same reason. THE V0 PROFILE narrows that to text and integer keys. Not for elegance: a float key is F6 — every language's float comparison says -0.0 == 0.0 while their canonical forms differ, so a map holding both cannot be represented at all in a host whose map is a hash table. A byte-string key is refused for the mirror reason, being unhashable outright in some hosts and identical to a text key in others. THE TEXT SYNTAX writes a key as a word or a quoted string, and has a spelling for nothing else. So a document with an integer key is legal, encodes, decodes and hashes, and HAS NO TEXT FORM — the same way a document read back from CBOR has no comments. PROFILE CHECKING IS A SEPARATE ACT FROM DECODING, so a decoder that refuses a key the profile ALLOWS — text, or an integer — is not being strict, it is being wrong. Six implementations refused an integer key in six different ways while this document and the v0 profile both said it was permitted, and one of them answered with a crash. AN IMPLEMENTATION MAY STILL REFUSE WHAT IT CANNOT REPRESENT, and must say which of the two it is doing. The reference refuses a float key at the decoder, and that is a representation limit rather than a profile check: its values live in a hash map, so f90000 and f98000 arrive as one entry and a pair is lost in silence. Refusing is the honest answer to that, and pretending it was the profile talking would hide a limit behind a rule." a-quantity-carries-its-unit-as-a-qualifier "`mass kg 5.97e24`, and the shape is normative even though the units are not. THE SYNTAX WAS ALREADY HERE. A bare word before a value is a qualifier and can be nothing else, so this parses, encodes and round-trips without any change to the format — what was missing was anybody saying it was THE way, which is why three documents in this repository invented three conventions: a units block mapping name-suffixes to prose, a unit buried in a column name, and a sibling key. No two agreed and nothing read any of them. WHICH UNITS EXIST IS NOT THE KERNEL'S BUSINESS. A list of units with meanings is a registry, a registry is governance, and this project deleted governance on purpose — the same reasoning that made tags hash-derived. So the kernel fixes the SHAPE, which makes the question `what unit is this quantity in` answerable for any document by anything, and vocab/qualifiers/units-v0.umsg carries a set documents may adopt, extend, or ignore in favour of their own. This is F19, and the answer is the one F21 arrived at from the other direction: mechanism in the format, policy in a composable extension." a-formatter-refuses-what-it-cannot-spell "It does not invent a spelling. Rendering the integer key 1 as `\"1\" 2` produces text that re-reads as a TEXT key and encodes to different bytes — a round trip that silently changes the document, which is worse than no round trip at all. WHAT THE REFERENCE DID INSTEAD, before this was written down, was worse than either: it rendered `{1: 2}` as `1 null`. The key stringified, the lookup for the stringified key found nothing, and the formatter emitted a pair that was neither in the document nor readable back." } -- =================================================================== -- LEXICAL STRUCTURE -- =================================================================== lexical normative { encoding "UTF-8" word-runes { doc "What may appear in a bare word, key, symbol name or sigil payload." accept [ :unicode-letter, :unicode-digit, :unicode-mark ] punctuation "_-./+" note "Marks are essential, not an edge case: without them the IPA affricate d͡ʒ and every pointed Arabic form are unwritable. Mathematical symbols are NOT word runes; quote them." } sigil-payload-runes { doc "After a sigil, ':' and '=' are additionally permitted, so that #sha256:9f2a and ~b64:iVBORw0KGgo= lex as one token." adds ":=" } sigils [ | glyph token payload example | "@" :identifier "word" "@ord-88213" | "->" :reference "an identifier or a path" "-> @cust-4471, -> @lexicon.entries.kitāb" | "#" :hash "algorithm:hex" "#sha256:deadbeef" | "~" :bytes "encoding:data" "~hex:ff00 or ~b64:iVBORw0KGgo=" | ":" :symbol "word" ":dispatched" | "!" :extension "name, then value" "!acme/hint { zone 4 }" | "^" :array "type, or type:shape; then [ elements ]" "^f64 [ 1.5, 2.5 ] or ^i16:2x3 [ … ]" | "|" :table-row "cells" "| sku qty" | "--" :comment "to end of line" "-- a note" | "%" :header "unimsg only" "%unimsg 0 label" ] byte-payloads "SETTLED 2026-09-02, closing issue 006. `~hex:` takes an even number of hexadecimal digits in either case and nothing else; the formatter emits lower case. `~b64:` takes the RFC 4648 section 4 alphabet — A to Z, a to z, 0 to 9, + and / — padded with = to a multiple of four characters, and nothing else: no base64url (- and _), no unpadded tail, no whitespace, no line breaks. Two spellings of one byte string is what canonical encoding exists to prevent, so the url-safe alphabet is not a second option; it is malformed. Stated because it was not, and the silence cost a real document: a producer tested only on a runtime whose decoder accepts base64url shipped a blob that a browser refused, and no check could have told it so, because nothing here said which language ~b64: was. The examples were standard and padded; that was the intent, and now it is the rule. The rejects table under conformance carries the vectors." reserved-sigils { doc "Held back so a future addition cannot silently reinterpret text that parses today. Encountering one is an error." glyphs "&$*?\\" spent "^ became the typed array sigil on 2026-09-02 — the first of these to be used, and used for exactly what they were held back for: an addition that reinterprets no text that parsed before it" } reserved-literals { doc "Bare words that are values rather than annotations. A fixed lexical list, so no coercion is value-dependent: NO is never false. These may NOT be used as symbol names — ':true' is a symbol named true, which is never what an author means, so it is an error." nor-as-a-bare-key "The same rule, for the same reason, in the other position: `null 1` is a pair whose key is the word null, which is never what an author means either. Quote it — `\"null\" 1` — and the intent is unambiguous to a reader as well as to a parser. This was found by four independent implementations refusing a document the reference implementation accepted. The specification had ruled on symbol names and been silent on keys, and every independent reader concluded the rule extended; only the reference concluded it did not. Four to one is not a majority vote on a language question, but it is a strong signal about what the text actually says, and the consistent reading is the one adopted here." words [ :"true", :"false", :"null", :"nan", :"inf", :"-inf" ] } separators { value-must-start-on-the-key-line true doc "A key-value pair ends at a separator, so a value may not begin on the line after its key — that newline would end the pair. Long strings wrap freely, since a string may span lines. A key-value pair ends at a separator. Newline and comma are two spellings of the same thing, one of them invisible. Runs of separators collapse." forms [ :newline, :comma ] } strings { delimiter "\"" escapes [ :backslash-n, :backslash-t, :backslash-r, :backslash-quote, :backslash-backslash ] note "May span lines. Any other escape is an error." } numbers { doc commentary "A leading '-' begins a number only when a digit follows; otherwise it belongs to a word such as by-sequence." forms [ | form example lexes-as note | :integer "-42" :int "arbitrary precision" | :decimal "19.99" :decimal "UNMARKED — the general case" | :float "2.5f" :float "marked with a trailing f" | :scientific "6.022e23" :decimal "exact unless marked f" ] note commentary "A number immediately followed by word characters is a malformed number, not a number then a word: 12abc is an error." the-marker-does-not-require-a-point "`f` marks the number binary rather than exact. It attaches to ANY of the forms above and imposes no shape on the mantissa: `0f`, `2f` and `-3f` are floats exactly as `2.5f` is, and `6e23f` is one with no point in it at all. Said outright because the table shows only `2.5f` and two implementations read the single example as a requirement, refusing `0f` while accepting `6e23f` — which is the same absence of a point and was accepted because the exponent made it look decimal. A corpus document writing `min [ 32.81f, -35.2f, 0f ]` is what surfaced it. Nothing canonical turns on this. Every formatter here renders float zero as `0.0f`, so the two readings differ only over what an author may WRITE, and the disagreement could not show up in a comparison of bytes." } timestamps { doc "Tried before numbers, since 2026-08-03 would otherwise lex as 2026 followed by an unexpected '-'. At least YYYY-MM is required, so a bare year stays an integer." pattern "^\\d{4}-\\d{2}(-\\d{2}([T ]\\d{2}:\\d{2}(:\\d{2}(\\.\\d+)?)?(Z|[+-]\\d{2}:\\d{2})?)?)?" note "The literal text is preserved verbatim. Precision is information: 2026-08 is a month, never a widened instant, and an offset describes the observer." the-match-is-bounded "AT MOST 64 CHARACTERS, and an implementation may match against a window of that size rather than against the rest of the document. Said because leaving it unsaid cost three implementations the same defect. THE PATTERN IS ANCHORED AND SHORT, so `match what is left` and `match the next few characters` accept exactly the same input. They are not the same program. Where the rest of the input is a borrow — a rust slice, a C pointer — or where the matcher takes a start index, as kotlin and dotnet do, the first spelling is free. Where it is a copy, as it is in go, dart and javascript, it is O(n) per token and quadratic per document, and it stays invisible until somebody parses a megabyte. See F31." } times-of-day { doc "A wall-clock time, tried after the date-rooted form and before numbers, for the same reason: 03:52 would otherwise lex as 03 followed by an unexpected colon. It shares the timestamp tag, because a time of day is a temporal value at a precision rather than a new kind of thing." pattern "^\\d{2}:\\d{2}(:\\d{2}(\\.\\d+)?)?" no-offset "A time carrying a zone but no date is a confusing object, and what opening hours, watch changes and prayer times actually need is the wall clock. Adding an offset later breaks nothing." two-digits-for-the-minute "Which is what keeps ratios out. `25:1` and `1:1` are quoted strings in a document here, and a one-digit minute never matches, so they stay strings. A literal that swallowed ratios would change the meaning of a document that already parses." ranges-are-not-checked "The shape is checked and the clock is not, which matches the date form: `2026-13-45` is accepted today, so `25:10` is too. Both are shapes rather than assertions about the calendar, and inheriting one rule beats inventing a second." } } -- =================================================================== -- GRAMMAR -- =================================================================== grammar normative { notation "EBNF" productions [ | rule definition | :document "[ header ] , { separator } , [ pair , { seps , pair } ] , { separator }" | :header "'%unimsg' , int , [ word ] , separator" | :pair "key , { annotation } , value" | :key "word | string" | :annotation "word | identifier" | :value "map | seq | table | array | scalar | tagged" | :map "'{' , { separator } , [ pair , { seps , pair } , { separator } ] , '}'" | :seq "'[' , { separator } , [ value , { seps , value } , { separator } ] , ']'" | :seps "separator , { separator }" | :table "'[' , { separator } , row , { seps , row } , { separator } , ']'" | :row "'|' , { cell }" | :array "'^' , element-type , [ ':' , extent , { 'x' , extent } ] , '[' , { separator } , [ element , { seps , element } , { separator } ] , ']'" | :element "int | decimal | 'nan' | 'inf' | '-inf' | 'null'" | :cell "value" | :scalar "int | decimal | float | string | symbol | timestamp | time-of-day | bytes | reserved-literal" | :tagged "identifier | reference | hash | extension" ] rules { annotations-precede-the-value "After a key, zero or more bare words are annotations, then exactly one value. gloss en \"student\" is key gloss, annotation en, value \"student\"." bare-words-are-always-annotations "A bare word is never a value. Symbols are written :name. This removes a positional rule and with it a silent ambiguity: [ a b ] would otherwise mean one annotated value while [ a, b ] means two symbols, and both round-trip cleanly." identifiers-may-annotate "An identifier at the end of a value position is the value; followed by more, it annotates. Unambiguous either way, since it is always spelled @name. This is what lets 'lexeme @lex/x { … }' parse with no extra grammar." a-separator-between-items-is-required "The grammar above says seps, meaning one or more, and not { separator }, meaning zero or more. An earlier version said the latter, which made 'a 1 b 2' and '[ 1 2 3 ]' grammatical while the prose beside it said a pair ends at a separator. Four implementations read the prose and refused them; a fifth read the grammar and accepted them, which is how the contradiction was found — the EBNF had never been implemented from before, only read. Required is the correct reading: optional separators reintroduce the whitespace ambiguity that the symbol sigil was adopted to remove." duplicate-keys-are-an-error "In any map, at any depth. JSON leaves this undefined and it is a real interop hazard; under content addressing it is worse, because two documents that decode identically would hash differently." what-a-reference-denotes "A reference names one of two things, and which one is visible in the reference itself. A name containing no dot denotes an identifier — something declared with @ somewhere. A name containing a dot denotes a path from the root of the document holding the reference, each segment a map key. The distinction costs no syntax because infix dot is already reserved for paths, and both forms already parse: a reference is stored as one opaque string either way, dots included. Why both. An identifier is a stable name that survives the thing being moved, and must be declared. A path is a location that needs no declaration and breaks if the document is reorganised. Neither is better; they answer different needs, and requiring only one imposes its cost everywhere. Requiring identifiers makes every entry of a keyed collection write its own name twice, with nothing checking the two agree. Requiring paths makes every reference fragile. A key that itself contains a dot cannot be reached by a path, because the dot is a separator. Such a key needs an identifier. A PATH IS ROOTED AT THE DOCUMENT, AND ALSO AT ITS SOLE TOP-LEVEL BLOCK WHEN THERE IS EXACTLY ONE. Almost every document is one named block — `language @lang/akk/ob/v0 { … }` — and inside it nobody writes `@language.sources.x`, because the block is named for what the whole document is about and repeating that inside its own document is noise. They write `@sources.x`, and they are right. MEASURED BEFORE THE RULE WAS CHANGED, across fifty-seven documents: fourteen dotted references resolved from the document root, one hundred and forty-two resolved only from inside the sole top-level block, and five resolved neither way. A rule that a hundred and forty-two references disagree with is not a rule anybody is following. The fallback applies ONLY when there is exactly one top-level block, so resolution stays deterministic: a document with two of them has one root and no ambiguity about which." a-reference-outside-the-set-says-so "`external`, written as a qualifier on the reference: src external -> @genbank/OQ441238 WITHOUT IT A CHECKER CANNOT REPORT ANYTHING USEFUL. A GenBank accession, an IMO ship number and a customer id in an example about sigils all resolve to nothing, and so does a typo — and nothing in the document distinguishes them, so a checker either stays silent about both or shouts about both. Marking the deliberate ones is what makes everything left over a real question. It changes no encoding and no resolution: a reference is inert either way, and a checker still reports rather than judges. What the mark does is move a reference out of the checker's questions and into the document's statements, where a reader can see it." key-it-if-you-will-reference-it "A consequence worth stating, because it decides how a document is shaped. A path segment is a map key, so a row of a table cannot be addressed: rows are sequence elements and have no names. An identifier written in a row annotates that row's first value rather than naming the row. So a collection whose members will be referenced should be a map keyed by their names, not a table. A table is right for rows that are read together and pointed at never — which is most tables, and exactly why the distinction is easy to miss until the first reference is needed." references-are-still-inert "None of the above is resolved while reading. A decoder produces the reference and stops, exactly as before — resolution is a separate act performed by something that has chosen a set of documents to resolve against, and is no more part of decoding than following a hyperlink is part of parsing HTML." no-external-target-syntax "SETTLED 2026-09-02. The format defines no way to say WHERE an external target lives — no URL form, no repository name, no locator. A reference is an identifier or a path, `external` says the author knows the target is not in this document, and that is the whole of what the format carries. Locating the document that holds @genbank/OQ441238 is the business of whatever chose the document set, exactly as resolution is, and a locator inside the reference would make the reference wrong the day the target moved. A vocabulary that needs to say where its members are found carries a locator beside the reference as an ordinary value; the kernel does not." resolution-is-relative-to-a-set "A path is rooted at its own document; a reference across documents therefore uses an identifier. Whether any given reference resolves depends on which documents are in hand, so a reference that resolves in no set is not thereby invalid — the format cannot know what else exists. A checker reports; it does not judge." table-is-sugar "A table encodes exactly as a sequence of maps with identical key sets. Header cells name the fields; each row must supply exactly as many cells. Cells need no separator because an annotated value is self-delimiting." authored-order-is-kept-beside-the-value "A table's columns are map keys, and map keys sort canonically, so the order a table was written in is not recoverable from the encoding. That is correct and must stay correct: two documents that decode identically have to encode identically, or an address means nothing. But the order carries meaning to a reader — an identifier, then what was given, then what is expected — and losing it makes a vector table materially harder to use. So the authored order is kept OUT of the value, alongside the comments, which solve the same problem the same way: a decoder produces the map, and the document separately records which order its columns were written in, keyed by path. AND A BLOCK'S PAIRS ARE THE SAME CASE, which took longer to see only because a block does not display its keys side by side where the loss is obvious. A specification whose sections arrive conformance, grammar, kernel, scope is in an order nobody chose, and the reader is left to reassemble the argument. So a document records both: the order each block's pairs were written in, and the order each table's columns were written in — both outside the value, each keyed by the path of the block or the table, with the empty path for the document body and the indexed form a.b[2].c for a sequence element, so that two tables in one sequence do not record under the same key. A SEQUENCE OF BLOCKS THAT RENDERS AS A TABLE never had a header to record, because it was not written as one. Its columns take the pair order of its first row instead — the same record answering the same question one level down. Without that clause the two ways of writing the same value render differently, which is the sugar failing to be sugar. What follows from that, and is the honest cost: the order survives a text round trip and does not survive an encode and decode. A document read from CBOR has canonical order throughout and no way to know what the author intended, exactly as it has no comments. Anything that needs the authored order must read the text." a-final-row-needs-no-separator "SETTLED 2026-08-23, in favour of the seven implementations that already accepted it, and against the grammar as it was written. `x [ | a b, | 1 2 ]` is well formed: a separator goes BETWEEN rows, and the closing bracket ends the last one. The production was the defect. A map's rule carries a `{ separator }` tail and so does a sequence's; the row rule was written without one, which made the table the single place in the format where a closing bracket could not follow the last item. That is an oversight in an EBNF line, not a decision anybody took. WHAT DECIDED IT was asking the two strict implementations the same question about the other two containers. Rust accepts `x [ 1, 2 ]` and `x { a 1, b 2 }` and refuses only `x [ | a b, | 1 2 ]` — so the strict reading was not a principle either of them held, it was this production leaking into one code path. Dart did the same. The precedent stands unchanged: a-separator-between-items-is-required is about BETWEEN, and always was. `x [ | a b | 1 2 ]`, with no separator between the two rows, is still malformed. What was never at stake: any canonical document. The formatter always ends a block table's final row with a newline, so every document this project has emitted satisfies both readings, and the disagreement was reachable only by a hand-written single-line table. It was found by writing one into a conformance vector and discovering the vector had never been executed." header-cells-are-keys "A header cell names a map key, so it may be written the way a key is written anywhere else: a bare word, or a quoted string when the name contains a space. The first version of this rule allowed only bare words, which made a table header the one place in the format where a key could not be quoted. The failure was silent and misleading: `| hazard what happens` parsed as three columns, and the error arrived on the following row as a cell-count mismatch, naming neither the header nor the cause. It caught the same author four times in one day across four documents, which is a property of the grammar rather than of the author. Purely additive: every table already written stays valid and encodes to the same bytes, because a bare word means what it always meant." rows-are-separated "RESTATED 2026-09-02 with a vector, because the rule was written and not enforced. a-final-row-needs-no-separator says `x [ | a b | 1 2 ]` is malformed, and on the day that sentence was checked the reference and four of eleven implementations accepted it while five refused it — and the five were being reported as the odd ones. A rule five implementations follow and the reference does not is a rule nothing pinned. `x [ | a b | 1 2 ]` is now a reject vector, the reference refuses it with a message that names the fix, and the four follow. WHY MALFORMED RATHER THAN CONVENIENT: a row's cells carry no separator, because an annotated value is self-delimiting, so the '|' is the only thing that says where one row ends and the next begins. Letting a row begin mid-line makes '|' do that job in two positions, and the one place this format has been fragile is exactly the row boundary — F27, F29, and a cell ending in an identifier swallowing the cell after it. One reading: a row begins after a separator. The formatter has always emitted that, so no canonical document changes." a-header-with-no-rows-is-a-table "SETTLED 2026-09-02. `x [ | a b ]` is well formed, encodes to the empty sequence — `a1617880` is a vector — and its header is kept beside the value exactly as a table's column order is: recorded by path, honoured by the formatter, and lost through the bytes. So a formatter renders it back as the header-only table it was written as, and not as `[]`. This is not a new mechanism. authored-order-is-kept-beside-the-value already records every table's columns by path; a table with no rows is the case where that record is ALL there is to keep. The alternative reading — an error — would refuse the idiom this repository's own design documents use for a review table nobody has reviewed yet, and the other alternative — accept and emit `[]` — is a formatter silently discarding what an author wrote, which is the failure F16 and F25 were. The tenth implementation accepted it on the ground that refusing would be inventing a rule; that ground was right, and the formatting half of the answer was missing." } } -- =================================================================== -- ARRAYS -- =================================================================== arrays normative { doc commentary "A homogeneous numeric array as a kernel value. Proposed in docs/49 on a measurement of one implementation, measured again in docs/52 in five, and adopted 2026-09-02: peak memory for a sequence of 800,000 floats runs from 8x the encoded bytes in the leanest implementation to 145x in the heaviest, a 16,000-row table encodes at 1.5x its own text because every row repeats every key, and a ten-million-value variable could not be decoded at all. The array is the one value model under which an eight-byte number costs eight bytes." text-form "A sigil, an element type, an optional shape, and the elements written as numbers in order, separated as a sequence's items are: ^f64 [ 1.5, 2.5, 3.5 ] ^i16:2x3 [ 1, -1, 2, -2, 3, -3 ] ^u8 [ 1, null, 3 ] THE TYPE IS THE MARKER. Elements carry no f; an unmarked 1.5 inside ^f64 is a binary64 because the array says so, and 1.5 inside ^u8 is an error. Integer elements are decimal integers within the type's range. Float elements are any numeric literal, or nan, inf and -inf. null IN AN ELEMENT POSITION MEANS MISSING — a property of the position, not a value at it — and puts a validity mask on the array. A fill value is a number pretending not to be one; docs/52 read three of them out of one file by hand. A SHAPE IS EXTENTS JOINED BY x, at least two of them; the element count must equal their product or the document is in error. A one-dimensional array's shape is its length and is not written. Rows run along the last extent." element-types [ | type tag width holds | :u8 64 1 "unsigned integer" | :u16 65 2 "unsigned integer" | :u32 66 4 "unsigned integer" | :u64 67 8 "unsigned integer" | :i8 72 1 "two's-complement integer" | :i16 73 2 "two's-complement integer" | :i32 74 4 "two's-complement integer" | :i64 75 8 "two's-complement integer" | :f16 80 2 "IEEE 754 binary16" | :f32 81 4 "IEEE 754 binary32" | :f64 82 8 "IEEE 754 binary64" ] encoding "RFC 8746's, and no other: the substrate already registers a tag per element type, and a parallel list would be the vendored-table mistake made as a design decision. Elements are packed BIG-ENDIAN into one byte string under the type's tag — network order, which is how every other number in a CBOR stream is written. The little-endian tags (69–71, 77–79, 84–86), the clamped uint8 (68), binary128 (83, 87) and column-major (1040) are REFUSED by a decoder: two encodings of one value is what determinism forbids. A SHAPE is RFC 8746's tag 40, row-major: 40([ [ extents… ], array ]), written only when there are two or more extents. A MASK is tag 70107: 70107([ bitmap, array ]), where the array is the shaped or unshaped array, the bitmap is a byte string of one bit per element, least significant bit first, 1 for present, padding bits zero, and every masked position holds zero bits in the payload. A mask with every position present is not canonical and is refused." equality "NEEDS NO NEW RULE. D1 says byte equality of the canonical encoding; the encoding above is deterministic; so two arrays are equal exactly when their element type, shape, mask and element bit patterns are — which is what docs/49 asked for, and it falls out of the format rather than being added to it. ONE CONSEQUENCE: only the canonical quiet NaN — 7e00, 7fc00000, 7ff8000000000000 — may appear in a float array. The text has one spelling for NaN, so a NaN with a payload would be a value with no round trip, and a decoder refuses it." one-text-per-float "THE BURDEN, stated here rather than met in the twelfth implementation. A float element is written in the SHORTEST digit string that reads back as the same value AT THE ARRAY'S WIDTH — binary16, binary32 or binary64 — laid out as ECMAScript lays out a Number: positional when the decimal exponent lies in [-6, 21), otherwise d.ddde±N with the exponent's sign always written; -0 keeps its sign because the bit pattern is the value. 0.1 in ^f32 is `0.1`, not `0.10000000149011612`. Scalar floats follow the same rule with their marker appended (see formatting.floats-keep-their-marker), so a float has one text wherever it stands. WHERE TWO DIGIT STRINGS OF THE SHORTEST LENGTH BOTH READ BACK AS THE VALUE, the canonical text is the one nearer the value, and on an exact tie the one whose last digit is even. SO THE SEARCH TRIES BOTH NEIGHBOURS at each digit length — the exact expansion truncated, and that truncation incremented — and not one candidate from a round-to-nearest. Added 2026-09-03, found by the JavaScript implementation on a binary32 element of a real file: 26.6640625 is `26.664062` and `26.664063` at eight digits and only the first is canonical. A formatter that rounds the exact value at n significant digits gets this wrong wherever its language rounds ties away from zero, which most do, and no vector had caught it because no vector held a tie. Chosen because it is completely specified, one implementation's runtime already does it natively, and every other language here can do it from its shortest-float routine and a dozen lines of layout. Pinning any one language's %g would have pinned that language's habit." guards "An array is one level of nesting holding any number of elements, so a guard framed in depth guards nothing here. A decoder checks the byte string's length against the element width before it allocates anything, checks a shape's product against the count with overflow detection, and MAY refuse an array above an element count it declares — a fixed-arena implementation refused 354,753 in docs/52, honestly and with a position. profiles/v0 says nothing about element counts yet, and that is the open question docs/49 raised and this specification has not closed." not-in-v0 "Compression, chunking, complex numbers, sparse matrices, a datetime element type, and a unit on the array rather than on its elements. docs/49's not-in-v0 table says why for each; docs/52 confirmed the datetime deferral against a real file. Units, named axes and uncertainties are the measurement vocabulary's, not the kernel's." } -- =================================================================== -- ENCODING -- =================================================================== encoding normative { determinism { profile "RFC 8949 section 4.2 core deterministic encoding" required true note commentary "Mandatory, not a mode. Shortest-form integers, definite lengths, shortest-form floats, map keys sorted by their encoded bytes. An encoder must refuse to emit anything else." } decoding { doc commentary "The specification was encoder-only until independent implementations built decoders anyway and each invented its own strictness rules. A reader that accepts input an encoder could never have produced is not interoperable with one that refuses it." reject-non-deterministic-input true reject-trailing-data true how "The simplest conforming check, and the one least able to miss a rule: re-encode the decoded value and compare the bytes with the input. Enforcing determinism rule by rule during decode verifies the causes; this verifies the property, and cannot be incomplete. Report the offset of the first differing byte." note commentary "A decoder must not repair. Sorting an out-of-order map or shortening a non-shortest integer silently turns non-conforming input into conforming output, which is how divergence spreads." } byte-order-mark { doc commentary "A leading U+FEFF is an encoding marker, not content. Implementations observed splitting on this: two rejected it, one silently consumed it, which is exactly the kind of gap a specification exists to close." permitted true ignored true note commentary "Permitted and skipped, following RFC 8259's treatment for JSON. Rejecting is defensible but hostile — editors emit these without being asked — and the marker carries no data, so ignoring it loses nothing." } tags [ | number name content notes | 4 :decimal "[ exponent, mantissa ] both integers" "RFC 8949 3.4.4; value = mantissa * 10^exponent" | 70100 :symbol "text" "the name, without the leading colon" | 70101 :annotated "[ [ annotation… ], value ]" "annotations are symbol or identifier tags" | 70102 :identifier "text" "the name, without the leading @" | 70103 :reference "text" "target name WITHOUT the leading @; never resolved by the parser" | 70104 :hash "[ algorithm-text, digest-bytes ]" "digest is decoded from hex" | 70105 :extension "[ name-text, value ]" "name without the leading !" | :note :sigils-stripped "@ -> ! are syntax, not part of the stored name" "applies to 70102, 70103, 70105" | 70106 :timestamp "text" "the literal, verbatim" | 70107 :validity "[ bitmap-bytes, array ]" "a typed array with missing positions; see arrays" | 40 :shape "[ [ extents… ], typed-array ]" "RFC 8746 multi-dimensional array, row-major" | "64–67" :unsigned "byte string, big-endian" "RFC 8746: u8 u16 u32 u64" | "72–75" :signed "byte string, big-endian" "RFC 8746: i8 i16 i32 i64" | "80–82" :float "byte string, big-endian" "RFC 8746: f16 f32 f64" ] tag-numbers-are-provisional "70100–70106 are placeholders. The eventual scheme is hash-derived, so these will change. They are fixed here so that independent implementations can agree today." mapping [ | text-form cbor | "{ … }" "map, text keys" | "[ … ]" "array" | "| rows" "array of maps — identical to the longhand" | "123" "integer, or tag 2/3 beyond 64 bits" | "19.99" "tag 4 decimal fraction" | "2.5f" "float, shortest form" | "\"…\"" "text" | ":word" "tag 70100" | "word value" "tag 70101" | "@name" "tag 70102" | "-> @name" "tag 70103" | "#alg:hex" "tag 70104" | "!name value" "tag 70105" | "2026-08" "tag 70106" | "~hex: / ~b64:" "byte string — a kernel type, no tag" | "^f64 [ … ]" "tag 82 over a byte string; 64–67, 72–75, 80–82 by element type" | "^f64:2x3 [ … ]" "tag 40 [ [ 2, 3 ], typed array ]" | "^u8 [ 1, null ]" "tag 70107 [ bitmap, array ]" | "true false null" "simple values" | "nan inf -inf" "floats" ] decimals-preserve-what-was-written "Mantissa and exponent are stored as written, not canonicalised. 1.00 and 1.0 are DIFFERENT values, because trailing zeros carry significance in money and measurement. 19.99 becomes [ -2, 1999 ]." comments-never-reach-the-binary "Comments are lexical. Encoding a document with and without commentary must produce identical bytes, or content addressing breaks: a signature would be invalidated by fixing a typo in a comment." } -- =================================================================== -- DECISIONS -- =================================================================== decisions normative [ | id subject ruling rationale | :D1 :key-equality "byte equality of the canonical encoding of the key" "native equality diverges: -0.0 == 0.0 but NaN != NaN" | :D2 :map-ordering "unordered; canonical sorts keys by encoded bytes" "required for content addressing" | :D3 :determinism "RFC 8949 4.2, mandatory" "hashing and signing depend on it" | :D4 :finiteness "values are finite" "indefinite length is streaming, not infinity" | :D5 :normalisation "none; text preserved byte-for-byte" "normalising silently alters data" | :D14 :bidi-controls "content, not a threat the encoding may edit away" "txt-005: the hazard is a renderer mixing trusted and untrusted text" | :D6 :comments "lexical only; survive text to text" "in the bytes they would break content addressing" | :D7 :tag-space "hash-derived eventually; fixed numbers for now" "defers governance without foreclosing it" | :D10 :key-types "any canonical value in the kernel; text and integer in the v0 profile" "profile restrictions are reversible; kernel changes are not. See a-key-is-answered-at-three-layers" | :D11 :decimal-default "unmarked literals are exact decimals; floats marked f" "the general case should be unmarked" | :D12 :units "a qualifier on the value; the kernel fixes the shape, a vocabulary the set" "a unit registry is governance, and governance was deleted on purpose" | :D13 :time-of-day "a wall-clock literal, sharing the timestamp tag, no offset" "F22: a time of day is ordinary data and could not be written at all" ] -- =================================================================== -- CANONICAL TEXT FORM -- =================================================================== formatting normative { doc commentary "There is exactly one canonical rendering and no options. The point of a canonical formatter is to end the argument, not to host it. Formatting must be idempotent. The rendering is the layout block below, settled 2026-09-03; tools/fmtcompare measures every implementation's text against the reference's, document by document, and agreement on the whole corpus is a freeze gate." ordering { rule "The authored order where the document records one, and otherwise single-line values first, multi-line after, alphabetical within each group." authored-order-comes-first "A block's pairs and a table's columns render in the order they were written, read from the record that travels beside the value. See authored-order-is-kept-beside-the-value, which is the same rule stated where tables are defined. A formatter that ignored the record would make the record useless. The authored order survives a text round trip and nothing else, and the formatter IS the text round trip — so a formatter that sorts is not a formatter with a different opinion about presentation, it is the thing that destroys what the record exists to keep. The reference formatter did exactly that for a while, against this specification, while three other implementations honoured the record and were right against it." a-record-may-be-partial "The keys a record names, and that the value still carries, render first in the recorded order; any remaining keys follow under the fallback rule, and a name the value no longer carries is skipped. That is what keeps the rule useful when a document is edited between the parse and the format. The stricter alternative — honour the record only when it accounts for every key exactly — throws the whole authored order away on the first pair anything adds." it-is-still-idempotent "Reading back what was just emitted records exactly the order just emitted, so a second format changes nothing." one-rendering-per-document "A document is its value AND what travels beside it, so there is still exactly one rendering and no options. Two documents with equal values and different records render differently for the same reason two with different comments do, and they encode identically, which is the property that has to hold." note commentary "None of this matches the binary key order of D2, and it does not have to: the binary sorts independently, so text order affects no bytes. It need only be deterministic." } comment-binding { rule "A comment binds to the path of the pair it precedes, never to a line number, and moves with that pair when ordering changes. A comment that FOLLOWS a value on the same line binds to the pair just read, and terminates that pair as a separator would." trailing-comments-are-legal true note-on-trailing "Left unstated, this split four implementations three to one: three accepted a trailing comment and one refused it, correctly, because the specification described only comments that precede. Authors reach for the trailing form immediately — the first document written in anger used one — so it is permitted rather than forbidden." why "A comment left in place while content moves around it ends up describing something it was not written about — a confident claim about the wrong value, which is worse than deleting it." note commentary "A block carrying comments never collapses to flow form, since the comments would have nowhere to live." } inline { budget 56 rule "Maps and sequences render on one line when every value is single-line and the result fits the budget; otherwise block form. Flow form separates with commas, block form with newlines." measured-against "SETTLED 2026-09-02: `the result` is the composite's own inline text — `{ a 1, b 2 }` or `[ 1, 2 ]`, brackets included — measured against the budget and nothing else. Not the key before it, and not the indentation it will sit at. That is what the reference has done since its first formatter, so this pins a convention rather than changing one, and it is the reading under which a value's rendering does not depend on where it sits: render once, reuse everywhere, and the exponential formatter the implementer's guide spends its length on cannot be built. Several implementations measured the key or the indentation as well, and emitted non-canonical text for values near the budget that nothing saw: the byte checks go through the encoder, and the order check compares order. `tools/fmtcompare` now compares formatter text against the reference, document by document, which is the check `format-idempotently` and `one canonical rendering` always needed." } tables { rule "A sequence renders as a table when it holds two or more maps with identical key sets, no multi-line cells, header names needing no quoting, and the inline form does not already fit." note commentary "The final column is not padded: trailing whitespace breaks diffs and editors strip it, which would break idempotence." } layout normative { doc "SETTLED 2026-09-03, the first freeze gate of docs/50: the rules the reference formatter had always followed and the specification had never stated, so that eleven formatters can agree byte for byte and tools/fmtcompare can say so. Where the reference's habit was a bug rather than a rule — a quoted key padded to its unquoted width, a Japanese key padded by its bytes, a paragraph folded into one line of escapes — the rule below is the fix and the reference was changed to match. Every width is measured in Unicode code points of the rendered text: not bytes, which over-pad a non-Latin key, and not display columns, which no format can define." document [ | rule detail | "the header line, then one blank line, then the root" "`%unimsg 0 label` or `%unimsg 0`; a headerless document has neither the line nor the blank. The blank line after the header is the only blank line in canonical text" | "the header is THREE FIELDS, not a line of text: the sigil and version, and the label when there is one, each separated by ONE space" "a document whose header was authored with two spaces renders with one. Stated 2026-09-03 after the Julia implementation found corpus/cases.umsg doing exactly that, and nothing saying whether to keep it" | "a root map renders as pairs at depth zero; any other root value renders as itself" "the closing newline of the last line ends the output" | "indentation is two spaces per depth" "a pair's value that opens a block puts `{` or `[` on the pair's line and the closing bracket at the pair's own indentation" ] pairs [ | rule detail | "a pair is the rendered key, one space, the rendered value" "the key is bare when every character is a word character and it is not a reserved literal; otherwise quoted. An empty key renders as `\"\"`" | "BARE MEANS IT LEXES AS ONE WORD, not merely that its characters are word characters" "every character of `2024`, `1.5` and `2026-08-06` is a word character and none of them lexes as a word — a lexer returns an integer, a decimal and a timestamp — so all three are QUOTED as keys, in a pair and in a table header alike. Corrected 2026-09-03: the reference had emitted them bare and could not read its own output back, costing three corpus documents their round trip. The JavaScript and Go implementations found the two halves of it independently, and the implementer's guide had carried the rule since the tenth implementation" | "keys in a block are padded to a column when the widest RENDERED key is 16 code points or fewer" "each rendered key is followed by spaces up to the widest, then the one separating space. When the widest exceeds 16 no key is padded" | "the column is measured on keys as rendered" "a key that needs quoting counts its quotes, so it lines up with its neighbours" ] comments [ | rule detail | "a comment renders as `--`, one space, its text, on its own line at the indentation of what it precedes" "the text is what followed the dashes when read, with surrounding whitespace trimmed; an empty comment renders as the bare `--` with no trailing space" | "a pair's comments precede the pair; an element's precede the element; a block's tail comments follow its last pair at the same indentation" "a trailing comment read from the same line as a value becomes a preceding comment of that pair — but only WITH NO SEPARATOR BETWEEN. A comma has already ended the pair, so a comment after one precedes whatever comes next; `p 1, -- note` puts the note above `q`, and `r 1 -- note` puts it above `r`. Stated 2026-09-03, found by the Python implementation, and the two readings differ on two corpus documents" | "a comment after the last element of a sequence, or the last row of a table, belongs to the ENCLOSING block: it precedes the next pair, or is the block's tail comment" "so the sequence itself carries no comment and may still render inline or as a table" | "a comment on the SAME LINE as a table row binds to that row as an element comment, and so declines the table" "only a comment on its own line after the last row flows out to the enclosing block. A row is not a pair, so the two rules above overlapped on it until 2026-09-03, when the Rust implementation named the case" | "a block or sequence with any comment BENEATH it never renders inline and never renders as a table" "beneath means at a path strictly under the composite's own — a pair inside it, an element of it. A comment AT the composite's own path belongs to the pair that holds it and renders above that pair, so `a { c 2 }` with a note stays inline. Reading beneath as `at or beneath` turns four corpus documents into blocks; the word carries the whole distinction" ] strings [ | rule detail | "a string is quoted with exactly four escapes: `\\t`, `\\r`, `\\\"` and `\\\\`" "every other character, a control character included, is written as itself" | "a NEWLINE INSIDE A STRING IS WRITTEN RAW" "`\\n` and a raw newline read as the same value; the canonical text writes the newline, so a paragraph stays a paragraph. The characters after the newline are part of the value and are written exactly, so a string is never re-indented" | "a string containing a newline is a multi-line value" "it has no inline form, so a block holding one never collapses to flow form, a sequence holding one never becomes a table, and it sorts with the multi-line group under the fallback order" ] inline-and-block [ | rule detail | "a map or sequence renders inline when every value has an inline form and the composite's own text is at most 56 code points" "`{ a 1, b 2 }`, `[ 1, 2 ]`: one space inside each bracket, `, ` between members; a pair inside is key, one space, value, with no padding" | "otherwise a map renders as `{`, its pairs one per line at the next depth, `}`" "and a sequence as `[`, its elements one per line, `]`, or as a table when the tables rules allow" | "an empty map is `{}` and an empty sequence `[]`" "except a sequence that carries a recorded header and no rows, which renders as the header row alone in block form" ] tables [ | rule detail | "a sequence renders as a table when it holds two or more maps, every map has two or more keys, every map has the same key set, no cell is multi-line, no comment lies beneath the sequence, and the inline form does not fit" "a header cell is a key and is quoted where a key would be, so a header such as `\"what it would do\"` keeps its table" | "a cell that renders ending in an identifier disqualifies the table unless it is in the last column" "`@name` before a value annotates it, so such a cell would swallow the next" | "column order is the recorded header; failing that the recorded pair order of the first row; failing that the fallback order of the first row's keys" "the recorded orders are what the parser keeps beside the value" | "a cell is a value at the path `row[i].column`, so a block inside a cell records and renders its pair order like any other block" "settled 2026-09-03 when the .NET implementation found the reference losing that order and failing its own idempotence on one document" | "A ROW MAP RECORDS NO ORDER OF ITS OWN: the header is the table's record, so a sequence of maps that does NOT become a table renders each map in the fallback order" "the two pull opposite ways and both are required. An implementation that stores the authored order on the row map — the obvious place — renders a one-row or table-refused sequence in header order instead of alphabetically, with every byte check still green. Found by the C implementation, 2026-09-03" | "a recorded header with no rows renders as the header row alone: `|`, then each header cell after one space, unpadded" "the table rule applied to a table of one row" | "each row is `|`, then for every cell one space and the cell padded to the column's width; the last column is never padded" "the width is the widest rendered cell in the column, header included, in code points" ] arrays [ | rule detail | "a typed array renders inline when its whole text fits the budget" "`^f64 [ 1.5, 2.5 ]`; an empty array is `^u8 []`" | "an unshaped array that does not fit renders TEN ELEMENTS PER LINE" "so the first element of every line has an index that is a multiple of ten, and a changed value never reflows the lines around it" | "a shaped array that does not fit renders one innermost row per line: as many elements as the last extent" "so a matrix reads as a matrix; a last extent of zero renders as an empty array" | "elements on a line are joined by `, `, and each line sits at the next depth" "a missing position is `null`; every element's text is one-text-per-float's" ] scalars [ | rule detail | "an exact decimal renders from its mantissa and exponent, not from the digits as authored" "with d the mantissa's digit count and e the exponent: e = 0 is the digits then `e0` (`5e0`, `10e0`); e > 0 or d + e <= -6 is scientific — the first digit, then a point and the remaining digits when there are any, then `e` and the normalised exponent with no plus sign: `6.022e23`, `2.1e-36`, `1e5`, `1.0e6` for mantissa 10; otherwise positional: `0.00017`, `0.000001`, `1974.3781`. Where the author put the point is not information the value carries" | "a float is one-text-per-float's digits with a point and the marker" "`0.5f`, `1e+21f`, `nan`, `inf`, `-inf`" | "bytes render as `~hex:` and lowercase hex; a hash as `#alg:` and lowercase hex" "an identifier as `@name`, a reference as `-> @name`, an extension as `!name` one space and its value, annotations as the bare words then one space then the value" | "a symbol renders bare after its colon when its name is a non-empty run of word characters and not a reserved literal, otherwise quoted after the colon" "the sigil-payload predicate, which is wider than a bare key's: `:1`, `:3d`, `:-1`, `:a.b`, `:2026-08-06` are bare; `:\"null\"`, `:\"a b\"`, `:\"a:b\"`, `:\"\"` are quoted" ] fallback-order "Where no order is recorded — a document read from bytes, or built rather than parsed — a block's pairs render single-line values first, then multi-line, each group ascending by the key's code points. Single-line means the value has an inline form." } floats-keep-their-marker "A binary float always renders with a trailing f and a decimal point, so it can never re-read as an integer or as an exact decimal. Its digits and their layout are arrays.one-text-per-float's, settled 2026-09-02: shortest round-trip digits in ECMAScript's layout, so 0.00001f and 1e+21f rather than one language's %g." } -- =================================================================== -- ERRORS -- =================================================================== errors normative { doc commentary "A format is exactly as friendly as its worst parse error." requirements [ :every-error-carries-line-and-column, :errors-name-the-fix-where-one-is-obvious, :decoding-is-total-and-side-effect-free, :nothing-is-resolved-or-fetched-during-parsing ] examples [ | input message | "x [ a b ]" "annotation \"b\" has no value — did you mean \":b\"?" | "x :true" "\":true\" is a symbol named \"true\", not the literal — write true" | "a 1, a 2" "duplicate key \"a\"" | "x 12abc" "\"12abc\" is not a valid number or word" ] } -- =================================================================== -- CONFORMANCE -- =================================================================== conformance normative { must [ :round-trip-text-to-cbor-to-text-to-cbor-byte-identically, :format-idempotently, :reject-duplicate-keys-at-every-depth, :preserve-the-integer-float-distinction, :preserve-decimal-mantissa-and-exponent-as-written, :preserve-timestamp-precision-and-offset, :emit-only-deterministic-encodings ] integer-float-note "Language runtimes without distinct numeric types must use a wrapper. JavaScript cannot represent the distinction at all: it maps float 1.0 onto the canonical form of integer 1, silently." self-test { doc commentary "An implementation is correct when it agrees byte-for-byte with the reference encoder on every document in the test corpus." command "go run ./tools/compare -- " note commentary "That command needs a repository an implementer working from this document alone does not have. The vectors below are the part of the conformance criterion that travels with the specification." } -- Each source line encodes to exactly these bytes. Machine-checkable, and -- sufficient on its own to pin every tag number, the annotation shape, the -- decimal representation and the integer/float distinction — all of which -- a first independent implementation otherwise had to infer. vectors vector [ | source canonical-hex | "a 1" "a1616101" | "x [ | a b, | 1 2, ]" "a1617881a2616101616202" | "x [ | a b, | 1 2 ]" "a1617881a2616101616202" | "x [ | \"a\" \"b\", | 1 2, ]" "a1617881a2616101616202" | "a -1" "a1616120" | "a 1.0" "a16161c482200a" | "a 1.00" "a16161c482211864" | "a 03:52" "a16161da000111da6530333a3532" | "a kg 5" "a16161da000111d58281da000111d4626b6705" | "a 1.0f" "a16161f93c00" | "a 0f" "a16161f90000" | "a \"x\"" "a161616178" | "a :x" "a16161da000111d46178" | "a true" "a16161f5" | "a null" "a16161f6" | "a nan" "a16161f97e00" | "a inf" "a16161f97c00" | "a []" "a1616180" | "a {}" "a16161a0" | "a [ 1, 2 ]" "a16161820102" | "a @x" "a16161da000111d66178" | "a -> @x" "a16161da000111d76178" | "a #sha256:00ff" "a16161da000111d882667368613235364200ff" | "a ~hex:00ff" "a161614200ff" | "a !e/x 1" "a16161da000111d98263652f7801" | "a usd 1.99" "a16161da000111d58281da000111d463757364c4822118c7" | "a 2026-08" "a16161da000111da67323032362d3038" | "a 2026-08-03T14:22:07Z" "a16161da000111da74323032362d30382d30335431343a32323a30375a" | "a 18446744073709551615" "a161611bffffffffffffffff" | "a 340282366920938463463374607431768211456" "a16161c2510100000000000000000000000000000000" | "a { b 1, c 2 }" "a16161a2616201616302" | "a [ { b 1, c 2 }, { b 3, c 4 } ]" "a1616182a2616201616302a2616203616304" | "x [ | a b ]" "a1617880" | "a ~hex:00FF" "a161614200ff" | "a ^u8 [ 1, 2, 3 ]" "a16161d84043010203" | "a ^f64 [ 1.5 ]" "a16161d852483ff8000000000000" | "a ^i16:2x2 [ 1, -1, 2, -2 ]" "a16161d82882820202d849480001ffff0002fffe" | "a ^u8 [ 1, null, 3 ]" "a16161da000111db824105d84043010003" | "a ^f16 [ 1.5, -0, nan ]" "a16161d850463e0080007e00" | "a ^f32 [ 0.1 ]" "a16161d851443dcccccd" | "a ^i64 [ -1 ]" "a16161d84b48ffffffffffffffff" | "a ^u8 []" "a16161d84040" ] -- The other half of conformance: inputs a decoder MUST refuse. Each source -- line fails to parse. The harness passes an implementation on one of -- these when its command exits non-zero, and FAILS it when the command -- prints bytes — a decoder that accepts a malformed document has widened -- the language, and two implementations with different widenings read -- the same file differently while both report success. rejects vector [ | source why | "x [ | a b | 1 2 ]" "a row begins after a separator; see rows-are-separated" | "a ~b64:aGk" "base64 is padded to a multiple of four; see byte-payloads" | "a ~b64:aG-_" "base64url is not the alphabet; see byte-payloads" | "a ~hex:0" "hexadecimal digits come in pairs; see byte-payloads" | "a ~hex:0g" "g is not a hexadecimal digit; see byte-payloads" | "a ^f64:6 [ 1, 2, 3, 4, 5, 6 ]" "a one-dimensional shape is the length and is not written; see arrays" | "a ^u8 [ 256 ]" "256 does not fit u8" | "a ^u8 [ 1.5 ]" "a fraction in an integer array" | "a ^f64:2x2 [ 1, 2, 3 ]" "the shape holds 4 and 3 were written" | "a ^f64 [ 1.5f ]" "the type is the marker; no f inside an array" | "a ^x8 [ 1 ]" "no such element type" ] -- Bytes a decoder must refuse, as hex. The text above cannot produce them, -- which is why they are listed as bytes: a decoder that accepts them has -- accepted a second encoding of a value, or a value with no text. reject-bytes vector [ | hex why | "a16161d856483ff8000000000000" "tag 86 is a little-endian f64 array; only big-endian is canonical" | "a16161da000111db824107d84043010203" "a validity mask with every position present" | "a16161d852487ff8000000000001" "a NaN with a payload; the text has one NaN" | "a16161da000111db824105d84043010203" "a masked position holding non-zero bits" | "a16161d8524300ff00" "three bytes is not a whole number of f64 elements" | "a16161d828828103d84043010203" "tag 40 with one extent" ] } }