Omnist

Contents

1. 01  The problem
2. 02  The model
3. 03  What it enables
4. 04  Boringly correct
5. 05  The ports

One shape, hidden behind four formats

• JSON, YAML, TOML, and XML encode the same tree of data, but each ships its own parser, its own types, and its own edge cases — there's no shared model underneath.
• Schema evolution is unchecked: "will every old document still validate?" is a reviewer's guess, not a computed answer.
• Trimming a schema to what one service needs, or learning one from examples, means hand-editing — there's no principled operation for either.
• Omnist exists to close these gaps: one model, one schema language, and decidable operations over both.

A simple yet expressive formal model

OML — a document is a tree

host: "api.internal"
port: 8443
databases: {
    type: "prod"
    server: "db1.internal.example.com"
    port: 5432
}
databases: {
    type: "test"
    server: "db2.internal.example.com"
    port: 5433
}
tags: "prod"
tags: "us-east"

graph LR
    Root(("node")) -->|host| Host["api.internal"]
    Root -->|port| Port["8443"]
    Root -->|databases| D1(("node"))
    Root -->|databases| D2(("node"))
    Root -->|tags| T1["prod"]
    Root -->|tags| T2["us-east"]
    D1 -->|type| D1T["prod"]
    D1 -->|server| D1S["db1.internal.example.com"]
    D1 -->|port| D1P["5432"]
    D2 -->|type| D2T["test"]
    D2 -->|server| D2S["db2.internal.example.com"]
    D2 -->|port| D2P["5433"]
        

This is OML — Omnist Markup Language. The two databases edges share one label — that repetition is the array; there is no separate list node.

The same document, in JSON, YAML, TOML, XML

{
  "host": "api.internal",
  "port": 8443,
  "databases": [
    {"type": "prod", "server": "db1.internal.example.com", "port": 5432},
    {"type": "test", "server": "db2.internal.example.com", "port": 5433}
  ],
  "tags": ["prod", "us-east"]
}

host: api.internal
port: 8443
databases:
  - type: prod
    server: db1.internal.example.com
    port: 5432
  - type: test
    server: db2.internal.example.com
    port: 5433
tags:
  - prod
  - us-east

host = "api.internal"
port = 8443
tags = ["prod", "us-east"]

[[databases]]
type = "prod"
server = "db1.internal.example.com"
port = 5432

[[databases]]
type = "test"
server = "db2.internal.example.com"
port = 5433

<host>api.internal</host>
<port>8443</port>
<databases>
  <type>prod</type>
  <server>db1.internal.example.com</server>
  <port>5432</port>
</databases>
<databases>
  <type>test</type>
  <server>db2.internal.example.com</server>
  <port>5433</port>
</databases>
<tags>prod</tags>
<tags>us-east</tags>

Every reader converges on the same Document — converting between any of them is just read one, write another.

OSD — a schema is an automaton

record Database {
    "type": string,
    "server": string,
    "port": integer,
}
record Service {
    "host": string,
    "port": integer,
    "databases" [1,]: Database,
    "tags" [0,]: string,
}
root Service

graph LR
    Svc(("Service")) -->|"host [1,1]"| String(("string"))
    Svc -->|"port [1,1]"| Integer(("integer"))
    Svc -->|"databases [1,∞)"| Db(("Database"))
    Svc -->|"tags [0,∞)"| String
    Db -->|"type [1,1]"| String
    Db -->|"server [1,1]"| String
    Db -->|"port [1,1]"| Integer
        

Every record is a state; a field is a labeled, cardinality-counted transition to another state. Fields are order-free, so this is a fan, not a sequence. Comparing two such automata is what makes compatible_with decidable.

Schema algebra — the operations

compatible_with(A, B) -> bool   # can readers of B still read A's documents?
equivalent(A, B) -> bool        # same documents, different shape?
normalize(S) -> Schema          # canonical minimal form
extract(S, keep) -> Schema      # minimal subschema for a label set
infer(samples) -> Schema        # learn a schema from example documents
prune(S) -> Schema              # drop unreachable / unsatisfiable structure

Six operations, every port implements all of them. Decidable because the schema is an automaton, not a heuristic over examples.

Validating a document against a schema

record Database {
    "type": string, "server": string, "port": integer,
}
record Service {
    "host": string, "port": integer,
    "databases" [1,]: Database, "tags" [0,]: string,
}
root Service

host: "api.internal"
port: 8443
databases: { type: "prod", server: "db1.internal.example.com", port: 5432 }
tags: "prod"
tags: "us-east"

Every port validates this the same way, because it's the same schema, the same algebra, the same answer.

Genuinely hard, made decidable

Why Omnist, and not a heuristic

• Records are closed by default, and scalar types are never composed into enums or unions — the one deliberate opening is an explicitly marked any.
• That discipline is what makes compatible_with, equivalent, extract, infer, and normalize well-defined, decidable operations — not best-effort heuristics.
• Most schema tools check one document at a time; Omnist's algebra compares two whole schemas and proves the answer.
• Every port exposes the same operations as a library API, and — where implemented — a command-line tool. The algebra is identical; the surface each language offers around it may vary.

Checked, not asserted

• A two-track conformance harness — structural fixtures plus a JSON test-vector suite — that every port runs against the same omnist-spec pin.
• Formal ABNF grammars for OML and OSD, verified against every port's own parser.
• The schema algebra is cross-checked three separate ways against the same test cases, so no single method's blind spot can hide a bug.
• Each port independently gates its own coverage in CI — 100% in four ports, 99%+ in the fifth — and its own static-analysis discipline; the tools differ by language, the enforcement doesn't.