Omnist

Contents

1. 01  The problem
2. 02  The model
3. 03  What it enables
4. 04  Boringly correct
5. 05  Get started

One shape, hidden behind four formats

• JSON, YAML, TOML, and XML encode the same tree of data, but each ships its own parser, its own types, and its own edge cases — there's no shared model underneath.
• Schema evolution is unchecked: "will every old document still validate?" is a reviewer's guess, not a computed answer.
• Trimming a schema to what one service needs, or learning one from examples, means hand-editing — there's no principled operation for either.
• Omnist exists to close these gaps: one model, one schema language, and decidable operations over both.

A simple yet expressive formal model

OML — a document is a tree

host: "api.internal"
port: 8443
databases: {
    type: "prod"
    server: "db1.internal.example.com"
    port: 5432
}
databases: {
    type: "test"
    server: "db2.internal.example.com"
    port: 5433
}
tags: "prod"
tags: "us-east"

graph LR
    Root(("node")) -->|host| Host["api.internal"]
    Root -->|port| Port["8443"]
    Root -->|databases| D1(("node"))
    Root -->|databases| D2(("node"))
    Root -->|tags| T1["prod"]
    Root -->|tags| T2["us-east"]
    D1 -->|type| D1T["prod"]
    D1 -->|server| D1S["db1.internal.example.com"]
    D1 -->|port| D1P["5432"]
    D2 -->|type| D2T["test"]
    D2 -->|server| D2S["db2.internal.example.com"]
    D2 -->|port| D2P["5433"]
        

This is OML — Omnist Markup Language. The two databases edges share one label — that repetition is the array; there is no separate list node.

The same document, in JSON, YAML, TOML, XML

{
  "host": "api.internal",
  "port": 8443,
  "databases": [
    {"type": "prod", "server": "db1.internal.example.com", "port": 5432},
    {"type": "test", "server": "db2.internal.example.com", "port": 5433}
  ],
  "tags": ["prod", "us-east"]
}

host: api.internal
port: 8443
databases:
  - type: prod
    server: db1.internal.example.com
    port: 5432
  - type: test
    server: db2.internal.example.com
    port: 5433
tags:
  - prod
  - us-east

host = "api.internal"
port = 8443
tags = ["prod", "us-east"]

[[databases]]
type = "prod"
server = "db1.internal.example.com"
port = 5432

[[databases]]
type = "test"
server = "db2.internal.example.com"
port = 5433

<host>api.internal</host>
<port>8443</port>
<databases>
  <type>prod</type>
  <server>db1.internal.example.com</server>
  <port>5432</port>
</databases>
<databases>
  <type>test</type>
  <server>db2.internal.example.com</server>
  <port>5433</port>
</databases>
<tags>prod</tags>
<tags>us-east</tags>

Every reader converges on the same Document — converting between any of them is just read one, write another.

OSD — a schema is an automaton

record Database {
    "type": string,
    "server": string,
    "port": integer,
}
record Service {
    "host": string,
    "port": integer,
    "databases" [1,]: Database,
    "tags" [0,]: string,
}
root Service

graph LR
    Svc(("Service")) -->|"host [1,1]"| String(("string"))
    Svc -->|"port [1,1]"| Integer(("integer"))
    Svc -->|"databases [1,∞)"| Db(("Database"))
    Svc -->|"tags [0,∞)"| String
    Db -->|"type [1,1]"| String
    Db -->|"server [1,1]"| String
    Db -->|"port [1,1]"| Integer
        

Every record is a state; a field is a labeled, cardinality-counted transition to another state. Fields are order-free, so this is a fan, not a sequence. Comparing two such automata is what makes compatible_with decidable.

Schema algebra, in code

s1.compatible_with(s2)   # can readers of s2 still read s1's docs?
s1.equivalent(s2)        # same documents, different shape?
s1.normalize()           # canonical minimal form
s1.extract("host", "port", "databases", "type", "server")  # drop tags, keep the rest

Same three operations named on the next two slides — this is what actually calling them looks like. Decidable because the schema is an automaton, not a heuristic over examples.

The shortest possible tour

from omnist import parse_schema, doc

s = parse_schema('''
    record Database { "type": string, "server": string, "port": integer }
    record Service  { "host": string, "port": integer,
                       "databases" [1,]: Database, "tags" [0,]: string }
    root Service
''')

s.validate(doc({"host": "api.internal", "port": 8443,
                "databases": [{"type": "prod", "server": "db1.internal.example.com", "port": 5432}],
                "tags": ["prod", "us-east"]})).ok   # True

Genuinely hard, made decidable

Why Omnist, and not a heuristic

• Records are closed by default, and scalar types are never composed into enums or unions — the one deliberate opening is an explicitly marked any.
• That discipline is what makes compatible_with, equivalent, extract, infer, and normalize well-defined, decidable operations — not best-effort heuristics.
• Most schema tools check one document at a time; Omnist's algebra compares two whole schemas and proves the answer.
• Every operation is also a command, not just a Python call — the omnist CLI puts convert, validate, infer, and the schema comparisons on the command line too.

Checked, not asserted

• 100% line coverage, and a mypy --strict CI gate.
• Property-based fuzzing, plus formal ABNF grammars for OML and OSD verified against the parsers.
• The schema algebra is cross-checked three separate ways against the same test cases, so no single method's blind spot can hide a bug.
• Performance is measured, not implied: a 100k-edge document validates in about a third of a second.