Schema algebra

Six decidable operations over schemas — comparison, minimization, extraction.

Most schema tools check one document at a time. Omnist's algebra compares two whole schemas and proves the answer — decidable because a schema is a closed automaton, not a heuristic over examples.

Why compatible_with matters: you're about to ship a schema change. Will every document ever written under the old schema still validate against the new one? That's not a question you want a reviewer guessing at — it's exactly what a CI gate can prove before merging:

v1 = parse_schema('record R { "host": string }\nroot R')
v2 = parse_schema('record R { "host": string, "port" [0,1]: integer }\nroot R')

v1.compatible_with(v2)     # can readers of v2 still read v1's documents?
v2.compatible_with(v1)     # the reverse -- v1 has no 'port' field to require

Output

v1.compatible_with(v2): True   -- adding an optional field is backward compatible
v2.compatible_with(v1): False  -- v1 can't produce the 'port' field v2 might require

Why equivalent matters: two schemas can describe the exact same set of documents while looking structurally nothing alike — different record names, different nesting, the same shape underneath. That happens constantly: two teams independently modeling the same data, or a refactor that reorganizes records without changing what they accept. equivalent answers "did anything actually change?" without a human eyeballing a diff:

a = parse_schema('record Left  { "x": string, "y": integer }\n'
                  'record Right { "x": string, "y": integer }\n'
                  'record Root  { "a": Left, "b": Right }\nroot Root')
b = parse_schema('record Pair { "x": string, "y": integer }\n'
                  'record Root { "a": Pair, "b": Pair }\nroot Root')

a.equivalent(b)

Output

True   -- Left and Right are two different names for the same shape;
         a and b accept exactly the same documents

Why normalize matters: the schema above (a) carries dead weight — Left and Right are two spellings of one record. That's easy to end up with after independent authoring, or after infer (next step) drafts one record per sample instead of noticing the repeat. normalize collapses it to the canonical minimal form — same documents accepted, fewer records to maintain:

a.normalize()

Output

record Left {
    "x": string,
    "y": integer,
}
record Root {
    "a": Left,
    "b": Left,
}
root Root

Why extract matters: a shared schema often has more fields than any one consumer needs — a microservice reading only host shouldn't have to carry port's validation rules too. extract(*labels) computes the minimal subschema recognizing only documents built from the kept labels, dropping anything the removal makes unreachable. Deleting a mandatory field is an error, not silently allowed:

v2.extract("host")   # subschema with only "host" -- "port" dropped

Output

record R {
    "host": string,
}
root R

Why is_empty/prune matter: a schema can accidentally describe no documents at all — a mandatory reference cycle with no base case, easy to introduce by typo in a large hand-written schema and very hard to spot by reading. is_empty() catches it before it ships; prune() strips never-emittable fields and unreachable records left over from other edits:

empty = parse_schema('record A { "x": B }\nrecord B { "y": A }\nroot A')
empty.is_empty()
empty.compatible_with(v1)   # vacuous: an empty schema accepts no documents

Output

is_empty: True
empty.compatible_with(v1): True

Why lint matters: a pre-flight check for a schema you're about to ship or one a tool generated — the same duplication normalize fixes, flagged first without changing anything, so a CI gate can require a human look at it:

from omnist import lint

dup = parse_schema('record A { "x": string }\nrecord B { "x": string }\n'
                    'record Root { "a": A, "b": B }\nroot Root')
for finding in lint(dup):
    print(finding)

Output

LintFinding(code='lint.duplicate-record', severity='warning', location='A, B',
            message="records 'B' are structurally identical to 'A'; merge them with `schema normalize`")

Schema algebra

Learn more