Skip to content
← back to projects
2026·shipped·Forward Deployed Engineer

Data Contract Enforcer

Schema validation and lineage across heterogeneous sources. Generate contracts from samples, validate live data, and attribute violations back to the upstream system that broke them.

10 Academy TRP · 1-week brief
PythonSchema ValidationLineage GraphsETL
Code ↗

Pipeline

1
ContractGenerator
Infers contract from sample
↓
2
ValidationRunner
Checks live data
↓
3
ViolationAttributor
Walks lineage to source
↓
4
SchemaEvolutionAnalyzer
Tracks drift over time

Highlights

  • ▸ContractGenerator infers contracts from sample data, no schema hand-writing required.
  • ▸ViolationAttributor walks the lineage graph to identify the specific upstream system that broke the contract.
  • ▸SchemaEvolutionAnalyzer tracks statistical drift over time and flags silent corruption.

Components

  • ContractGenerator: takes a sample and infers a contract (types, nullability, distributions).
  • ValidationRunner: checks live streams against contracts on a schedule.
  • ViolationAttributor: when a downstream contract fails, walks the lineage graph to locate the actual source of the break.
  • SchemaEvolutionAnalyzer: tracks distribution drift, not just schema changes.

Why attribution matters

The frustrating case in data engineering is not the loud break. It’s the silent one, where a downstream dashboard is slowly wrong for a week because an upstream field started drifting. The Attributor exists to shorten that loop from a week to minutes.