Senior Data Engineer at IRU, running the data platform end to end: Kafka, Snowflake, dbt, Airflow, Terraform. Building agent-first since 2026.
Every AI system is only as good as the data underneath it. I build the platforms, standards, and guardrails that make that data trustworthy, cheap, and boringly reliable.
Streaming ingest, warehouse modeling, access control, and cost. Each project started as a written spec and ended in production, with a rollback plan.
p.01cost
−94%
Retiring a pipeline that ate a third of the warehouse
Tuned down, then retired, a vulnerability-matching pipeline that was 31% of all warehouse spend and still growing. I rebuilt it on change-data-capture at 99.9999% parity and cut over in 12 days.
[ok] spec → prod · rollback ready
p.02modeling
131 tables
An ingest that writes itself
Turned a hand-built data-lake ingest into a generator. It introspects live data, classifies dimensions vs facts, and emits typed models and tests. 131 of 143 tables were modeled in two weeks, with 1,173 passing build steps.
[ok] spec → prod · rollback ready
p.03streaming
~97% cheaper
Kafka at 1,000+ partitions
Moved streaming ingest off a realtime connector onto S3 sinks, cutting per-topic cost ~97%. Along the way I diagnosed connector OOMs from source code and shipped a 23-minute merge-to-rollback when a change misbehaved.
[ok] spec → prod · rollback ready
p.04access
13+ teams
Access control as code
Snowflake roles and grants for the whole company in one Terraform repo, with Okta for login. I also led a 7-checkpoint provider migration with audits and smoke tests at every gate.
[ok] spec → prod · rollback ready
p.05performance
35 h → <1 h
From O(n²) to O(n)
Replaced a paginated Kafka emit that re-scanned ~29 TB per run with a streamed cursor, and proved it with an A/B harness before shipping.
[ok] spec → prod · rollback ready
p.06matching
macOS → Windows
LLM-assisted vulnerability matching
Regex for precision, an LLM for the long tail. This matching layer helped the product expand from macOS-only to Windows detection.
[ok] spec → prod · rollback ready
transform→orchestrate
§03stage: orchestrate
Agent-first, human-gated.
I build with a personal multi-agent harness. Cheap models search and read, mid-tier models build, top-tier models review adversarially, and I make the calls. The result is more output without lowering the bar: every change still ships with a spec, a test, and a rollback.
I founded my team's standards repo and wrote its ingest standards and playbooks so that engineers and AI agents follow the same steps. I've interviewed and onboarded three engineers, and I run knowledge-transfer sessions until the systems I built no longer need me.
selected work · pages in progress
Essay[up next]
Standards agents can run
Rule IDs, registries, and playbooks: writing team standards that a bot can execute and a human can review.
Field notes[up next]
How to retire a pipeline
Build replacements first, prove parity, soak, then delete, and keep history readable.
Deep dive[later]
Access control as code
One repo, three role layers, and Okta for login only. Why human access belongs in Terraform.
Field notes[up next]
How to build a greenfield
Standing up a cybersecurity data pipeline from nothing: Kafka in, regex + LLM matching, incremental models out, and keeping it alive after the PoC became a product.
Incident notes[later]
Kafka Connect at 1,000 partitions
Heap math, restart loops, and the rebalancing change I rolled back in 23 minutes.
Explainer[up next]
My agent harness
Model routing, skills, adversarial review, and why the human gate matters.