Work

Re-architected hourly ad impression and conversion processing from mutable tracker state to stateless, idempotent Iceberg pipelines. Runtime down 35%, Trino latency down 20%, incident recovery from a day to two hours.

An agentic Hive-to-Iceberg migration platform with mandatory human approval gates. Migrated 450 tables across 40 workflows with no production incidents, delivered by three engineers instead of roughly ten.

Migrated 2,500 Spark jobs from on-premises infrastructure to GCP Dataproc with phased cutovers. A separate AWS Keyspaces to GCP Bigtable migration delivered zero downtime and lowered annual datastore cost from approximately $2M to $200K.

Built in-house sensitive-data discovery for the ads data platform instead of buying it, then closed the loop between analyst corrections and training data to move precision from 84% to 91% without giving up recall.

A governed MCP data-access layer over roughly 5,000 tables, enforcing schema, lineage, ownership, identity, and metric certification before a query runs. Adopted by 120+ platform engineers.

Redesigned deletion across a petabyte-scale data lake using virtual-identifier rotation for immediate disassociation, cutting end-to-end fulfilment 94%. Separately re-architected data download from MapReduce to Spark for a 50× intake increase.

Productionised Slack incident-history and Confluence runbook-retrieval MCP servers for an internal AI incident assistant, with read allowlists, least-privilege scopes, and button-gated writes. Adopted by 20+ teams and 150+ engineers.