This is a point-in-time record of the week of 2026-07-13 to 2026-07-19. It was written from the repository's history for that week and is not revised afterwards, so later pages may supersede what it says.
Week of 2026-07-13 to 2026-07-19
Window: 2026-07-13 00:00:00 to 2026-07-19 23:59:59, local author time Recapped: 2026-08-24 Commits in window: 88 (63 non-merge, 25 merges)
This is a backfill written 36 days after the window closed. It records a week in which the lakehouse stopped being an architectural intention and became a working, repeatedly corrected system: Lakekeeper, Spark, Iceberg, production-derived snapshots and MLflow provenance were joined into one local rehearsal path, then the harder DuckDB-to-Iceberg migration began.
What shipped
Plan 124 — contain browser failures inside the solver
Plan 124 was opened after a
browser process consumed enough memory to threaten the host. 2d09363 added
container memory, swap and PID limits plus tests and the plan document;
678f31f raised the browser allowance after the first limits proved unstable,
and 8354e14 aligned the tests and documentation. None of those three commits
names Plan 124 in its own text, so they remain in the unattributed ledger even
though their changes are described here.
Plan 112 — prove the open-lakehouse foundation
Plan 112 needed a
reproducible experiment substrate before adaptive-refresh policy work could be
trusted. The week audited the existing features, stood up an isolated
Lakekeeper REST catalog, proved Spark/Iceberg writes and time travel against
MinIO, rehearsed a real volatility-feature export, added a one-command local
snapshot-to-lakehouse path, and attached MLflow provenance. The eleven commits
that name the plan are 67a1d1d, cf92f02, 74ab140, 448ef35, 79050b0,
6c801e0, b9ce29a, c243cfd, 89043b1, 062aaaf and 10bc4c9.
The correction chain matters as much as the first implementation. It fixed
Lakekeeper image and health assumptions, bootstrapped the empty catalog,
created the CI bucket before warehouse registration, moved Spark from S3A to
Iceberg's S3FileIO, made cleanup use the catalog's real UUID path, removed an
x86-only JAVA_HOME, and separated the production analytics volume from the
CI Compose path. Those fixes are named in the unattributed section rather than
silently absorbed into the plan count.
Plan 120 — make production-derived fixtures portable
Plan 120 separates expensive
snapshot production from Plan 112's consumption of stable fixtures. Gate E
packaged and uploaded fingerprinted archives, and Gate F added an authenticated
ops/admin download path with alias and manifest cross-checks. The three
attributed commits were 92d610c, c1e7195 and the shared local-rehearsal
commit 89043b1. Follow-ups hardened the archive prefix, identity checks,
OAuth bypass for the authenticated download, MinIO dependency, cache reuse and
error handling.
Plan 125 — begin making Iceberg the analytics contract
Plan 125 exists because Plan 112 proved an Iceberg sidecar but left DuckDB as the real analytics contract. The week kept Lakekeeper behind catalog-neutral configuration, audited 22 models and their readers, proved dbt-spark against Iceberg, ported the ten-model volatility chain to exact parity, verified late-arrival behavior, and measured rather than guessed the first Gate C shape decisions.
The work also withdrew a lakehouse CI job after its cold-start fixture problem
survived an attempted patch; the local and VM proofs remained valid, but no
longer masqueraded as CI coverage. The thirteen attributed commits were
066be10, e063f94, 76aac3f, 498f7a6, 0bb36c8, 96926e6, 5bae544,
3f75283, 430fd15, b63c4f6, 863a2f2, ff33c99 and 9f34557.
Two future streaming documents also entered the record: Plan 126 framed a
small replayable event backbone around the existing staging-event seam, and
Plan 127 reserved adaptive scrape control for after that transport and its
inputs are stable. Their authoring commits (396911a, ac0f2c9) name no plan
number and therefore remain unattributed below.
What moved between states
Four revisions of docs/PLANS.md changed the index during the window:
| Plan | Before | After |
|---|---|---|
| 87 | backlog | superseded |
| 118 | build order | superseded |
| 124 | absent | build order |
| 125 | absent | build order |
| 126 | absent | build order |
| 127 | absent | build order |
The state changes match the architectural handoff: Plan 125 replaced Plan 118's broader migration placeholder, while Plan 126 replaced the Kafka-shaped Plan 87 with an event-semantics-first design.
What is still owed
- Plan 112 paused after foundation proof; adaptive-refresh model and policy work still depended on Plan 125 making Iceberg a real analytics contract.
- Plan 120 had shipped archive delivery, but its later completion gates and production verification were not closed in this window.
- Plan 125 Gate C still owed production-scale evidence and the actual reader cutover. The week ended with partition specifications decided but not applied.
- Plans 126 and 127 were future designs only. The transport plan depended on a stable Plan 125 substrate; adaptive scrape control depended on the transport and on settled feedback semantics.
- Plan 124 had limits in code, but production verification and the corrected distinction between host-wide and container-local OOMs came later.
Unattributed commits
Thirty-seven non-merge commits name no numbered plan in their own subject or body. Each is included in the work above or described by its subject here; the missing fact is the durable plan attribution, not the work.
| Commit | Subject |
|---|---|
2d09363 |
Add Trawl memory guardrails |
64df879 |
Align audit script not_null checks with schema.yml and doc |
678f31f |
Increase trawl memory limits to fix unstable browsers. |
8354e14 |
fixed tests to match new defaults, and updated docs. |
78b157b |
fixing doc wording on lakehouse_subtrate doc |
d530ec2 |
fix lakekeeper image |
91f68a2 |
Fixing Lakehouse healthcheks. |
24dfada |
fixing the health check to check on infra not catalog. |
7ff95ff |
fix dependencies for pytest. |
5a5244a |
Fix Lakekeeper warehouse payload schema + run A2 round-trip in lakehouse-worker |
057b2cb |
Fix lakehouse-worker build: pin base image to bookworm for openjdk-17 |
c0a4e22 |
Fix CI lakehouse job: create bronze bucket before warehouse registration |
efbcc62 |
Fix warehouse registration: bootstrap the Lakekeeper server first |
6d3eeb4 |
Fix Lakekeeper warehouse payload schema + run A2 round-trip via S3FileIO |
ec68a12 |
Fix cleanup: drop PURGE, derive MinIO prefix from real table location |
fa9f739 |
Fix lakehouse-worker JAVA_HOME: resolve arch-suffixed JVM path dynamically |
c0bf40e |
Mark Gate A2 verified: real VM run results, close out hedged docs language |
5741988 |
Fix Gate A3: split analytics volume mount out of CI-shared compose file |
09b0a22 |
Fix Gate A3: add missing pytz dependency for DuckDB timestamp conversion |
0a73077 |
Fix Gate A3: duckdb .arrow() returns a RecordBatchReader, not a Table |
a0a6b55 |
Fix Gate A3: add setuptools for the distutils shim PySpark 3.5.3 needs |
bc2305d |
Mark Gate A3 verified: real VM run results, close out A3 open items |
99205d1 |
Fix A4 local harness: MinIO cred defaults, drop expected/ from seed check |
9c8c0ba |
Add .cache/ to .gitignore |
0846459 |
Constrain Gate F alias-referenced keys to the known archive prefix |
43dbc78 |
Cross-check alias/manifest snapshot_id against the requested snapshot_id |
1a55627 |
Route snapshot downloads around browser OAuth |
687e6c4 |
Add ops MinIO client dependency |
8a1c6fa |
Allow snapshot manifests reused by archive cache |
7b7fd49 |
Address review: broaden Gate F error handling, fix stale docs claims |
1700761 |
Add Docker build context ignore rules |
fcaaa1f |
Address review: clean CLI input errors, runbook table-lifecycle + Smoke B network caveats |
5a9a9a5 |
Use Python 3.12 for MLflow image build |
4c1eff1 |
Package MLflow provenance client in image |
396911a |
Add streaming roadmap plans |
ac0f2c9 |
Document reference-table snapshot and streaming path |
ff5ee6b |
Fix Gate A review findings: vacuous parity, stale examples, weak guard |
Merges
Twenty-five merges are named and not recapped as separate work.
| Merge | PR | Branch |
|---|---|---|
8dcab21 |
#154 | hotfix/plan-124-trawl-memory-guardrails |
624a387 |
#155 | feature/plan-112-iceberg-preflight |
3f0394d |
#156 | fix/trawl-resources |
344da84 |
— | remote-tracking reconciliation merge |
bc98fc7 |
#157 | feature/plan-112-iceberg-preflight |
3b9c95d |
#158 | feature/plan-112-iceberg-preflight |
3c07f1a |
#159 | feature/plan-112-iceberg-preflight |
ff02f2c |
#160 | feature/plan-112-iceberg-preflight |
82660ce |
#161 | feature/plan-112-iceberg-preflight |
59161c3 |
#162 | feature/plan-112-iceberg-preflight |
51a7833 |
#163 | feature/plan-112-iceberg-preflight |
616a1d7 |
#165 | feature/plan-120-gate-e-archive-packaging |
d19e1d5 |
#166 | feature/plan-112-a4-local-snapshot-lakehouse |
d13c905 |
#167 | feature/plan-120-gate-f-snapshot-downloads |
05d2493 |
#168 | feature/plan-120-gate-f-snapshot-downloads |
bf04734 |
#169 | feature/plan-120-gate-f-snapshot-downloads |
89116a7 |
#170 | feature/plan-120-gate-f-snapshot-downloads |
c8c90d7 |
#171 | feature/plan-112-a4-local-rehearsal-runner |
2ceb617 |
#172 | feature/plan-112-mlflow-tracking-bridge |
aa7288e |
#173 | feature/plan-112-mlflow-tracking-bridge |
1740e81 |
#174 | feature/plan-112-mlflow-tracking-bridge-clean |
4148749 |
#175 | docs/plan-112-consolidation-plan-125 |
03723ad |
#176 | feature/plan-125-catalog-neutral-preflight |
3f9b00e |
#177 | docs/plans-126-127-streaming |
daad69d |
#178 | feature/plan-125-portability-audit |
Deferred to the next recap
241 commits sit after this window's end as of 2026-08-24.
git rev-list --count --all --since='2026-07-19 23:59:59' # 241
That raw tail includes the July 20–26 recap being written alongside this one and the already-covered weeks ending August 2, 9, 16 and 23. Once both files in this backfill are present, none of the following complete weeks is genuinely uncovered.