This is a point-in-time record of the week of 2026-06-29 to 2026-07-05. It was written from the repository's history for that week and is not revised afterwards, so later pages may supersede what it says.

Week of 2026-06-29 to 2026-07-05

Window: 2026-06-29 00:00:00 to 2026-07-05 23:59:59, local author time Recapped: 2026-08-24 Commits in window: 39 (30 non-merge, 9 merges)

This is a backfill written 50 days after the window closed. It is the week July began by changing the physical storage contract: silver small files gained a compactor, bronze compression was measured rather than guessed, and the Parquet lake moved to normalized month-level paths. A separate production fix stopped successfully scraped but dealer-unenriched listings from being fetched again every fifteen minutes.

What shipped

Plan 109 — compact the silver small-file stream

Plan 109 exists because a five-minute flush cadence produced roughly 150 Parquet files per source per day, multiplying query overhead and weakening compression. 2a0037c added daily, sorted, watermark-protected compaction with a reader-safe temporary-file sequence and a bounded backlog drain. Review follow-ups be01ca7, 2ce776a and b8bd1c1 made delete failure abort publication, isolated integration prefixes, and bypassed PyArrow's conflicting Hive inference. Those follow-ups name no plan and remain in the unattributed ledger.

Plan 115 — a successful scrape must close its retry circuit

Plan 115 fixed a production loop where customer_id IS NULL meant both “never fetched” and “successfully fetched without dealer metadata.” dd14e04 added last_detail_scraped_at, made only detail writes advance it, and suppressed another detail attempt for seven days. The same commit removed roughly 145 seconds of unit-test TCP timeouts by mocking external I/O. 9c9f6eb updated direct integration-test callers for the new write contract.

Plan 116 — measure recompression before building a migration

Plan 116 was a read-only gate for Plan 110's proposed bronze rewrite. 6212e85 added the estimator; 6f4cda5 replaced a prefix-wide buffered listing with a streaming paginator; and cffc1bd recorded the production result: level 9 saved a consistent 8% across May and June, too little to justify rewriting millions of old objects. The decision was to improve new writes and pursue higher-value storage work.

Plan 110 — normalize storage before experiments depend on it

Plan 110 broadened from HTML compression into storage-layout hygiene and Iceberg readiness. Its seven named commits were 9723180, dd52f31, 313f58c, 0c2f6f8, 2767ac9, 39b118d and b63a24a.

The implementation raised new bronze writes to zstd level 9, supplied an operator-only recompression tool, audited Parquet using listings and footers, selected a normalized month-level layout, and built a non-destructive rewrite with baseline checks and write-before-rename publication. The unnumbered 18c2121 then cut writers and dbt readers over to silver_normalized/ and ops_normalized/; 1e35c99 moved CI fixtures, while 3e2d308 removed physical partition columns so historical rewrites and fresh flushes shared one schema.

Plan 117 connected that storage work to the next feature, experiment and production-policy stages. 58a87af made Plans 110–113 a single evidence-driven arc: normalized storage, interpretable features, reproducible backtesting, then conservative deployment.

What moved between states

Three index revisions introduced six build-order entries:

Plan Before After
110 absent build order
111 absent build order
112 absent build order
113 absent build order
114 absent build order
115 absent build order

Plan 109 shipped in the window but produced no net state change in the index. Plan 116 also opened, measured its question and closed without appearing in a net transition above.

What is still owed

Unattributed commits

Eighteen non-merge commits name no numbered plan in their own subject or body. They are included in the narrative above where their work is clear.

Commit Subject
be01ca7 fix: address three code review findings in compact_silver
2ce776a fix: ruff lint errors in compact_silver files
b8bd1c1 fix: use pq.ParquetFile().read() to bypass hive partition inference
9c9f6eb fix: update integration processing tests for last_detail_scraped_at upsert param
f687daa fix: ruff lint errors in analysis scripts (E501, E741, I001)
6f4cda5 fix: use boto3 paginator in iter_prefix to avoid blocking s3fs.ls
1be9ba0 docs: fix Phase 6 deploy intent and compose service names
d41d6bd docs: fix Phase 6 Steps 4+5 to use host curl for archiver endpoints
54c9760 docs: fix Phase 6 Step 10/11 ordering — validate before unpausing DAGs
713b5e3 docs: fix rollback completeness after Step 10 controlled flush
924cd3c docs: remove fake CLI flags from Phase 6 Option B recovery
6171a8d docs: add ops_normalized validation to Phase 6 Step 10 and rollback
4088829 docs: check all five ops/ prefixes in Step 10 and fix flushed=0 wording
d19ccd1 finalizing audit script for furture use.
18c2121 Implement normalized parquet cutover
1e35c99 ci: update MinIO seed paths to normalized layout
c724d08 Pin dbt core below 2 in runner
3e2d308 fix: drop Hive partition cols from rewritten Parquet physical schema

Merges

Nine merges are named and not recapped separately.

Merge PR Branch
0d33512 #130 feature/plan-109-silver-compaction
a8d2622 #131 bugfix/detail-unenriched-circuit-breaker
7ee58c7 #132 feature/plan-116-estimate-recompression-savings
6310858 #133 feature/storage-refresh-implementation-plans
1217f9f #134 feature/plan-110-storage-layout-hygiene
3cd7fde #135 feature/plan-110-storage-layout-hygiene
a6ece65 #136 feature/plan-110-storage-layout-hygiene
5f7237e #137 feature/plan-110-storage-layout-hygiene
11f736d #138 feature/plan-110-storage-layout-hygiene

Deferred to the next recap

419 commits sit after this window's end as of 2026-08-24.

git rev-list --count --all --since='2026-07-05 23:59:59'   # 419

That raw tail includes the July 6–12 recap being written alongside this one and the already-covered weeks ending July 19, July 26, August 2, August 9, August 16 and August 23. Once both files in this backfill are present, none of the following complete weeks is genuinely uncovered.