This is a point-in-time record of the week of 2026-07-06 to 2026-07-12. It was written from the repository's history for that week and is not revised afterwards, so later pages may supersede what it says.
Week of 2026-07-06 to 2026-07-12
Window: 2026-07-06 00:00:00 to 2026-07-12 23:59:59, local author time Recapped: 2026-08-24 Commits in window: 90 (74 non-merge, 16 merges)
This is a backfill written 43 days after the window closed. It was an unusually large week: normalized storage became the input to adaptive-refresh features, production-derived snapshot export grew from a scaffold into an atomic filtered writer, and dbt's largest models gained explicit resource and incremental contracts. In parallel, a Cloudflare/solver incident drove runtime scraper controls that were mostly not tied to numbered plans in their commits.
What shipped
Plans 111 and 112 — define the feature layer and its experiment boundary
Plan 111 asks which listings
deserve frequent detail refresh, without changing production policy. 411be7f
added state fingerprints, runs and interpretable volatility features; eded499
pinned a stable dbt 1.10 stack after dependency drift. d048bf9 corrected a
run-boundary expectation but did not name the plan.
Plan 112 then separated
feature definitions from reproducible policy experiments. 01b11fd rewrote
the roadmap around snapshots and evidence rather than an immediate production
policy. Other roadmap edits (5c72cc0, b86749b, ffab8ec, 5cc02fd,
f4fa967) are unattributed by the commit-text rule.
Plan 120 — turn production lake data into a safe fixture
Plan 120 exists so CI and
local Spark/dbt work can consume a reproducible, production-derived fixture
without querying production directly. Its ten attributed commits were
b7a051a, d674c8d, 77d0d69, 2062d55, d3a8c73, 0ae4e5d, bb95f8c,
e6bf546, d9a92bf and e306ba6.
The work progressed from an archiver-owned export scaffold through read-only source audit, selectors, cohort closure, a profile-gated planning worker and fingerprinted cache, then Gate D's filtered Parquet writer. Review corrected several load-bearing details: selector dates were anchored to the requested window, closure was limited to vehicle-identity edges, generation paths became atomic, cache and promotion validation tightened, and a boundary VIN was seeded so a selector's evidentiary row could not be filtered out. Progress logging and physical production-shaped fixtures made long runs inspectable and meaningful.
Plan 123 — make dbt resource use bounded and incremental semantics explicit
Plan 123 required evidence before
incrementalizing models merely because they looked expensive. Its twelve named
commits were 755c39c, 5ae6f56, 37a92f6, 13b73e5, e908aec, 9cf11ed,
5de25ff, e468bde, 1762764, f216283, 3c43aa2 and 55f8a5a.
The week added dbt memory/thread guardrails, split hourly and feature cadences, incrementalized state and observation fingerprints plus affected-VIN models, and extended the full source history into observation runs. Fixture tests were converted from shadow SQL assertions into phased builds that exercise bootstrap, incremental arrival, correction and control-VIN behavior. Several commits in that test-hardening chain do not name Plan 123 and are therefore listed below.
Scraper availability and runtime controls
A separate production incident raised the detail timeout, hardened Cloudflare
session replay, corrected browser impersonation, pinned FlareSolverr, and added
the trawl solver sidecar (6a063f9, b47241d, dc3de90, 57208c6,
262bfbd, e5460b5). 63ce9a3 moved throttling into runtime settings rather
than another image rebuild. These commits explain why Plan 122 entered backlog,
but none names that plan in its own subject or body, so they remain unattributed.
What moved between states
Three index revisions produced these net changes:
| Plan | Before | After |
|---|---|---|
| 110 | build order | absent |
| 111 | build order | absent |
| 118 | absent | build order |
| 119 | absent | build order |
| 120 | absent | build order |
| 121 | absent | build order |
| 122 | absent | backlog |
Plans 110 and 111 left the live index after their immediate implementation work; Plans 118–121 expressed the broader lakehouse execution, governance, fixture and staging tracks; Plan 122 captured the operational lesson from the timeout incident.
What is still owed
- Plan 120's Gate D writer existed, but archive packaging, authenticated delivery and end-to-end local lakehouse rehearsal belonged to the next week.
- Plan 123 had incremental semantics and tests, but the VM resource baseline still had to be filled from production evidence rather than the template.
- Plan 112 had an experiment roadmap but not yet the Lakekeeper/Spark/Iceberg substrate or MLflow provenance bridge.
- Plans 118–121 were newly filed implementation tracks, not delivered systems.
- Plan 122 entered backlog because timeout and throttling had been changed in code; a complete supported runtime configuration surface was still owed.
- The scraper fixes were operationally urgent but weakly connected to plan text: 49 of 74 non-merge commits in the week carry no numbered attribution.
Unattributed commits
Forty-nine non-merge commits name no numbered plan in their own subject or body. They are still part of the recap; their subjects preserve what changed.
| Commit | Subject |
|---|---|
d048bf9 |
fix: remove next_state_started_at from test_runs_three_runs_correct_boundaries expected rows |
1268f0c |
fix: pin dbt/Dockerfile (tools profile) to stable 1.10 stack |
684c605 |
fix: support Airflow 3 DagBag tests |
5c72cc0 |
docs: update lakehouse roadmap for delta track |
b86749b |
more doc work, settling on plans. |
4dd57ca |
fix: return 400 for malformed snapshot payloads, validate window months |
640a147 |
refactor: extract shared DuckDB/MinIO S3 connection helper, dedupe across scripts |
e340fe0 |
fix: add pytz to archiver runtime requirements |
fbfbd61 |
feat: run initial lake snapshot selectors |
6a063f9 |
fix: increase detail scrape timeout |
ffd73b6 |
fix: aggregate selector counts in SQL, gate source_base_path, wire coverage |
ffab8ec |
adding docs |
63ce9a3 |
fix: throttle detail scraping via runtime settings |
b47241d |
fix: harden cloudflare session replay |
dc3de90 |
fix: align curl cffi with chrome 148 |
57208c6 |
fix: avoid unsupported chrome 148 impersonation |
262bfbd |
fix: pin flaresolverr image |
e5460b5 |
chore: add trawl solver sidecar |
d6ebebe |
style: wrap cloudflare header helper signature |
dfd084e |
fix: harden lake snapshot seeder/downloader per review findings |
40ced38 |
fix: expose export_ci_lake_snapshot DAG params via DAG(params=...) |
4a12dba |
fix: anchor price_changed_7d/30d_only to real window_end, not MAX(event_at) |
e104f2e |
refactor: move lake snapshot selector SQL and metadata out of Python |
2ef598a |
refactor: split selector config loading out of lake_snapshot_selectors |
3dadd3a |
fix: make lake-snapshot fixture physically production-shaped for silver |
da9506b |
fix: install s3fs in CI seed step, consolidate CI deps to requirements files |
e6d3954 |
fix: resolve CI deps in a single pip invocation to avoid version drift |
e45363e |
fix: unpin apache-airflow to avoid starlette/prometheus-instrumentator conflict |
48a30ae |
fix: isolate Airflow into its own CI venv instead of unpinning |
fa9c477 |
Add progress logs for snapshot planning worker |
b50c94a |
Configure logging in the snapshot-worker CLI entrypoint |
5cc02fd |
add updated docs |
a8d410e |
Constrain snapshot closure to vehicle identity edges |
5c41269 |
Add boto3 to archiver requirements for planning cache writes |
6736e18 |
Implement Gate D: fingerprint-addressed filtered Parquet writer |
fa49505 |
Fix Gate D publish-atomicity gaps: generation paths, manifest write failures |
255ff51 |
Make selector coverage non-blocking by default; fix stale_listing as-of semantics |
1ae6daa |
Fix stale_listing evidentiary rows and enforce selector checks in dry-run too |
e5af22c |
Seed stale_listing's boundary vin into cohort closure |
c8ebe14 |
Bump COHORT_ALGORITHM_VERSION for boundary-vin closure seeding change |
f4fa967 |
Update analytics and staging roadmap plans |
b133928 |
Update Gate D integration expectation |
964c49b |
Dedupe artifact_id within a single incremental batch |
3435721 |
Fix CI parse error: unit tests need explicit is_incremental override |
3560706 |
Remove stale days_on_market from int_price_history unit test fixtures |
4757605 |
Fix observation-fingerprint dedupe to distinguish true corrections from newer fetches |
281ac04 |
Replace shadow dbt incremental tests with phased lake fixture coverage |
d4c0e74 |
Address review: clarify watermark comment, extend runs full-refresh check to control VIN |
e3a2e1a |
Fix mismatched expect-row columns in observation_runs unit test |
Merges
Sixteen merges are named and not recapped separately.
| Merge | PR | Branch |
|---|---|---|
4ae87d6 |
#139 | feature/plan-110-storage-layout-hygiene |
c809681 |
#140 | feature/plan-110-storage-layout-hygiene |
b27893d |
#141 | hotfix/flaresolverr-detail-timeout |
df4338b |
#142 | fix/403-issues |
18148fb |
#143 | fix/403-issues |
f3ee1ff |
— | origin/master into feature/plan-112-refresh-backtesting |
80b1574 |
#144 | feature/plan-112-refresh-backtesting |
b57fa09 |
#145 | feature/plan-120-worker-progress-logs |
0ba20d5 |
#146 | feature/plan-123-dbt-resource-guardrails |
1e315b3 |
#147 | feature/plan-123-dbt-cadences |
5560e3a |
#148 | feature/plan-123-incremental-fingerprints |
2ae213e |
#149 | feature/plan-123-affected-vin-incrementals |
f6f2b9a |
#150 | feature/plan-123-affected-vin-incrementals |
cf61c82 |
#151 | feature/plan-123-phase-5-profiling-scaffold |
031c3f0 |
#152 | feature/plan-123-scrape-volume-incremental |
4a65ea7 |
#153 | feature/plan-123-observation-runs |
Deferred to the next recap
329 commits sit after this window's end as of 2026-08-24.
git rev-list --count --all --since='2026-07-12 23:59:59' # 329
That raw tail is already covered by the recaps ending July 19, July 26, August 2, August 9, August 16 and August 23. None of the following complete weeks is genuinely uncovered.