This is a point-in-time record of the week of 2026-08-10 to 2026-08-16. It was written from the repository's history for that week and is not revised afterwards, so later pages may supersede what it says.
Week of 2026-08-10 to 2026-08-16
Window: 2026-08-10 00:00:00 to 2026-08-16 23:59:59, local author time Recapped: 2026-08-21 Commits in window: 55 (42 non-merge, 13 merges)
The week bronze storage stopped being a byte problem and became an inode problem, and then stopped being either. Three plans ran end to end: 114 closed with a measured no, 129 went to production, and 131 went from first draft to deleting real objects in four days. Four new plan documents were born out of what those runs found.
What shipped
Plan 114 — sectioned HTML, rejected on the numbers
Plan 114 asked whether detail-page HTML could be split into stable reusable sections and stored as a verified manifest, which would have bought far longer artifact retention than a 30-day window. It was audit-first by design: prove it before putting it in the scrape path.
1f0e847 closed that question against 60 real production artifacts. The
correctness gates passed completely — 100% byte-identical reconstruction, 100%
parser-equivalent, zero failures — and the storage case did not survive contact
with the real backend. Whole-section reuse across listings is 0.65%, and
MinIO's ~8 KB per-object floor turned 60 artifacts into 556 section objects
costing 4.3 MiB of padding against a 1.9 MiB baseline: a 223% regression, with
no granularity at which it wins. The value of the commit is that it says no on
a measurement rather than on taste, and names the two successors that inherit
the problem — 129
(reversible) and 130 (not).
Plan 129 — dictionary compression, shipped mid-rollout
Plan 129 compresses bronze HTML against a zstd dictionary trained on a corpus sample instead of compressing each page as an independent frame. It matters because it is reversible — nothing is discarded, every artifact stays independently decompressable — which is why it ships before Plan 130.
38d7e5apassed Stage 0's gate at −73.15% on held-out listings from held-out months, against a 40% bar, and reported four splits rather than one because each holdout can be gamed by the sample. It also promoted the backfill from out-of-scope to Stage 4: Stages 1-3 change new writes only and free zero existing bytes, and with the volume at 98% that distinction is the plan.9f078c6restored plain level-9 recompression as a first-class mode. The preceding commit had made--dictionary-idmandatory, which removed the only recompression route needing no migration, no registered dictionary and no service rebuild — exactly the route that is usable while the disk is full.f98e69bfixed a checkpoint that was O(n²): measured mid-run at 66,130 keys the checkpoint file was 6.4 MiB and cost ~51 ms per object, a hard ceiling of 19 objects/sec from bookkeeping alone, with the observed rate already decayed from 42/s to 8/s.f4e0f0brecorded the production rollout and what it taught, including the finding that a full disk blocks the backfill entirely — MinIO's minimum-free threshold refuses even the small writes that would free space.
Plan 131 — packed cold storage, draft to production deletes in four days
Plan 131 exists because Plan 129 cut bronze bytes ~73% and changed object count by zero. Object count is the constraint compression cannot touch, and at ~65,500 inodes/day the volume had about 61 days of headroom. Thirty-three commits, the whole arc:
Scoping — 3282919 opened the plan; 6bc6723 corrected it three ways,
moving the work into the archiver rather than a script, promoting grouping from
a settled decision to the plan's biggest open question, and recording why
deferred compression was rejected.
Stage 0, the gate that could fail — 48ee624 measured against production
and killed the cheap alternative: results pages are 0.7% of the object
population, so deleting every one ever captured buys about half a day of inode
headroom. 54cc075 then withdrew its own conclusion, on the grounds that the
5% penalty estimate had assumed away the base/delta asymmetry that is the
entire premise of packing by listing. 18807e2 added the script to measure B
and D, the two numbers the layout decision actually needs; ddeddbb recorded
that the archiver image had to be rebuilt before it could pack at all;
f447ee8 made the script refuse an undictionaried baseline rather than
silently overstate the win. 01f8d60 settled it: monthly capture buckets, 1.9
physical points behind the per-listing ceiling, B = 8,578 bytes and D = 2,142.
Stages 1-2 — 6d0092a landed the pack format and the packer.
705095b replaced an arbitrary 60-day age threshold with month-completion
eligibility, which as of that day was the difference between packing April-May
and April-July. 815864d made packs cut on stored bytes rather than raw
(~20x conservative otherwise). 9e607f7 caught the code up to the spec change
705095b made mid-build. e23acdd fixed two things only production could
show: a listing phase that logged nothing for ~1M objects, and a free-space
floor measuring the wrong filesystem. 5e18c82 is the interesting one — the
first real pack came in under projection because frames were cut at a fixed
8 MiB and split half the listings across frame boundaries, forfeiting the
adjacency that ordering by listing_id exists to create. 1ec6fb8 packed
April in full: 557,065 objects into 32 packs, 8,704x reduction, every member
verified byte-identical, nothing deleted.
Stage 3, the read path — cd9e1a8 made read_html fall back to the pack
index transparently, which is the precondition Stage 4's deletes stand on.
60bc9c6 and 115a001 recorded the measured latency for April and May: warm,
a packed read costs what an object read costs (5.90 ms against 5.79); cold is
36x, and the cost is the sidecar scan rather than decompression.
Stage 4, the only step that removes data — ba1b677 cut the proposed
14-day grace period to zero, because neither argument for it survived being
asked what it protects against, replacing a timer with a gate that gets
satisfied. 257cef0 implemented the deleter with three non-optional checks per
member. 4ec7216 ran it: 100 objects, 100/100 verified, 0 refused, the first
inodes the plan freed at all. 020ba6b confirmed by readback that the deleted
artifacts are still readable — the property the whole plan was built to
establish, demonstrated on production data rather than argued from the format.
Run sheets and prompts — b30c47d, dba8d8f and e564061 built and
corrected the Stage 3/4 run sheet, the last two for failures of the document
rather than the code: a readback step that could not be executed as written,
and a detached run whose output had nowhere to go. d0b2a9d, b52b6b6,
c8d9500 and d1f91e1 tracked the build prompts.
Stage 5 — 2cbb7c3 made the case for a stage that was reading as tidy-up:
run continuously, packing collapses a closed month from ~1.1M objects to about
82, so inodes stop being the binding constraint at all and the ceiling moves
out roughly three years. 4be8ee9 fixed a cap of zero that deleted nothing and
reported success. 5de59bc gave the two pack endpoints the same failure
contract their CLIs have, so raise_for_status() means something. c317c1d
added single-flight locking and a deploy pause, both being a check at a
boundary the jobs already had.
Plan 133 — two read-path defects, recorded rather than fixed
61461c9 opened Plan 133 for
two defects found while verifying Plan 131 Stage 3 in production: object_exists
has no pack fallback, and the sidecar index cache defaults to 4 packs against
months holding 32-41 while the resolver scans upward from pack-00000, so each
scan evicts the entries the next one needs. Neither blocked Stage 4. They were
recorded rather than fixed because Stage 3's gate had just been established over
365 real members with zero failures, and changing the path immediately after
verifying it would have re-opened the gate for no gain.
Plan 135 — Loki's log level, and the alerting gap behind it
cf30421 capped Loki's server log level after its container stdout reached
8.6 GB on the root disk. Investigating that turned up the defect that became
Plan 135: node-exporter bind-mounts
host root at /rootfs but never passes --path.rootfs, so it has never
reported /mnt/data. Both disk alerts only ever evaluated / — leaving the
196 GB data volume, which had filled to 99-100% twice that same week, with no
capacity alerting at all.
What moved between states
Read from docs/PLANS.md's own history across the window — eight index
revisions — rather than from the index as it stands today.
| Plan | Before | After |
|---|---|---|
| 129 | absent | build order |
| 130 | absent | build order |
| 131 | absent | build order |
| 132 | absent | build order |
| 133 | absent | build order |
Five plans entered the index and none left it. That is worth noticing on its own: the week shipped Plan 114's conclusion, Plan 129's production rollout and four of Plan 131's five stages, and the index recorded none of those as a completion. Every state change in the window was an arrival.
What is still owed
- Two plan documents were created in the window with no index row at all.
Plan 134 was split
out of Plan 131's D5 decision in
5de59bc, and Plan 135 was opened incf30421. Neither appears in any state change above, because neither was filed. A plan document in no table is exactly the defect Plan 146 was written to fix. (As of 2026-08-21 both have since been filed — 134 as build-order row 6, 135 in closeout with a 2026-08-23 gate.) - Plan 131 Stage 4's processed-check policy was proposed in
ba1b677and described in that commit as still awaiting agreement. scripts/verify_pack_read_path.pywas written and not yet run atcd9e1a8; it was run two commits later at60bc9c6, inside the window.- The sidecar index cache defect was left unfixed on purpose (
61461c9), sequenced before Plan 132 Stage 2 and not before Plan 131 Stage 4. - Retention is named in
f4e0f0bas no longer theoretical — the disk hit 100% twice during Plan 129's own rollout, and the second time it blocked the plan — and was left needing its own plan. - Two papercuts recorded in
f4e0f0band not addressed:deploy.shandredeploy.share tracked100644and so are not executable on a fresh checkout, andpsis absent from the processing image.
Unattributed commits
Two of 42 non-merge commits name no plan in their subject. Both are recapped above or below; what is missing is the link to a why, not the work.
| Commit | Subject | What is known |
|---|---|---|
a80b123 |
emergency commit to try and free space | No body at all. Its --stat shows 12 files and ~1,300 insertions — shared/compression.py, shared/minio.py, scripts/train_html_dictionary.py, migration V041 — which is the bulk of Plan 129's Stages 1-4 implementation. The following commit 9f078c6 says so explicitly and records the notes for it, but that is another commit's testimony, not this one's, so it stays listed here. |
820c944 |
Make two CLI path assertions platform-independent | Its body attributes it to nothing, deliberately: "Unrelated to Plan 131; it rides along because the same checkout has to be green to trust the rest of this branch." It made two argparse type=Path assertions compare Path to Path instead of str to a POSIX literal, which had the suite green in CI and red on a Windows checkout. |
Merges
Thirteen merges, none recapped as work — the commits they bring in are in the window and are recapped above, so recapping both would double the week.
| Merge | PR | Branch names |
|---|---|---|
788bb33 |
#184 | Plan 114 |
c552c11 |
#185 | Plan 129 |
6dddbcc |
#186 | Plan 129 |
ed6a08c |
#187 | Plan 131 |
4c55520 |
#189 | Plan 131 |
eb99964 |
#190 | Plan 131 |
c4147b3 |
#191 | Plan 131 |
91409b0 |
#192 | Plan 131 |
feeae79 |
#193 | Plan 131 |
4613d0b |
#195 | Plan 131 |
36f1fbd |
#196 | Plan 131 |
196411f |
#197 | Plan 131 |
dd90287 |
#198 | Plan 135 |
Together they bring in 43 commits: the window's 42 non-merge commits, plus
8cab72a from 2026-08-08, which 788bb33 carried in from before the window
and which belongs to the previous week.
Deferred to the next recap
117 commits sit after this window's end, as of 2026-08-21. The tail is more than twice the window, and it is deferred on purpose — the week of 2026-08-17 is not complete work for this recap to describe.
git rev-list --count --all --since='2026-08-16 23:59:59' # 117