Sealed Publication Generations
Sealed Publication Reads describes how a publication is written and read today, and ends with what is not closed. This page is that remainder: the layout that closes it, what each reader must do, and the order the migration has to land in.
Every phase is landed (1, 2, 3, 4 and 5 — phase 3's pin half by measurement, see "Where this stands"). Every section says which phase it belongs to, and that section says exactly what is live. The order the phases have to land in is a property of who publishes and what they pin, which is measured below rather than assumed.
The defect this removes
A publication is replaced in place. publish-bake-bundles.sh deletes _complete, uploads over
the live files, and re-seals — deliberately, because unsealing first is what makes "sealed" mean
"every listed bundle is here".
Interleave two publishers on one prefix and the sealed-skip's answer is stale for whichever loses
the race: both unseal, both upload, and the last to seal writes a sentinel over a directory holding
some of each one's bytes. That is one seal with one generation — self-consistent to every
consumer, and wrong. The read-side generation cannot see it (there is nothing stale to refuse), the
boot seeder cannot see it (the sentinel is present and every listed bundle exists), and the first
symptom is dependency record mismatch — built against mvid:…, live is mvid:… on a portal that
renders nothing.
🚨 The publisher's byte-level postcondition (#3496) turns that from a silent mix into a loud
refusal, and it is not the same thing as preventing one. It re-reads every uploaded file's digest
immediately before the seal and refuses when any is foreign. What it cannot cover is the interval
between that last read and the _complete upload: a publisher overwriting a file inside that single
write still lands under this run's seal. The exposure drops from a whole ~90-second publication to
one file upload — a smaller window, still a window. A postcondition is not mutual exclusion.
🚨 And "give the prefix one owner" is not the answer either — that was measured. Of the overlaps actually observed on 2026-09-06, zero were between the two lanes and four were two runs of the same lane on one identity. A rule about which repository owns a prefix does not address a lane racing itself. The numbers are in Sealed Publication Reads → "How many writers, measured".
The layout
Each publication is written into its own directory, and a one-line pointer says which one applies:
<root>/<identity>/<source>/_current the pointer: one line, a bare directory name
<root>/<identity>/<source>/<generation>/… one publication INSTANCE
<root>/<identity>/<source>/<generation>/_complete its seal, still written strictly last
<root>/<identity>/<source>/<generation>/modules/… its module set, + _index
<generation> is the publisher's publication token — <repository>-<run id>-<attempt>, already
minted by publish-bake-bundles.sh and already restricted to [A-Za-z0-9._-] so it is a legal bare
name on every backend. It is unique per run by construction.
That one property is the whole design:
- Two publishers never write the same bytes. Disjoint directories, so there is nothing to interleave. A mix becomes unrepresentable, not merely detectable.
- Publishing ends with one small write. A reader either sees the previous generation — intact, sealed, still on the shelf — or the new one.
- A reader that resolved early is not torn. Its generation is still there; the pointer moving does not delete it.
No pointer means the flat layout
A source directory with no _current is its own publication directory. That is exactly
today's behaviour, it stays legal for as long as any reader needs it, and it is what makes the
migration below possible at all. Resolution is opt-in by the writer, never by the reader.
Why a pointer and not an atomic directory rename — measured, 2026-09-08
The obvious design is to stage the publication and then rename it over the live one. It is not available on this store through this client, and that is a measurement rather than a recollection.
Read off azure-cli 2.90.0 — the same version the publisher's read-back query was measured
against:
| group | commands |
|---|---|
az storage file |
copy · hard-link · metadata · symbolic-link · delete · delete-batch · download · download-batch · exists · generate-sas · list · resize · show · update · upload · upload-batch · url |
az storage directory |
create · delete · exists · list · show |
There is no rename in either, directory delete is documented "Delete the specified empty
directory", and there is no lease command under either group — so a lease per identity is not
reachable from this lane either. And even in the REST API, where Rename Directory has existed
since API version 2021-04-10, a rename cannot replace an existing directory: the swap would be
rename-away plus rename-in, two operations with a gap in which the live path does not exist at all.
🚨 So the smallest write this store actually offers is one small FILE, and that is what the pointer is. A design that assumed an atomic directory rename would have reintroduced a partial-visibility window wearing a better story.
What is still a window, stated honestly
The pointer is one small file, and writing it is not atomic on every backend: az storage file upload is a create-then-put-range, so a reader can catch it empty or short.
That does not produce a mix. An unreadable, empty, escaping or dangling pointer resolves to the source directory — see the resolution rules below — so the worst case is:
| phase | a torn pointer read gives | what the reader does |
|---|---|---|
| flat copy still present (phases 1–4; a prefix nothing has published since) | the previous publication, whole | serves it; correct, just not the newest |
| flat copy gone (since phase 5) | a directory with no _complete |
"being republished right now" → 503 + Retry-After, which every consumer already waits out — and, for a reader that DECIDES on the reading rather than serving it, "cannot tell": see the reader contract phase 5 changes |
So the trade is: ~90 seconds in which a mix can be sealed becomes the duration of one small
file write in which a reader may be told to come back. The failure mode changes, not only its
size — from "a sealed mix nobody can detect" to "read it again". Where the backend offers an atomic
rename (Azure Files' Rename File, present in the REST API since 2021-04-10 though not exposed by
every az storage file build), the writer phase should publish the pointer that way and remove even
that.
The reader contract
One function resolves the pointer, and every path is composed under its result:
ShippedPrebuiltBundles.PublicationDirectoryOf(sourceDirectory, logger)
🚨 The one way to get this wrong is silent, so the API is shaped against it. A reader that
resolves the pointer for the listing and then composes its file paths under the source
directory serves the flat publication's bytes under the generation's token — the very mix the
generation exists to prevent, wearing a token that says it is not. During the migration both paths
exist, so it is a wrong answer rather than a missing one. That is why SealedPublicationOf and
SealedModulesOf hand the resolved directory back on the reading (Directory) instead of
leaving each caller to resolve a second time.
Resolution rules. It never throws; it falls back. Each of these resolves to the source directory:
| pointer | why it falls back |
|---|---|
| absent | the flat layout — the normal state today |
| empty or blank | being replaced right now; the previous generation still applies |
unreadable (IOException) |
same, mid-write |
., .., anything with a separator, anything rooted |
🚨 a pointer is a NAME. It must never be able to address bytes outside its own source directory — refused and logged as a warning, because a publisher wrote something this reader will not follow |
| names a directory that is not on disk | a retention sweep outran the pointer; logged as a warning |
Generation retention
Follow Released Artifact Retention (#3842). The 30-day age window and release/consumer references supersede the earlier 24-hour previous-generation grace proposal.
- Preserve the generation
_currentadvertises and every generation still consumed, including a same-major adopted fallback. A pointer swap does not prove that all readers or portals have finished with the old generation. - Unreferenced continuous generations become eligible after 30 days. Supported official releases and their complete artifact closure survive throughout support.
- Publication must establish protection before moving
_current. Generation cleanup must participate in the same publication/consumer ordering as other artifact cleanup; running an independent purge after a pointer swap is insufficient. - Missing consumer or publication inventory prevents deletion. A retry/backoff duration is not an adoption lifetime and cannot authorize collecting a serving fallback.
Where it runs. PrebuiltBundleStore — the portal's own sweep, one level down from the identity
rules it already applies, sharing that pass's abort discipline, its report-only mode and its ledger.
It is deliberately NOT in the publisher: the publisher has no consumer inventory, no release
markers, no adoption stamps, and it holds none of them at the moment it publishes.
The rule, ORed as KEEPs, applied only inside an identity the pass RETAINS. A generation is collected when all of these hold, and kept when any one fails:
| keep when | |
|---|---|
| the pointer | _current names it — it is the live publication, however old |
| the pointer's own health | _current could not be resolved to a directory on disk: unreadable, blank, refused, or dangling. Then every generation of that source is kept |
| age | its newest write is inside PreWarm:PrebuiltBundleRetention:MinimumAge (at least 30 days) |
There is deliberately no "it could not be read" rule, and it is not missing: a generation the
sweep's walk could not enter contributes no files, so it is never identified as a publication and
never becomes a candidate. Unreadable still means kept — it arrives by not being enumerated rather
than by a branch, which is the stronger of the two. Nor is there a rule on whether the generation was
sealed: a superseded one is collected on age whether it was sealed, torn, or abandoned mid-upload.
The entry records which (HasSentinel) so an operator can tell a wave of abandoned publications —
a different problem — from ordinary supersession, and that field decides nothing.
One walk per identity, not one per generation. The bytes, the newest write and the marker files
all come from the single recursive enumeration the identity-level sweep already performs. That is
not a micro-optimisation: the root is an Azure Files share where every enumeration is a network round
trip, and per-generation walks would have made the cost grow with the number of generations — the
exact quantity this cleanup exists to bound. It is why HasSentinel reports the sentinel's
presence rather than verifying its listing the way the source-level IsSealed does; verifying
would cost one file read per generation per pass to answer a question no rule asks.
A generation under a collectable identity is not listed separately: that identity's directory goes whole, and its byte count already includes them.
🚨 Why this is sound without a per-generation consumer inventory — the one thing that could make
it unsound. A non-current generation is unreachable, not merely unfashionable. Every read of a
published source directory composes under
ShippedPrebuiltBundles.PublicationDirectoryOf(sourceDirectory) — the boot seeder, the same-major
adopted fallback, SealedPublicationIndex, PublishedBundleCatalogue and ServedModuleBytes alike
— and nothing anywhere enumerates a source's subdirectories looking for a publication. So the set a
consumer inventory would have to protect is exactly {the generation _current names} ∪
, and both are kept above by construction. The sweep therefore uses the
readers' own resolution (ShippedPrebuiltBundles.ResolvePublicationPointer, which is
PublicationDirectoryOf with the fallback reason kept rather than discarded) instead of a second
copy of those rules: a sweep that decided reachability differently from the readers would delete
bytes a portal was still resolving, and the two would drift silently.
🚨 "Publication must establish protection before moving _current" is satisfied by AGE, not by a
lease. A generation is protected from the instant its first byte lands, because it is younger than
the window. A publication in flight can therefore never be collected, and the publisher needs no
claim, no lock and no ordering with the sweep — which is what makes this implementable at all across
two repositories that share no lock. UnsealedGrace is deliberately not consulted here: it is
hours where the window is at least 30 days, so a branch for it could never fire, and a check that
cannot fail is not a check.
🚨 A directory is a generation only on POSITIVE evidence that it is a publication — it carries
source-commit.txt, repository.txt or the completion sentinel, or the pointer names it. It is not
"every subdirectory except the ones I know about". The publisher's modules/ sits beside the
generations in the flat compatibility copy, and under an exclusion list every bookkeeping directory
added later would become collectable the day it was added, silently. Under positive identification
it is simply retained, and the worst case is bytes that stay.
Who actually publishes — measured, 2026-09-07
The migration order is decided by this table, so it is a measurement rather than a recollection.
Every row was read off the producing repository's own ci.yml (or main-cd.yml) on 2026-09-07.
| producer | prefix (bake-source) |
which publish-bake-bundles.sh it runs (platform-ref) |
behind core main |
|---|---|---|---|
core CD plugins-bake (main-cd.yml) |
plugins |
this run's own core commit (needs.gate.outputs.sha) |
0 — it IS the tip |
MeshWeaver.Plugins ci.yml |
plugins |
aa40758329216d57dcefc2d8e8a52101efd5f225 |
231 commits |
| MeshWeaver.Reinsurance | reinsurance |
1b5350d547473a5e2ca81e793e774cc962acfeb3 |
284 commits |
| MeshWeaver.SocialMedia | socialmedia |
1b5350d5… |
284 commits |
| MeshWeaver.Manufacturing | manufacturing |
1b5350d5… |
284 commits |
| MeshWeaver.Education | — | — (it calls no node-repo-publish-bake) |
— |
Three things follow, and each of them changes the plan:
- 🚨
pluginsis the ONLY prefix with two producers. Every other prefix has exactly one, and Education publishes no bake at all. The coordination this migration needs is therefore one pair — core CD and MeshWeaver.Plugins — not five repositories. Everything else flips on its own schedule, one repository at a time, with nobody to coordinate with. - 🚨 Core CD has no pin to move. It checks the platform out at its own gate sha, so the day the
writer merges, core CD runs it. If the writer were unconditional, the
pluginsprefix would become a new-writer/old-writer pair that same day, against a MeshWeaver.Plugins 231 commits behind — exactly the half-migration this page exists to prevent. That is what makes the selector below mandatory rather than tidy. - Nothing is near the tip. The nearest producing pin is 231 commits back and none of the four carries phase 1. A plan that assumes a pin will "have moved by then" is assuming something that has not happened in a month.
🚨 The correction: "past phase 1" is not a satisfiable precondition
This page used to say phase 2 was "every producing repo's publish-bake pin moves past phase 1".
Measured against what phase 1 actually changed (a4109d422), that instruction is a no-op: it
touched src/ — the portal image's readers — plus documentation and one comment block in the
publish script. It changed nothing a pinned lane executes. A producer whose platform-ref moves past
it runs byte-identical behaviour, so the condition can be satisfied by the whole fleet without
bringing the migration one step closer.
What a producer must be past is the writer commit itself — and a writer that is on by default cannot be got past, because it takes effect the moment a pin reaches it. Hence the selector, and hence the order below.
The migration, in order
Phase 1 — readers tolerate the pointer (landed, a4109d422)
PublicationDirectoryOf plus every read routed through it: the boot seeder,
PublishedBundleCatalogue, ServedModuleBytes, and the registry's four prebuilt routes. Behaviour
on a share with no pointer is unchanged, byte for byte.
Nothing writes a pointer yet, so this changes nothing observable — which is why it ships with a suite that builds the generation layout by hand and asserts the readers serve it, including the arm that catches the compose-under-the-source-directory mistake.
Phase 2 — the writer LEARNS the layout, selected per caller (landed)
🚨 Historical, as this phase shipped: the selector defaulted to flat. Phase 4 moved that
default to generation; everything in this section describes the mechanism, not today's default.
publish-bake-bundles.sh gains generation publishing behind an explicit selector: a
publication-layout input on node-repo-publish-bake.yml, carried into the script as an environment
variable, valued flat (the default) or generation. bake-scope.sh and carry-forward-bundles.sh
resolve the pointer in the SAME commit, because node-repo-publish-bake.yml fetches all three at one
platform-ref and no pin can carry half of them.
At flat the script must behave byte-identically to today, and that is provable rather than
asserted: test-publish-bake-overlap.py runs the real script, and its three control cases (a settled
publish, a republish of new content, an already-published skip) are the regression suite for the flat
path. The new mode earns its own cases — two interleaved publishers each seal their OWN directory,
neither directory holds a byte of the other, and _current names exactly one of them.
At generation the writer uploads into <source>/<publication token>/, verifies there (the #3496
postcondition still applies, now over a directory nobody else writes), seals it, also writes the
flat copy so a portal image that predates phase 1 keeps working, and moves _current last.
(Historical, as this phase shipped: since phase 5 the writer DISPOSES of the flat copy instead of
writing it, once the pointer has moved and been read back.)
Every read that decides what is already published — the architecture marker, the sentinel, the
source-commit marker, the module index — resolves the pointer first and reads inside the resolved
directory, or the writer and the readers disagree about which publication is live. The resolution
rules are the reader's, unchanged: an absent, blank, unreadable, escaping or dangling pointer means
the source directory. carry-forward-bundles.sh must read the generation the listing came from,
which is the shell analogue of the reader's If-Match.
🚨 Any new marker file is LISTED and UPLOADED before architecture.txt. That file is the LAST
upload before the postcondition, and the overlap harness hooks its second publisher onto it — a
marker written after it silently stops the harness detecting overlaps while every case still reads
green. (repository.txt was added under this rule and says so in place.)
What actually shipped, and the one deliberate deviation
- The selector is
publication-layoutonnode-repo-publish-bake.yml, carried into the script asBAKE_PUBLICATION_LAYOUT. It defaulted toflatwhen this phase shipped and defaults togenerationsince phase 4. An unrecognised value is refused, never silently read as flat: the value decides where a publication is written and which directory every reader resolves. bake-scope.shandcarry-forward-bundles.shresolve the pointer in the SAME commit, by the same rules, so the writer and the two readers of the publish lane cannot disagree about which publication is live.carry-forward-bundles.shresolves it itself rather than being handed the directory — its caller may be pinned to a workflow copy that knows nothing about generations, and the one-publication postcondition it already carries is what pins it to a single generation if the pointer moves between the listing and the downloads.- 🚨 THREE Azure-direct readers composed their paths under the bare PREFIX — not two, which is
what this page said for a week.
compose-sealed-modules.sh(the module-set index and each module, on the OIDC fallback path) andnode-repo-gate.yml's inlinedownload-batchwere routed in phase 3. The third, found on 2026-09-14 after phase 4 had landed, ischeck-release-availability.sh— the upstream gatemain-cd,release.ymland every node repo'spublish-bakerun — and it is the one that was already producing a false red. See "The third reader, and the false hold it was already producing" below. None of the three is a reader the portal image carries, so none was covered by phase 1. - 🚨 The pointer moves BEFORE the flat compatibility copy, not after it. "Last" in this page is
about the generation: a reader must never be pointed at a directory still being filled in, and
moving the pointer straight after the seal satisfies that exactly. The flat copy is a different
audience — readers that cannot follow a pointer at all — and it is the one part of a generation
publication another publisher can still be writing. Ordering it after the pointer means an overlap
on the flat copy costs the compatibility copy (refused, as today, and the run goes red) instead
of costing a publication that is already whole, sealed and disjoint. Ordering it before would let
a race on the OLD layout withhold a publication that is correct on the NEW one — which would make
flipping a prefix deliver nothing at all until the flat copy is dropped. The same ordering
argument carries phase 5's DISPOSAL, which replaced that write: the flat seal goes only after
_currenthas moved and been read back, so at no instant is a reader left with neither. - ✅ Retention landed — see "Generation retention" above. It was the precondition on flipping:
generations accumulate at ~45 small files each, and nothing the writer does deletes anything it
did not create. The collector is the portal's own
PrebuiltBundleStoresweep, one level down from the identity rules it already applies.
Phase 3 — the remaining readers resolve the pointer (landed), and every producer's pin reaches phase 2 (open)
Two halves, and the code half is done.
The two Azure-direct readers now resolve the pointer. compose-sealed-modules.sh (the
--storage-target OIDC fallback) and node-repo-gate.yml's seed step download-batch composed
their paths under the bare prebuilt-bundles/<identity>/<source>/ prefix, because phase 1 routed
the readers the PORTAL IMAGE carries and neither of these is one. Both now call the same resolver
by the same rules.
compose-sealed-modules.shalso records the generation its module index came from and pins each module download to it — the storage-path equivalent of the registry path'sIf-Match, and a property that lane never had: a pointer that moves between the listing and a download can no longer mix two publications' module bytes, because a generation directory is not rewritten in place.- 🚨 The gate's
seedis the one reader that may NOT take the reader's fall-back, and it is because of the BATCH.download-batchrecurses and the CLI flattens what it finds. For a point read, "fall back to the source directory" yields the previous publication whole; for a recursive batch over a prefix holding generations it yields a union — and same-named files overwrite each other, so the mix is invisible to the step's own count as well as to the compiler. So a pointer that EXISTS and could not be followed is a refusal there, not a fall-back. It is transient by construction (the pointer is one small file write) and a re-run reads what now applies.
🚨 Neither Azure path had ever been executed by anything. test-sealed-module-compose.py runs
the script with --registry-url only, and its gate case runs seed with TARGETS="" — so the
whole download-batch block was dead to every harness in the repository.
test-publication-pointer-readers.py executes both, against a share carrying a flat copy and a
generation whose files share names and differ in bytes, so the verdict is which publication
landed. Measured on the pre-change readers: 5 of its 8 cases fail, the three that pass being
the no-pointer controls that pin today's flat behaviour as byte-identical. bake-scope.sh --self-test — one of the two proofs this page leans on for phase 2's reader half, and until now
run by no workflow at all — is wired into the same lane.
🚨 And #4172 made this load-bearing rather than tidy. A downstream publication now seals only
its OWN modules, so an upstream's module bytes are reachable through the upstream's own seal and
nowhere else. There is no downstream copy left to fall back on, which makes
compose-sealed-modules.sh the single path to them.
The pin half is still open. A producer past phase 2 can write generations and is still writing flat. Core CD needs no pin move; the satellites move theirs the way they always do.
Phase 4 — flip (landed: core's half of plugins, then the lane default for every prefix)
Set publication-layout: generation on every producer of one prefix. The multi-producer prefix is
plugins and nothing else: core CD's plugins-bake and MeshWeaver.Plugins' publish-bake. Every
other prefix has exactly one producer.
What is landed, and the precondition that was measured before it was
main-cd.yml's plugins-bake passes publication-layout: generation. From that merge, core's
publications of prebuilt-bundles/<identity>/plugins are written into their own run-named
directory, sealed there, and pointed at.
🚨 And then the LANE DEFAULT moved, which is what flipped the other five. The five
single-producer prefixes — crm, education, reinsurance, socialmedia, manufacturing — were
described here as "one PR each, in its own repository". They are not, and treating them that way
was leaving the more important half undone: the thing that decided their layout was the
publication-layout default, and while that default was flat, a node repo that said nothing
published in place — including every node repo that will ever be created. The default is
generation from this change; none of the six callers passes the input, so all six prefixes are
flipped by it and no satellite PR is involved. What a per-repo PR would still buy is the same thing
the MeshWeaver.Plugins caller buys: a declared layout rather than an inherited one.
The script's own ${BAKE_PUBLICATION_LAYOUT:-flat} fallback deliberately stays flat. It is the
direct-invocation default — the overlap harness, a manual run — and the lane always passes the
input through explicitly, so no lane run ever reaches it. Reading that line as "the fleet publishes
flat" is the mistake it used to invite, and the comment above it now says so.
🚨 The prefix-ownership mechanism lives in the PUBLISHER, so "either order" is a claim about
which publisher each producer is RUNNING — a producer resolving a publish-bake-bundles.sh that
predates #4249 still refreshes the flat copy
in place and never moves the pointer. That is the one thing that had to be true before flipping and
it is checkable rather than assumed. Measured 2026-09-14 across all six node repos — Plugins,
Crm, Education, Reinsurance, SocialMedia, Manufacturing — every uses: of a node-repo-*.yml lane
is @main and every scripts-ref: is main. No pinned publisher and no pinned reader of this
prefix exists in the fleet. The exception is core's own plugins-bake, which passes no
scripts-ref and so resolves its scripts at platform-ref = the commit the run publishes; on a
reconcile run that commit can be older than #4249, and such a run refreshes the flat copy without
moving the pointer — which leaves pointer-following readers on the newer generation, the safe
direction, and is the same flat-copy residual phase 5 removes.
What the satellite's own flip buys is therefore explicitness, not correctness: it replaces a
discovered layout (announced by a ::warning:: on every publish) with a declared one. The same is
true of a per-repo PR on any of the five single-producer prefixes now that the default has moved.
The third reader, and the false hold it was already producing
🚨 Phase 4 did not only expose a phase-5 problem; for one reader it created a live false red, and
it had been live for plugins since #4269 before this page noticed.
check-release-availability.sh answers "is this upstream published for the identity I am about to
build against?" by probing one file: prebuilt-bundles/<identity>/<source>/_complete. That is the
prefix's sentinel — the flat compatibility copy's — and under the generation layout the two
answers come apart on every publish:
publish_publication seals the generation |
the publication is complete |
… moves _current |
the publication is live to every pointer-following reader |
… then refreshes the flat copy, which publish_one_target begins by deleting its _complete |
the prefix has no sentinel for the whole upload-and-verify interval |
So for ~90 seconds per target per publish, a gate probing the prefix answers false and reports
no sealed publication under … — the one message that means an upstream has not published yet,
for a publication that is sealed, live and pointed at. That message is not a neutral one: it is the
wording that held MeshWeaver.Reinsurance 23 times in 24 hours (#3583), so a false instance of it
costs a reader the same investigation as a true one.
Fixed by routing it through the same resolution as its three siblings, with the same fall-back:
an absent, blank, unreadable, escaping or dangling pointer means the source directory, which is
exactly this gate's previous behaviour — so the change can only ever add an answer it used to get
wrong. test-publication-pointer-readers.py now executes three readers, and the fixture for this
one is the window itself: the generation sealed, _current naming it, and the prefix's own sentinel
removed. Against the pre-fix script that case fails with the false-hold message; its two siblings
(nothing sealed anywhere ⇒ still refuses; a flat prefix with no pointer ⇒ still sealed) pass on both,
which is what makes them controls rather than new behaviour.
🚨 THREE places print "no sealed publication", and their remedies are OPPOSITE
This is the half that matters more than the fix, because conflating these makes a genuine outage look transient — and the sentences differ by a preposition.
| the message | who prints it | what it means | does re-running help? |
|---|---|---|---|
no sealed publication **under** <path on the share> |
check-release-availability.sh |
since the fix above: a REAL absence. The probe follows _current, so the refresh window can no longer produce it — and since phase 5 a pointer that EXISTS and cannot be followed is reported as CANNOT DETERMINE instead, so this sentence keeps meaning exactly one thing. Before the fix it could be either, and the output did not distinguish them |
no (post-fix). A run from before the fix may have been the window |
no SEALED publication **at** <registry URL> … (404) |
node-repo-gate.yml's upstream_not_ready, from the registry probe — a definite 404, refused immediately on purpose: 408, 429, any 5xx and 000 are retried as transient, and everything else is a decision the registry has already made |
the upstream genuinely has not published for that framework identity | no. Look at whether the upstream's own seal is blocked |
no SEALED publication **under** <account>/<share>/<dir> |
compose-sealed-modules.sh |
the resolved publication carries no sentinel — pointer-resolved since phase 3 | no; not this window |
🚨 "Re-running helps" was never a property of the message — it was a property of the WINDOW, and
the window is what the fix removed. A sentinel probe answers false for a real absence and for a
refresh in flight identically, which is why the honest table says no everywhere now and leaves the
diagnosis to the identity, below. An earlier draft of this table said yes for the first row; that
would have told an operator to re-run a job that cannot pass. (Copilot's review.)
A live instance of the middle row on 2026-09-14: MeshWeaver.Education main asking for identity
s3b3fb865f… (platform set 8596) while the last MeshWeaver.Plugins publication was s6b704cd02…
(set 8581, 12:53Z), its seal blocked on a broken suite. Reading that as the share window would send
a reader to re-run a job that cannot pass, which is strictly worse than not diagnosing it at all.
🚨 The identity in the message is the tell. A share-window red names an identity the fleet is currently publishing for; a registry 404 names one nothing has published for yet. Read the identity before the verb.
🚨 The residual phase 4 does NOT remove: a flat publication beside a generation one
(Phase 5 removed the refresh this section is about; what survives it is point 2 of "The residues, named" below — a flat publication on a prefix with no resolvable pointer, which the same guard keeps at zero.)
Raised by the review of the default move, and real. A generation publication writes its generation,
moves _current, and refreshes the flat compatibility copy last. A FLAT publication of the same
prefix writes only the flat directory and never touches the pointer. So if a flat run's whole
publication lands after a generation run's seal, there is no overlap for the byte-level
postcondition to refuse, and flat readers end on the flat run's bytes while pointer-following readers
stay on the generation. That is the half-migration in its last surviving form.
What keeps it at zero is that the flat arm has no reachable writer, and that is enforced rather than observed:
| a run takes the flat arm when | reachable? |
|---|---|
its caller passes publication-layout: flat |
NodeRepoLaneHostGuard.NoCallerInThisRepository_PublishesFlat refuses one in this repository, and no satellite passes the input at all (measured 2026-09-14). A satellite that wanted it would have to write the line in its own ci.yml |
it resolves a live _current while asking for flat |
not the flat arm — #4249 promotes it to a generation and names the caller |
its publish-bake-bundles.sh predates #4249 |
yes — core's plugins-bake on a reconcile at an old platform-ref. No code added to today's script can help this one: it is not running it. Such a run refreshes the flat copy and does not move the pointer, so pointer-followers stay on the newer generation — the safe direction |
A check on today's flat arm (re-resolve _current before the final seal, refuse if one appeared)
would therefore only ever fire for a population the guard above already keeps empty, which is why
there is no such check. Phase 5 removes the flat copy and the residual with it.
🚨 Every reader fails safe to today's behaviour. ResolvePublicationPointer degrades an absent,
blank, unreadable, escaping or dangling pointer to the source directory, where the flat copy still
sits, still sealed — so the worst case of a pointer that cannot be followed is the pre-flip serve.
The single reader that may not take that fallback is node-repo-gate.yml's seed, because
download-batch recurses and flattens; it refuses instead, transiently, and a re-run reads the
publication that now applies.
🚨 "In one change set" was stated here as the precondition, and it is not available. The two
producers of plugins live in two repositories, which cannot merge atomically, and one of them
publishes on every core build. So the flip must pass through a prefix where one producer writes
generations and the other still writes flat — the question is only for how long. Requiring an atomic
change set does not avoid that state; it just leaves it unhandled.
The layout belongs to the PREFIX, not to the caller (landed)
The publication-layout input's own description used to tell operators what happens in a mixed
state and then tell them not to create one: "the new one moves the pointer, the old one replaces the
flat copy in place and never touches it, so a pointer-following reader keeps serving its generation
and never sees the old writer's NEWER publication — a stale serve with nothing red anywhere." That
instruction was unenforceable, because the state is unavoidable.
A live _current is now what decides the layout. resolve_publication_dir already runs on every
run whatever the caller asked for; when it resolves a generation, the run publishes a generation
— announced as a ::warning:: naming the caller to flip. Two consequences, and the second is the
one worth stating:
- the half-migration is unreachable rather than forbidden. Nothing can write the prefix in a way
that orphans a newer generation, so
pluginsmay be flipped one producer at a time, in whichever order the two repositories merge; - every decision the run makes — already published? which architecture? which source commit? does it carry a module set? — is still read from the generation the pointer names, i.e. from the publication consumers are actually being served. Nothing about that changes.
🚨 A flat caller never takes the prefix back, and that is deliberate. Deleting or re-pointing
_current would invert a migration another producer already made, and it can move consumers
backwards: the pointer is moved before the flat compatibility copy is refreshed (the ordering
argued above), so a run that finds an older flat copy could skip, retire the newer generation, and
expose the older bytes. Going forward is always safe; going back is not. So a flat caller whose
content is already the live publication skips and changes nothing — asserted as its own case.
Setting publication-layout: generation on both producers is still what you should do: it makes the
prefix's layout explicit rather than discovered.
🚨 The pointer only moves FORWARD, and this is the one thing the generation layout could get silently wrong that flat could not
Under flat, two overlapping publications write ONE directory, so the byte-level postcondition
finds the other run's bytes and refuses. Under generation they write disjoint directories:
there is no mix to refuse, and whichever run finishes last moves _current. When that is the
run carrying older content, every reader is handed a publication that is complete, sealed,
self-consistent and out of date — with nothing red anywhere. That is the failure this whole layout
exists to prevent, reached from the other side.
The decision-time never-seal-backwards guard cannot see it. It reads the publication that was live
when publish_to_target resolved LIVE, roughly 90 seconds before the pointer is written; a
sibling that sealed in between is invisible to it. So the question is asked again, in
pointer_moved_past_us, immediately before move_pointer:
- re-resolve
_currentby the readers' own rules; - if it now names a different generation, read that generation's
source-commit.txtand order it against this bake'sSOURCE_SHAthrough the same compare API; ahead(newer, and containing ours) ⇒ do not move the pointer, and do not refresh the flat copy either — writing this run's older bytes there would do to pre-pointer readers exactly what the pointer refusal just declined to do to pointer-following ones. The run still succeeds: its content is contained in what is live, so there is nothing to report as a failure. Its generation stays on the share, sealed and named by nothing, which is precisely the state retention is defined over. Counted astargets-superseded=N, printed on every run including zero.
🚨 This is a postcondition, not mutual exclusion — the same thing verify_publication says of
itself, and for the same reason: there is no lease command under either storage group. What it buys
is that the window shrinks from a whole publication to the gap between that read and a one-line
upload, and that losing that race is no worse than flat's behaviour today. An unorderable pair
keeps today's behaviour and says so, exactly as the decision-time guard does; refusing on an
unanswerable comparison would mean a run whose content cannot be ordered never becomes live at all.
test-publish-bake-overlap.py executes it: the newer content publishes and points while the older
run is mid-upload, and the older run must leave the pointer alone, say so naming both commits, count
it, and leave the flat copy carrying the newer bytes. Against the publisher that lacks the check,
4 of those assertions fail and the pointer ends on the older publication.
🚨 The residual window is a pin bump inside one lane. Even a single-producer prefix has a moment where a run started before the flip is still in flight while a run after it writes a generation. The loser leaves the pointer naming a stale generation until that lane publishes again — which happens on its next merge, so it is self-healing and bounded by one publication rather than permanent. Worth knowing before reading such a serve as a defect.
Phase 5 — dispose of the flat copy (landed, 2026-09-17)
From here the mix is unrepresentable and the republish window is gone — for every reader, not only the pointer-following ones. What remains is the sub-second pointer write described above, and an atomic rename would remove even that.
The precondition, measured on 2026-09-17 — the census, with its denominator
| reading | |
|---|---|
build |
Ops/Status/build → cr.meshweaver.cloud/memex-portal-ai:3.0.0-ci.8411. The commit read the 09-14 census owed: main-cd run number 8411 is c84c6c055, and a4109d422 (phase 1) is an ancestor. Past phase 1 on a COMMIT now, not on identity equality. (The mapping is the run number the image tag carries: run 8798 = 43915af5c, the roll memex was on the same day, confirms it.) |
memex (memex.systemorph.com) |
mid-roll: replicas on 3.0.0-ci.8710 = afde4eabe and 3.0.0-ci.8798 = 43915af5c. Both past phase 1. |
memex-cloud (memex.meshweaver.cloud) |
3.0.0-ci.8411 = c84c6c055 on all four replicas. Past phase 1. |
pearl |
now RUNNING (it was Provisioning on 09-14): 3.0.0-ci.8080 = 67cbbe0ee, which contains phase 1. |
partnerre |
status: Planned, no estate: the record deliberately carries no host, cluster or pinnedImageTag until its first infra deploy, and its data lives in PartnerRe's own subscription. It reads nothing on this share. |
🚨 State the denominator: those are the FIVE Hosting/Deployment records on the control instance, and the records are not provably the whole population — an install that self-updates from the registry and has stopped doing so appears in none of them. The instrument that would name a running image per replica is Sample, and on this cluster it is blind (#4218).
Why an install that is NOT in the records still cannot be reached — and it is a mechanism, not an inference
A publication is written under the framework identity the bake resolved, i.e. the current platform's. The question is therefore whether a pre-phase-1 image can resolve the same identity as a current one, because only then would a phase-5 publisher write where such a reader looks.
It cannot, and Framework Identity Churn is why: Directory.Build.props deliberately lets the SDK append the commit to AssemblyInformationalVersion, so the identity moves on every core commit — measured at ≥ 18 distinct identities across 43 merges, of which only 5 touched the full-MVID set by design. The churn that page treats as a cost is, for this question, a guarantee: an image predating a4109d422 carries a different commit, therefore different assembly bytes, therefore a different identity, and nothing publishes under it. Its flat publication stays on the share, untouched by any phase-5 writer, until retention collects it — and retention already treats a registered instance's own reports as references.
So phase 5 cannot dark-serve an old install by writing somewhere it does not look. What it can do is the next section.
🚨 Phase 5 is NOT "stop writing the flat copy" — the copy already on the share is the hazard
This is the part the phase's one-line description hides, and it inverts the failure the design table assumes.
The table above says a torn pointer read with the flat copy gone finds a directory with no _complete, i.e. "being republished right now" → 503 + Retry-After, which every consumer already waits out. That is correct only if the copy is actually gone. A phase 5 that merely stops refreshing it leaves the last flat publication sitting at the prefix, sealed and complete, while _current keeps moving past it. From then on a torn pointer read resolves to a publication frozen at the day phase 5 landed — self-consistent, sealed, and older every hour. That is a stale serve with nothing red anywhere: this issue's own failure mode, reached from the third side, and permanent rather than bounded by one publication.
So phase 5 disposes of the existing copy, and the disposal is the publisher's own, incremental and per prefix: on a generation publish, instead of refreshing the flat copy, dispose_flat_copy deletes the flat _complete first and then its files. Removing the sentinel is what turns the prefix from "a complete older publication" into "being republished", which is the state every reader already handles and the one the design table assumes. It needs no bulk sweep of the share, it happens once per prefix on that prefix's next publication, and a prefix that is never published again keeps its flat copy — which is exactly right, because it is the one an old identity's reader still needs.
What landed, in the order the code runs it
| step | and why it is in that place |
|---|---|
| the generation is sealed | unchanged: the #3496 postcondition runs there and refuses, exactly as at phase 4 |
_current is moved |
unchanged |
_current is READ BACK (pointer_is_live) and must resolve to THIS RUN'S sealed generation |
🚨 never the upload's exit code. This share has reported SUCCESS for files it did not store (39 of 45, 2026-09-08) — and in exactly that case the PREVIOUS pointer survives, so a read-back that asked only "is SOME generation live" passes, counts the run as published, and disposes of the flat copy for a publication nobody points at (raised by the review of the phase-5 PR; reproduced as a negative control). The comparison is against $dest/$PUBLICATION |
| anything else live ⇒ which is established, never guessed | If _current now names a publication whose content is NEWER (pointer_moved_past_us, the same compare API the pre-pointer check uses), a sibling won the pointer in that gap: nothing is wrong, that run owns the prefix and its own disposal, and this one is superseded — green, and it disposes of nothing. Anything else is our own write not landing: RED, and nothing is deleted |
the flat _complete is deleted, alone |
the one write that turns a complete older publication into "being republished". If it fails, nothing else is touched and the target fails: a sealed flat copy that is no longer refreshed IS the frozen serve this phase removes |
the flat files are deleted (publish-bake-files.py dispose) |
only files POSITIVELY identified as the flat publication — *.zip, modules/*.module.nupkg, modules/_index, the markers — matched per directory so a pattern can never reach into a generation. An unrecognised file is LEFT and named; the worst case of a narrow pattern is bytes that stay. A failure here is a ::warning::, not fatal: the prefix is already unsealed, so a left-over is storage, not a publication, and the next publication retries it |
the flat _complete is read again, and removed again if it is back |
🚨 THE ONE INTERLEAVING THE ORDER CANNOT PREVENT, and the review of the phase-5 PR found it: a producer still running a pre-phase-5 publisher refreshes the flat copy — unseal, upload, VERIFY, seal LAST — so its seal can land after this sweep's deletes and leave the prefix SEALED over an emptied set, which a torn pointer read would be served. There is no lease on this store (no lease command under either az storage group), so this is a POSTCONDITION, not mutual exclusion: re-reading the sentinel and removing it turns that state into "being republished", which every reader already backs off from. A failure to remove it is the one state phase 5 must not leave behind — fatal, and named |
🚨 A SUPERSEDED run disposes of nothing. pointer_moved_past_us returns before the pointer moves, so such a run never reaches the disposal — and it must not: the newer run it lost to owns that prefix, and may be a producer still refreshing the flat copy (a reconcile at an older platform-ref) whose seal this run would otherwise tear out from under it.
"A reader is never left with neither" is read off the DELETES, not off the code's line order. test-publish-bake-overlap.py's fake share records, before every delete, what _current names, whether that generation is sealed, and whether the flat copy still is — so every line of that log must satisfy (generation sealed) OR (flat sealed), the first line must be the sentinel, and every line after it must show the flat copy already unsealed.
🚨 The reader contract phase 5 changes: a FAULTED pointer is no longer the flat copy
Every reader resolves _current and falls back to the source directory when it cannot follow it
(absent, blank, unreadable, refused, dangling). Those five used to be one answer because the
fall-back landed on the flat copy — a sealed publication. They are now two:
| the pointer | what the fall-back finds | what a reader must do |
|---|---|---|
| absent | the source directory IS the publication (the flat layout; a prefix nothing has published for yet) | read it, exactly as before |
| exists and could not be followed | nothing sealed — the copy was disposed of | 🚨 "cannot tell", never "nothing sealed" |
The readers whose "no sentinel" answer already backs off need no change and got none: the boot
seeder compiles instead, the registry's prebuilt routes answer 503 + Retry-After,
compose-sealed-modules.sh and the gate's seed refuse. The three whose answer was PERMISSIVE were
fixed with this phase:
SealedPublicationIndex.ReadSourcereported the source unsealed AND unattributable (there are no markers to read either), andSealedSyncGateanswers an unattributable source withGo— for every repository, since a source nobody could attribute may be any of them. A faulted pointer with nothing sealed behind it is nowSealedReadOutcome.Unreadable, whichRefusedForUnreadableIndexturns into a hold. The two unattended FIRST-IMPORT callers (ModuleDiscoveryService.FirstImport,InstanceAutoRegistrationService's boot default install) read the index withReadFor, which discards the outcome, so they would have provisioned a Space from the branch TIP; both now askRefusedFirstImportForUnreadableIndexfirst. 🚨 And the same false-for-errors class runs through every read in that file, which the review of the phase-5 PR named and which is fixed with it:Directory.Existsanswers false for an absent identity AND for one this process may not enumerate, so the walk is now attempted and its own exception decides (DirectoryNotFoundException+ nothing at the path ⇒ the clean empty reading; an entry that IS there, an ACL, an unmounted share ⇒Unreadable). The seal is read throughShippedPrebuiltBundles.ReadSealLines, where the OPEN classifies absence and every other I/O failure surfaces. AndReadMarkerno longer swallows a failed read into the samenullan absent marker gets — null attribution is precisely what the gate answers withGo, so an ACL onrepository.txtcould have opened the gate a torn publication is there to hold.PublishedBundleCatalogue.EverSealedBundles— the release gate's denominator — read the declaration of whatever the fall-back landed on, so a faulted pointer silently dropped that source's packages out of the floor. A smaller floor is the one direction that EXEMPTS a package instead of holding it, so it now refuses (SealedBundleFloor.Unreadable).PrebuiltBundleRetentionalready kept every GENERATION of such a source; the identity holding them was protected only by "the newest SEALED publication of this source", and such a source now reads unsealed. One more keep rule, with the same sentence: an inventory that could not be read licenses no deletion.check-release-availability.shclassified it as ABSENT —no sealed publication under …, the one message that means an upstream has not published, and the wording that held MeshWeaver.Reinsurance 23 times in 24 hours (#3583). It is now CANNOT-DETERMINE, naming the pointer, with the re-run that actually helps. 🚨 And the fault itself has two buckets, which the review of the phase-5 PR asked for and which this file's own "TWO BUCKETS, NEVER ONE" rule already required: a probe that ERRORED on_current(the existence query answered neither true nor false, or the download of a pointer that exists failed) means the gate could not ask which publication is live, so a sealed flat copy behind it is not evidence — it may be one a generation publisher is about to dispose of, or one a disposal already emptied. That is CANNOT-DETERMINE whatever the sentinel says. A pointer that WAS read and is merely unusable (blank, refused, dangling) keeps the reader contract's fall-back, so a sealed flat copy behind it still answerssealedexactly as it did before phase 4.
The residues, named
- A producer running a pre-phase-5 publisher re-creates the copy. Core's own
plugins-bakeon a reconcile resolves its scripts atplatform-ref= the commit it publishes, so an older run refreshes a flat copy that a phase-5 run then disposes of again on the prefix's next publication. It is bounded by one publication and reachable only by a torn pointer read in between. - A FLAT caller on a prefix with no resolvable pointer still publishes in place. The lane
default is
generationandNoCallerInThisRepository_PublishesFlatkeeps the reachable population at zero; #4249 promotes a flat caller that resolves a live_currentto a generation. MeshWeaver.Education'se2e/mesh/fetch-upstream.shhas an Azure-share FALLBACK lane that probes the prefix's own_completeanddownload-batches the prefix recursively. It is taken only whenMW_REGISTRY_URLis empty, which that repo's own preflight refuses, and it has been flattening generations since phase 4 — so phase 5 turns a silent union into a loud refusal on a lane the repo does not run. Fixing it is a satellite change; it is named here so the next reader of that script knows which phase it predates.
If the publication moves to an OCI registry
The fleet now has its own registry (cr.meshweaver.cloud) and a program to push plugin bundles into
it as OCI artifacts. It is worth writing down exactly what that does and does not close, because
"content-addressed" is easy to read as "the race is gone".
What it closes by construction. Blobs and manifests are addressed by the digest of their own bytes. Two publications pushing byte-identical content collide benignly — same digest, a no-op — and two publications pushing different content get different blobs, which cannot overwrite each other. A blob is immutable once pushed; there is no partial overwrite to interleave. So the mix — a set holding some of each publisher's bytes — becomes unrepresentable at the byte level, which is the same property the generation directory buys, obtained more cheaply.
🚨 What it does NOT close: a tag is a mutable, last-writer-wins pointer. If the publication is
addressed by tag, the defect simply moves from a directory prefix to a tag, and the migration
inherits it. Three conditions close it, and they are the rule already in force for images via
MW_IMAGE_DIGEST:
- Every bundle is pushed and recorded by DIGEST, never by tag alone.
- Each publication also gets an immutable, identity-qualified tag — so a publication can be named without that name being reassignable to different bytes later.
- The sealed set is a list of
(package, digest)pairs, so a mixed set cannot be written at all: the set names exact bytes, and a publisher that did not assemble those bytes cannot produce that list.
🚨 Content addressing does NOT make two bakes of one commit converge. The bundle compile is not byte-reproducible — measured 2026-09-08, 40 of 45 files differed between two bakes of the same source commit — so two publications of one commit produce different blobs and therefore different manifest digests. OCI does not merge them; what it does is make each one a complete, immutable, self-consistent object, so the only contention left is which one the reference names. That is a last-writer-wins between two correct sets, which is a different and far weaker thing than a mix.
🚨 Two invariants the migration must carry across, or it reopens something worse than it closed:
- The framework identity must stay in the reference. Today the directory is keyed
prebuilt-bundles/<framework-identity>/<source>/, and that is not decoration: a bundle is only adoptable by a portal that resolved the same identity, which is why the publisher refuses when an incumbent'sarchitecture.txtdisagrees. A repository path of the shapeplugins/<source>/<package>:<version>carries no identity, so two platform surfaces' bakes would collide on one reference and a portal would adopt bytes built against a surface it does not have —dependency record mismatch — built against mvid:…, this issue's original symptom by another road. - The release marker must survive.
prebuilt-bundles/_releases/<version>→<identity>is the only way anything outside the image learns a release's framework identity, and two release gates HOLD on its absence.
And the refusal must not be weakened, in either world. On the share it reads "never seal a sentinel over another publication's bundles". In OCI terms it is the same sentence one level up: never publish a reference to a set you did not assemble. That refusal is the only reason this was ever visible instead of silently shipping a mixed set.
Where this stands
Phase 1 is landed (
a4109d422) — the readers resolve the pointer, and the fallback is the previous behaviour exactly.Phase 2 is landed — the writer can publish generations, behind
publication-layout. The lane's default isgenerationsince phase 4; the SCRIPT'sBAKE_PUBLICATION_LAYOUTfallback is stillflatand is the direct-invocation default, which is why the three control cases intest-publish-bake-overlap.pyremain the regression suite proving the flat path is byte-identical to what it was.Phase 3's reader half is landed —
compose-sealed-modules.shandnode-repo-gate.yml'sseedresolve the pointer, both are executed bytest-publication-pointer-readers.py, andbake-scope.sh --self-testis wired into CI beside it. Its pin half is satisfied by measurement rather than by a change: every producer floats at@mainwithscripts-ref: main(all six node repos, measured 2026-09-14), so each one RUNS the newest publisher and there is no pin left to move. With phase 4 and phase 5 landed (below), that closes #3461. 🚨 The sentence that stood here — "until the writer flips, the window is shrunk, not closed" — was true before phase 4, half true after it (closed for readers that follow_current, live for the flat copy the writer kept refreshing), and is spent since phase 5: nothing is written at the prefix itself any more, so there is nothing left there to replace in place.🚨 What the postcondition costs while this is open, measured 2026-09-08. Of 30 core-CD runs, 9 executed the bake job; of the 9 publications (either lane) that had a same-identity run overlapping them in time, 2 failed — 22%, and both were the two halves of ONE mutual supersession: core CD runs
34205409381and34206854855publishing the same content (cfac152ef…) for the same identity, each winning one storage target and reddening on the other. Both shares ended sealed with the right bytes and both CD runs failed, so the run produced no sealed set and the platform pin did not move. All 9 overlaps were same-lane; zero cross-lane — the same finding as 2026-09-06, on a different day.The convergence verdict removes the half of that which is a false red — a supersession by a publication proved to be this bake's own content, sealed. It takes 2 to 1 on that incident. The residual is a sibling that has not sealed yet when this run's sweep ends (21 seconds, measured), and that is not shrinkable by any amount of checking: it is what phases 2–5 exist for.
Phase 4 is landed (2026-09-14) — first
main-cd.yml'splugins-bakepassingpublication-layout: generation, then the lane input's own default moving togeneration, which is what flips the five single-producer prefixes (crm,education,reinsurance,socialmedia,manufacturing) with no change in their repositories. Every reader that follows_currentstops reading a directory mutated in place, on every prefix. The satellite's own flip is explicitness, not correctness (phase 4 above).Phase 5 is landed (2026-09-17) — a generation publication DISPOSES of the flat compatibility copy once
_currenthas moved and been read back: the seal first, alone, then the files it positively identifies. The precondition was measured on the fiveHosting/Deploymentrecords the same day, with the commit readbuildowed (3.0.0-ci.8411=c84c6c055). The readers whose fall-back was permissive were fixed with it — see "The reader contract phase 5 changes".The precondition on moving the default, measured 2026-09-14 rather than inherited. The prefix-ownership mechanism (#4249) lives in
publish-bake-bundles.sh, so it is a claim about the publisher each producer RUNS. Acrossci.ymlonmainof all six node repos everyuses:of anode-repo-*.ymllane is@mainand everyscripts-ref:ismain; none of the six passespublication-layout; andpublish-bake-bundles.shis named in exactly one satellite file, in a comment — nothing in the fleet invokes it outside the lane. So there is no pinned publisher and no pinned reader of any of these prefixes, and moving the default reaches all six producers at once.Moving the default was safe for a portal of ANY age, which is what separated phase 4 from phase 5. A phase-4 generation publication still wrote the flat compatibility copy, so a reader that could not resolve
_current— a portal image predating phase 1, or a torn pointer read on any image — was served exactly what it was served before. Phase 5 is the change that HAS a portal-age precondition, which is why it carries a census of the deployment records rather than an argument.The measurement the flip waited on. Measured over 2026-09-12T08:00Z → 09-13T08:00Z, every
pluginspublish job of both lanes: coreplugins-bake25 executed (23 sealed), Pluginspublish-bake10 (8 sealed); 62 of 62 target verifications read "N/N file(s) hold this run's bytes … 0 byte-identical from another"; 0 MIX refusals; 20 distinct identities, 6 written by both lanes — all 6 at different Plugins commits, hours apart, with no overlap in the window. So the defect is latent, not active, and the postcondition is carrying it. The layout belongs to the prefix (phase 4 above), so the two producers may flip in either order and the interval between the two merges cannot produce a stale serve. What phase 3 removed was the reason a flip could not be attempted at all; what #4249 removed is the reason it had to be attempted atomically across two repositories; core's half is now merged.🚨 Flipping
pluginsdid not by itself stop the publish reds — phase 5 is what did. While the flat compatibility copy was still replaced in place, an overlap cost that copy and failed the job, with the postcondition carrying it exactly as before; the flip bought only that the publication survived an overlap intact and pointed-to instead of being lost. Since phase 5 the two runs of an overlap share NO directory at all, so both succeed — asserted on the interleaved case intest-publish-bake-overlap.py, which now reads the inner run's receipt as well as the outer's exit code.What the postcondition costs in TIME, and the 2026-09-08 change to it. The sweep used to be one
az storage file showprocess per file on top of oneaz storage file uploadprocess per file — 184 CLI launches for two targets, 5–10 minutes of every bake. It is now one process per phase per target on the Azure SDK (publish-bake-files.py), with the verdict logic untouched; the table and the measurement are in Sealed Publication Reads → "What the postcondition costs". This changes the duration of the in-place window (the interval between the last verification read and the seal, and the interval a sibling can overwrite inside), not its existence — the layout above is still what closes it.The reds the postcondition produces are the correct number and must not be loosened away — see Sealed Publication Reads → "What is NOT closed".
Verification
.github/scripts/test-publish-bake-overlap.py— 145 assertions, 145 passed / 0 failed (measured 2026-09-17 on phase 5, 136 before its review round; it was 105 before the phase, 98 at the phase-4 default move, 90 at #4249), executing the REAL publish script against a stub share (the stubazfor the per-target decisions, a fake share backend for the bulk helper's uploads and read-back) and reading every verdict off the BYTES. The writer half is covered by seven generation cases: one publisher writes and seals under its own token and the pointer names it; two interleaved publishers each seal their OWN generation, neither directory holds a byte of the other, both are complete, and_currentnames exactly one of them; "already published" is resolved through the pointer rather than off the prefix; every unusable pointer shape (escaping, rooted,.., blank, dangling) degrades to the prefix; an unrecognised selector is refused; and the two mixed-layout cases — a flat caller that republishes over a pointed-at prefix, and one whose content is already live — assert that the first publishes a GENERATION and moves the pointer to it while the earlier generation stays whole, and that the second changes nothing at all. The three flat controls are unchanged and are the regression suite for the default. 🚨 Negative controls, each labelled with the change that measured it — a figure quoted without its change is how a verification section goes stale. (phase 2) against the pre-selector script, 16 of the 67 fail. (#4249, the prefix-ownership change) againstorigin/main's script on 2026-09-14, 87 passed / 3 failed — exactly the three assertions about the flat caller publishing a generation, withgenerations=['Systemorph-MeshWeaver-3501-1']and_currentstill naming it; that case's "the earlier generation is untouched" control and the whole never-backwards case passed on main too, which is what makes them regression controls rather than new behaviour. (#4273, the pointer-ordering postcondition) against the merged77ec6d4c74publisher, 4 assertions fail and_currentends on the older publication. (phase 4, the default move) the script is unchanged apart from comments, so the harness is the regression control rather than a negative one: 98/0 before and after. The phase-4 default itself is guarded in C#, not here —NodeRepoLaneHostGuard.ThePublishBakeLane_DefaultsToTheGenerationLayout, watched failing in both directions (default reverted toflat; the step'sBAKE_PUBLICATION_LAYOUTexport removed). (phase 5, the disposal) six negative controls, each one mutation, each run against the same 136 assertions: the publisher atorigin/main, which still refreshes the copy, 19 fail; disposing BEFORE the pointer moves, 7 (including every "a reader is never left with neither" line); the files deleted before the seal, 3; the pointer read-back removed, 2; a superseded run that disposes too, 2; and the per-target subshell put back in anifCONDITION, 4 — that last one is the pre-existing defect phase 5 found: bash suspendsset -efor a subshell run as a condition, so from #2682 every "fatal byset -e" inside a target was a no-op, and a failed_completeupload printed "sealed:" while a failed pointer move printed "this publication is now the live one". The disposal's own evidence is the fake backend's delete log, which records what_currentnames and whether each copy is sealed BEFORE every delete. (phase 5's review round) three more, one mutation each: the read-back weakened to "any sealed generation" — 4 fail, and the flat copy is deleted for a publication nobody points at, which is the finding itself reproduced; the post-disposal re-read of the seal removed — 3 fail, the prefix ending sealed over an emptied set; and, intest-release-availability-refusals.py, the read-ERROR bucket removed — case 3c's 4 assertions fail while its control (case 3d, the same sealed sentinel with the pointer probe ANSWERING) passes, so the split cannot be mistaken for "the gate now refuses everything"..github/scripts/test-publication-pointer-readers.py— 26 cases over three readers (was 18): the phase-5 prefix — a pointer, its generation, and NO flat copy — forcompose-sealed-modules.sh, the gate'sseedandcheck-release-availability.sh; a dangling and a blank pointer over that prefix, which the availability gate must call CANNOT-DETERMINE rather than an absent upstream; and the control that a prefix with no pointer and nothing sealed is still reported ABSENT. Againstorigin/main's availability gate 2 of the 26 fail, both of them those two.AnUnreadableSealIndexHoldsTest(10, was 4),PrebuiltBundleRetentionTest(53, was 51) andReleaseGateDenominatorTest(9, was 7) carry phase 5's reader half, each new case with its control on the same fixture and each watched failing with its own fix reverted (1 of 7, 1 of 53, 1 of 9). The three cases the review round added make an unreadable read UNFAKEABLE rather than privileged: the marker and the seal are made DIRECTORIES, which nochmodand no root can turn into a successful read, so the fixture holds wherever CI runs it.bake-scope.sh --self-test— four pointer-resolution assertions, and the positive one is discriminating by construction: the flat copy and the generation record different baselines (a diverged commit versus an ancestor), so the verdict itself says which was read. A resolver that ignored the pointer answersfull; one that follows it answersnarrowedwith the generation's commit. The dangling-pointer control asserts the fallback SAYS so, so silence cannot pass for resolution.carry-forward-bundles.sh --self-test— a decoy bundle is planted at the prefix under the same name and different bytes, so carrying the publication's own bytes forward is only possible by following the pointer.test/Memex.Portal.Shared.Test/PublicationGenerationTest.cs— builds the generation layout on disk and asserts what is served. The fixtures make the flat copy and the generation differ in bytes under the same file names, so "which publication was read" is a fact off the archive rather than an inference from a path. It covers the pointed-to read, the flat fallback, every escape shape, a dangling pointer with and without a flat copy behind it, a blank pointer, a retained older generation, and that moving the pointer moves the generation token (without which the412that stops an N+1 read spanning two publications never fires).- Negative controls, run against this tree. With
PublicationDirectoryOfreduced to the pre-#3461 reader (return sourceDirectory), 3 of 15 cases fail — the three that require resolution — and the other 12 hold, because they pin the fallback, which is unchanged. With the name validation removed but resolution kept, 4 more fail: the escape shapes. A resolver that cannot fail either check is not a resolver.
Related: Sealed Publication Reads · CI Content Bake · Plugin Build Contract · Bake Identity Mismatch