Repository instructions and skills

AGENTS.md defines the repository rules. The maintained skill files live under .claude/skills/; the tracked relative symlink .agents/skills -> ../.claude/skills exposes the same files to Codex. Edit or add a skill in the canonical directory once. The directory link also exposes newly added skills without a second manifest or synchronization step. Codex follows symlinked skill folders.

A one-time import that copies skills into .agents/skills/ creates an independent snapshot. Those copies can become stale while the checked-in skills continue changing. If an existing checkout has such an untracked directory, preserve it outside the checkout and review any local edits before updating to the tracked link. Do not commit the copied tree or hide it with an ignore rule. Git must materialize symlinks (core.symlinks=true, with filesystem support); a plain text file containing the link target does not enable skill discovery.

SkillFrontMatterGuard verifies that Codex's discovery directory resolves to the canonical skills and that the source skills parse. NoConflictMarkersGuard permits only this exact relative alias; other tracked links, including absolute and dangling links, remain errors. Machine-local .codex/config.toml MCP settings are separate from shared skill content.

Quick reference

🚨 The portal hosts live in MeshWeaver.Plugins since 2026-08-26 (MeshWeaver#2293 — the GUI extraction). Memex.Portal.Monolith, Memex.AppHost and Memex.Portal.Distributed are under src/ in that repository, and so are the login pages (Memex.Portal.Gui). A core checkout on its own has no portal to run: following an older copy of this page from core alone gives a process with no login UI and no control views (#2367). Every command below assumes the standard sibling layout — MeshWeaver/ and MeshWeaver.Plugins/ side by side — which is what plugins' src/Directory.Build.props defaults MeshWeaverRoot to (../MeshWeaver); set -p:MeshWeaverRoot=<path> if yours differs.

When you change code in Memex.Portal.Distributed (or any project it references — MeshWeaver.AI, MeshWeaver.Graph, MeshWeaver.Hosting.Orleans, …), you have three ways to apply the change without touching the whole Aspire stack:

All three options restart only Memex.Portal.Distributed; containers, dashboard, and OTLP collector stay up.

Do NOT kill the whole aspire / Memex.AppHost process unless you changed AppHost wiring itself. A full restart costs 30–60 s, rebuilds every resource, re-launches the Postgres and blob-storage containers, and invalidates the dashboard's browser-token URL.

Starting Aspire

The image-based AppHost (src/Memex.Deploy.AppHost (MeshWeaver.Plugins), and any customer AppHost on the published MeshWeaver.Aspire.Hosting.Memex package) declares its instance as a Deployment record — builder.AddMemex("memex").WithImage(…).WithPluginRepo(…)… — the same record the Helm chart renders from and the portal binds at boot (Deployment:Record). --record-out <path> writes the final record for the setup wizard or a Provision action. ConfiguringAnInstanceFromAspire.

Three modes — pick by whether you want to hold a terminal and whether you need a build:

# A. Interactive (foreground) — builds, holds the terminal, prints live status.
#    Registers with `aspire mcp` so MCP tools can drive it. Ctrl+C to stop.
aspire run --project ../MeshWeaver.Plugins/src/Memex.AppHost

# B. Background daemon — detaches and returns immediately; survives across shells.
#    `--no-build` reuses the existing binaries (fast — no rebuild). Drop --no-build
#    to build first. Also registers with `aspire mcp`.
aspire start --no-build --project ../MeshWeaver.Plugins/src/Memex.AppHost
aspire ps                 # list running AppHosts (Path · PID · dashboard URL)
aspire stop               # stop the background AppHost
aspire logs [<resource>]  # tail logs without the dashboard

# C. Visual Studio's "F5" path.
dotnet run --project ../MeshWeaver.Plugins/src/Memex.AppHost

🚨 aspire start --no-build reuses the LAST build. It's the fast way to bring the stack back up, but it does not pick up source changes — if you edited code, either drop --no-build, build the changed project first, or use one of the per-resource reloads below. Use --no-build when the binaries are already current (e.g. you just stopped Aspire to run a build and want it back up).

aspire run / aspire start print a one-shot dashboard token URL:

Dashboard:  https://localhost:17200/login?t=<TOKEN>

Open that URL in your browser. The token is per-process — restart Aspire and you get a new one. To skip token authentication in development, add the following to ../MeshWeaver.Plugins/src/Memex.AppHost/Properties/launchSettings.json:

"environmentVariables": {
  "DOTNET_DASHBOARD_UNSECURED_ALLOW_ANONYMOUS": "true"
}

Visual Studio's F5 path does this implicitly via its launch profile, which is why it never prompts.

Full stop → rebuild → restart (clean cold-start after a code change)

Hot reload (Option 1) and the per-resource restarts (Options 2–3) are the default. But when you want a clean cold-start with new code — a non-trivial change spanning several referenced projects, or recovery from a wedged stack — use the explicit stop → rebuild → start cycle. The two steps people miss (and that cause the "I rebuilt but it still runs old code" / "my build won't compile, file in use" loops) are in bold:

On macOS/Linux (zsh/bash):

# 1. Stop Aspire — AND force-kill, because `aspire stop` can exit while leaving the
#    AppHost + portal processes alive:
aspire stop
pkill -f Memex.AppHost; pkill -f Memex.Portal.Distributed

# 2. 🚨 RELEASE THE BUILD LOCKS. Lingering MSBuild / VB-C# compiler-server nodes keep
#    obj/*.dll open AFTER the AppHost is gone — that is the cause of the rebuild failing with
#    "CS2012: cannot open '…/obj/…/X.dll' for writing" or "MSB3021: being used by another
#    process". Killing the portal is NOT enough; you must shut the build server too:
dotnet build-server shutdown

# 3. Rebuild the portal project (it lives in the sibling MeshWeaver.Plugins checkout; MeshWeaverRoot
#    resolves back to this repo). Building Memex.Portal.Distributed also rebuilds every project it
#    references here (MeshWeaver.AI, .Graph, .Mesh.Contract, …) and in plugins (.Blazor.Portal), so ALL your edits
#    compile — this doubles as your compile gate before bringing the stack up:
dotnet build ../MeshWeaver.Plugins/src/Memex.Portal.Distributed/Memex.Portal.Distributed.csproj --no-restore

# 4. Start fast, reusing the build from step 3 (no second compile):
aspire start --no-build --project ../MeshWeaver.Plugins/src/Memex.AppHost

On Windows (PowerShell), step 1's force-kill is instead:

Get-Process Memex.AppHost,Memex.Portal.Distributed,aspire -ErrorAction SilentlyContinue | Stop-Process -Force

Why --no-build is safe here: step 3 already produced current binaries for the portal and everything it references, so --no-build just launches them. The Postgres / blob containers are persistent and survive the stop, so the database is intact across the cycle.

Option 1 — Hot reload with dotnet watch

dotnet watch --project ../MeshWeaver.Plugins/src/Memex.AppHost

Save a file and Aspire detects the change, rebuilds the affected project, and restarts only that resource. The Postgres container, blob-storage container, and dashboard all stay up. Most edits apply within seconds.

What hot reload handles and what it doesn't:

Option 2 — Dashboard restart

Navigate to https://localhost:17200/ → Resources tab → find the resource (e.g. memex-portal-distributed) → click the ⋯ menu → Restart.

This is equivalent to running dotnet build and relaunching the resource process. The hub state is wiped (action blocks drained, hosted hubs disposed), but the dashboard, OTLP collector, and every other resource stay up.

Use this when:

• Watch isn't running.
• A hub-configuration change (WithHandler<T>, WithRoutes, ConfigureDefaultNodeHub) needs to take effect cleanly.
• Hot reload reports an unsupported edit.

Option 3 — Process kill (last resort)

pkill -f Memex.Portal.Distributed                                                   # macOS / Linux

Get-Process Memex.Portal.Distributed -ErrorAction SilentlyContinue | Stop-Process -Force   # Windows

Aspire's resource watcher notices the exit and restarts the resource, typically within a few seconds. Use this when the dashboard UI is unresponsive (for example, a layout area is hung in JS and is blocking dashboard rendering), or when you want to confirm a clean cold-start.

What NOT to restart

When aspire mcp doesn't see your AppHost

If mcp__aspire__list_apphosts returns [] even though Aspire is running, the AppHost was started via dotnet run. Only the aspire CLI writes the discovery file the MCP server reads — both aspire run and aspire start do; dotnet run does not.

Fix — stop the current AppHost and restart with either CLI form:

aspire run --project ../MeshWeaver.Plugins/src/Memex.AppHost     # foreground
aspire start --project ../MeshWeaver.Plugins/src/Memex.AppHost   # background daemon

After this, MCP tools (list_resources, list_traces, list_structured_logs, execute_resource_command) all work against the running AppHost.

Logging triage from the dashboard

Navigate to https://localhost:17200/structuredlogs?level=info and filter by:

For distributed traces across resources, open https://localhost:17200/traces and click any trace to see span timing across the AppHost, Portal, and Postgres boundary.

Verify like CI on a Mac

Three things used to make a Mac unable to verify the way CI does. Each one now has a supported route.

In-mesh C#: a reference set built on this host

CI compiles and runs in-mesh C# (<Type>/Source/*.cs, <Type>/Test/*.cs) against the platform image. That means the tester image's /app, and in MeshWeaver.Plugins the portal image's /app with the tester's extra MeshWeaver.* assemblies laid over it. A Mac could not use either source:

.github/scripts/build-host-reference-set.py publishes that closure for this host's RID, framework-dependent: the tester (tools/MeshWeaver.PluginTester), plus the portal (src/Memex.Portal.Distributed) when you name a Plugins checkout. The portal is the base and the tester adds only the MeshWeaver.* assemblies the portal lacks, the same rule platform-mount.py applies in CI. The result is one flat directory, written next to a meshweaver-reference-set.json manifest. compile-check.py classifies the directory as layout host-build only because of that manifest. A flat directory without it is still an image extraction, and an image extraction missing its shared frameworks is still refused. The manifest records the core commit. The set is refused (exit 3) once that checkout's origin/main has moved past it, the same rule as a stale sibling checkout, so rebuild after you update core.

# 1. Build the set from your CORE worktree, naming the Plugins checkout you test (≈1–3 min).
python3 <core>/.github/scripts/build-host-reference-set.py --plugins <plugins>
REFS=$(python3 <core>/.github/scripts/build-host-reference-set.py --plugins <plugins> --print-path)

# 2. The registry-served modules come from the Plugins checkout's own bin (never in an image).
#    Build them against the same core (compile-check names exactly these when they are missing):
dotnet build <plugins>/src/MeshWeaver.AI -c Release -p:MeshWeaverRoot=<core>/      # likewise .Markdown.Collaboration, .Maps

# 3. Run the launchers against THIS core's scripts and THIS set (from the Plugins checkout):
export MW_PLATFORM_SCRIPTS=<core>/.github/scripts
python3 scripts/canonical-compile-check.py --modules Hosting --refs "$REFS"
python3 devtools/run-node-tests.py Hosting --type InstanceAction --refs "$REFS"

Measured on darwin-arm64 (Apple silicon): the set held 224 assemblies; canonical-compile-check.py --modules Hosting compiled 48 of 48 NodeTypes clean in 64 s; run-node-tests.py Hosting --type InstanceAction ran 56 suites with 760 of 760 cases passing in 21 s. Old sets stay under ~/.cache/meshweaver/refs/host-* until you delete them. The default path is keyed by the SOURCE STATE: the core commit, plus a digest of its uncommitted changes when there are any (tracked edits and untracked, non-ignored files with their content, since a new .cs compiles by default), and the same for the Plugins checkout. A source change while the set is being published refuses the install, because the key would no longer name what was built. Many worktrees sit at one commit and share this cache, so two dirty worktrees never build into the same path. Each build stages in a directory of its own, and a set already at its path is kept, never replaced under a session that uses it as --refs. Only this host's RID is accepted: the manifest declares the set host-built, which is what lets run-node-tests.py execute it here.

Testcontainers on Colima: no hand-set variables

On Colima a published port reaches the Mac's localhost only while Colima's port forwarder is up. When it is down, every Testcontainers fixture times out on localhost:<port>, even though the port is open on the VM's own address. Ryuk also fails, because it mounts the Mac-side socket path, which does not exist inside the VM. The hand workaround (a pgvector container started at the VM IP, MW_TEST_PG_CONNECTION, TESTCONTAINERS_RYUK_DISABLED=true) is no longer needed, and it is why containers were left behind.

In MeshWeaver.Plugins, every project that references a Testcontainers* package compiles src/Testing.Shared/LocalDockerEndpoint.cs, a module initializer. When the current Docker endpoint is a Colima socket, it sets Testcontainers' documented Colima settings, each only while still unset: DOCKER_HOST, TESTCONTAINERS_HOST_OVERRIDE (the VM address from colima list --json, which needs a VM started with --network-address) and TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock. Ryuk then runs, so a killed test host's containers are reaped. For any other Docker endpoint, and on CI, it does nothing. A plain dotnet test works. Run it from the core checkout, whose global.json selects Microsoft.Testing.Platform:

cd <core> && dotnet test --project <plugins>/src/MeshWeaver.Hosting.PostgreSql.Test/MeshWeaver.Hosting.PostgreSql.Test.csproj \
  -c Release --no-build --filter-class MeshWeaver.Hosting.PostgreSql.Test.BlankDescriptionFilterPgTests

🚨 Never run memex-local up to get a database for a test. It runs az acr login against the default subscription.

Leftover test containers

Every MeshWeaver fixture labels its container org.meshweaver.test=<fixture>, beside Testcontainers' own org.testcontainers* labels, and disposes it on teardown. Ryuk reaps the containers of a host that died first. For anything older, scripts/clean-test-containers.py in MeshWeaver.Plugins lists labelled test containers older than --older-than hours (default 12), and removes them with --apply. It removes an unlabelled container only when named with --name, and only for a Postgres image. It refuses k8s_* containers and anything carrying a Kubernetes or Compose label (the memex-local stack), even when named. It also refuses a running Postgres container that has a client connected.

python3 scripts/clean-test-containers.py                    # dry run
python3 scripts/clean-test-containers.py --older-than 6 --apply

Common gotchas

"No AppHost is currently running" from aspire mcp. The AppHost was started via dotnet run, not aspire run. Restart with the latter (see above).

Builds fail with MSB3021: cannot copy … being used by another process or CS2012: cannot open '…/obj/…/X.dll' for writing. A previous AppHost/portal and/or a lingering MSBuild / VB-C# compiler-server node is holding the binaries open. Killing the portal alone is often not enough — the build-server nodes survive it. Force-kill the processes and run dotnet build-server shutdown, then retry. See Full stop → rebuild → restart for the full cycle.

Portal's chat hangs at "Allocating agent…". Almost always one of two causes:

1. The portal hub's TypeRegistry doesn't have AppendUserMessageResponse registered — the response arrives as RawJson and the original Observe never resolves. Fix: WithPortalConfiguration(c => { c.TypeRegistry.AddAITypes(); return c.AddData().WithGraphTypes(); }) in MemexConfiguration (already applied in MemexConfiguration.ConfigureMemexPortal).
2. Multiple portal hubs created per page navigation (the per-DI-scope shape) — a chat response routes to an already-disposed transient portal. Fix: the per-user portal address in PortalApplication.

Both issues are post-mortemed in commit messages on the routing branch — see git log --grep "portal hub".