Wrong answers — reporting the chat's mistakes

/feedback is for formal defects: a broken link, an error card, a timeout, a button that does nothing. It hands anything that reads as technical to the Systemorph triage agent, which may file a GitHub issue. That is the right addressee for a defect and the wrong one for this:

"The coach said it doesn't know my hourly rate. It is in the Firmenprofil."

Nothing is broken. The chat gave a poor answer — and whose problem that is depends on why it was poor. Sent through /feedback, the words "doesn't know" and "wrong" match the technical classifier (FeedbackHandover.LooksTechnical) and the report becomes a GitHub issue on a code repository, where nobody can act on it. /wrong-answer (Feedback/Skill/wrong-answer) exists so the report lands with the right person, carrying the evidence that person needs.

What the chat can know about its own answer

Every assistant message is a ThreadMessage node under the thread, and it already records what a reviewer needs: agentName, modelName (and requestedModelName when a different model answered than was asked for), harness, every toolCalls entry with its arguments and result, and updatedNodes. The skill reads that cell — the user does not have to describe it.

It also runs under the user's identity. A node the skill cannot get is one the answering agent could not read either. That single check separates "the chat is broken" from "the user lacks a grant", and it is the reason the skill classifies by checking rather than by what the user believes.

The kinds, and who each one belongs to

Only retrieval is filed as category: bug and handed over after Submit. Every other kind is filed as category: answer, which FeedbackHandover.ShouldHandOver keeps in this instance's Inbox regardless of how technical the message reads — the addressee is a person on this instance, not a repository.

What the report carries

The same Feedback/Feedback node /feedback files, so preview, Submit, the Inbox and the review lifecycle are shared. The difference is the content: the user's account verbatim in message, the thread and message in sidePanel, and a fixed labelled block in extraContext:

Kind: not-found
Thread: sglauser/_Thread/2026-09-18-1319
Message: sglauser/_Thread/2026-09-18-1319/a7f3
Agent: AgenticOffice/Agent/OfficeCoach
Model: claude-sonnet-4-5   (requested: same; harness: native)
Tools called: 1 — Get(AgenticOffice/01-Willkommen) → ok
Answer (excerpt): "Ich weiss leider nicht, welchen Stundenansatz du verrechnest…"
Expected: "Konzept-Ansatz CHF 150 steht im Firmenprofil"
Evidence: get sglauser/AgenticOffice/Firmenprofil → readable; no Search in toolCalls
Addressee: agent owner (AgenticOffice/Agent/OfficeCoach — the course's maintainers)

Labelled lines rather than typed fields, deliberately: the volume does not yet justify a schema change, the card renders extraContext as-is, and an aggregation can parse Kind: / Agent: / Model: without a migration (WrongAnswerReport, Feedback/Feedback/Source, is that parser). Agent: holds the agent's path — the user cell's agentName — because it is the key reports are grouped by and the address an eval case runs against. When the volume justifies a schema change, the fields to add to FeedbackContent are exactly those five.

What to do with the reports

Routing is the first use, not the only one. In order of value:

1. Route to the addressee (the table above). Today the Inbox reviewer does this by reading Kind: and Addressee: off the card and moving the item to Triaged; retrieval alone routes itself. The step to automate next: raise a Notification satellite on the agent node for the agent-owner kinds, so a course author sees their coach's misses without watching a shared Inbox.
2. Turn a report into an eval case — shipped; see the next section. A report is a labelled example — this prompt, in this context, must ground its answer in node Z / must not assert F — and replayed whenever the instructions or the model change, these cases are the only defence against fixing one hallucination and introducing another.
3. Aggregate by model and by agent — shipped; see the weekly digest below. A count of answer reports per Model: over a window is the cheapest quality signal the platform has, and it should inform Provider/AiModelTiers — a model that fabricates on the Utility tier is doing exactly the job that tier must never fail at. Per Agent:, the same count says which instruction sets need work.
4. Close the loop. The reviewer's Detail view and the Triaged / Resolved / WontFix lifecycle already exist; a Resolved report whose addressee changed something should say what, so the reporter learns the chat got better because they said so.

Eval cases — a report becomes a repeatable test of the agent

A wrong answer is a sample of a rate, not a defect: the same agent with the same instructions invents something else tomorrow on another prompt. So a category: answer report is never handed over per report (FeedbackHandover.ShouldHandOver); it becomes an EvalCase (Feedback/EvalCase, compiled live from Feedback/EvalCase/Source) — Plugins#2288, PR 1.

Lifting. After Submit, the Compose card of a /wrong-answer report offers Make this an eval case. It creates a Feedback/EvalCase node under Feedback/_Evals/{agentId}/{case} (as System; agentId is the agent path with / → _, so Agent/ExecutiveAssistant groups under Agent_ExecutiveAssistant) carrying ONLY source — the report's path. The case's own hub lifts the rest when it activates (EvalCaseLifting.LiftOnActivation, the same shape as the hand-over on activation): it reads the report and the thread the report names, and fills

Any client can do the same with one create of that node type with only source set — the lift is the hub's, not the button's. Not under the agent node: module agents live in system-synced partitions where a live write is reverted at the next sync, and built-ins are served from the engine's content pack. Not in the reporter's space: a case outlives the reporter.

Running. The Eval area (the case's default view) shows the definition, the last run and a Run button. A run starts runs (default 5) real conversations with the agent, one after the other, under the case ({case}/_Thread/…), on the case's model when one is pinned — the composer's explicit selection, which the engine honours over the agent's tier and the deployment default; a headless round decides its model before the thread hub has loaded its agents, so the tier alone does not reach it — else on whatever the engine chooses for any thread, waits for each completed reply (ThreadFlow.ObserveResponses) and scores it by three deterministic checks on the reply's own ThreadMessage record — no model-as-judge (EvalScoring). Whose identity: the thread node is created under a System scope (the case lives in the shared space, which the viewer cannot write), but each round runs as the person who pressed Run — the AI engine never runs a round as an infrastructure identity (HubThreadExtensions.CaptureSubmitter rejects the ambient System context and takes the real circuit user; ThreadSubmission dispatches under that submitter) — so the agent reads the mesh with that person's access, and lastRun.ranBy records who, because two runs are comparable only when it agrees. The checks:

1. Looked — toolCalls holds a Get/Search whose arguments or result name a mustGround node;
2. Did not invent — the text does not contain a mustNotAssert value: numbers compare by their last nine digits (the subscriber part survives +41, 0041, (0)), text after case and whitespace folding;
3. Correct — if mustAssert is set, the text contains it.

The run is summarised as rates: check 2 must be N/N; checks 1 and 3 report a rate against a threshold (0.8 and 0.6 until thresholds are configured per agent). The last run is written back onto the case (lastRun) with every thread's score and a link to it, so a reviewer opens the answer a score came from. With no usable language model on the instance — or none for the pinned model — the run starts nothing and says so; it never fabricates a result. A reply that is the model-configuration error, or one the engine answered on another model than the pinned one, is a non-answer, not an invention.

Who confirmed a case — a stamped field, never typed

confirmedBy is the identity a nightly run takes, so it cannot be an ordinary field: whoever may edit a case could otherwise write another person's id into it and have the night run conversations with that person's access. The Feedback/EvalCase type therefore declares it a stamped field ("stampedFields": ["confirmedBy"] in its definition — the platform rule in core's Doc/Architecture/StampedFields): the platform refuses every client write that would set or change it — a cross-hub patch, an MCP update/patch, a create that brings a value — naming the field. Only the case's own hub (EvalCaseConfirmation) writes it: it watches its node and, when a person saved it confirmed, writes confirmedBy = that person — the node's lastModifiedBy, which the platform sets from the authenticated writer. Unconfirming clears it. The runner takes the stamped person; a case that is confirmed with no stamped confirmer yet is skipped and says so.

This needs a platform that carries the rule (core's stamped-field guard); on an older one the declaration is ignored and the field is an ordinary one.

The nightly runner

A scheduled, bounded job in code (Plugins#2288, PR 2) — models drift with no commit, so confirmed cases are re-run every night. No model decides what runs.

• Where it runs. On the shipped Feedback/EvalNights node, which the package anchors (anchors in index.json), so its hub lives as long as the portal. It checks hourly. Off by default on every instance; an instance turns it on by configuration.

• Declared configuration (Feedback:Evals:Nightly:*):

  Key	Default	Meaning
  Enabled	false	the runner runs at all
  MaxCases	15	at most this many cases a night
  ConversationsPerCase	3	conversations per case — 45 a night at the defaults
  StartHourUtc	1	a night starts at 01:00 UTC
  Model	—	the model a case that pins none runs on; without it such a case is skipped
  Effort	medium	the reasoning effort every conversation starts at

  A value that does not parse falls back to its default and the night names it.

• A night is an activity on a node: Feedback/_EvalNights/{yyyy-MM-dd} is created as System with its inputs and the trigger requestedStatus: Running; planning writes selected and a line for every confirmed case that does not run; each case adds its line as it finishes; the end writes status (Succeeded, or Warning when a case was skipped or failed), finishedAt, and clears the trigger. A portal that restarts mid-night resumes the selected cases that have no line yet. A night is never run twice.

• Order and cap. Never-run cases first, then the oldest lastRun, ties by path — the same cases always plan the same night, and every case gets its turn. The first MaxCases runnable cases run; a case that cannot run takes no slot; the rest are over-cap lines and the night's note says the cap was reached.

• Model and effort, explicit. The case's pinned model, else the declared Model — and only an EU-routed one (the model node's own residency mark). A model that is not EU-routed, or whose mark cannot be read, skips the case; the night never substitutes another model.

• Authorize as the confirmer, execute as System. Immediately before a case runs, the stamped confirmer must still hold Update on the case (a check that cannot be evaluated skips it and says so, never reads as a denial). The conversations are started with the thread's creator set to the confirmer — a headless round runs with its creator's identity and access — while the night's own reads and writes run as System.

• Results land on the case (lastRun, with night, model and effort), which is where the weekly digest reads them, and as a line on the night. A case that faults is a failed line and the night goes on.

Whether the CI gate mesh can hold provider keys, so cases run on a PR that changes an agent's instructions, is still open.

The weekly digest — one standing issue per agent

The second consumer of answer reports (Plugins#2288, PR 3) is the triage agent, which drains a queue on a cadence — but not one item per report. Once a week the Feedback module folds every report of the last completed ISO week into a digest (Feedback/Digest, compiled live from Feedback/Digest/Source) and posts one signed digest event per agent; the triage agent keeps one standing issue per agent current from it.

When and where. The schedule (DigestSchedule.RunWeeklyDigest) runs on the hubs of Feedback/Digest — the shipped Feedback/Digests node and every digest — checking hourly whether the last completed week is due (Monday 06:00 UTC) and not yet recorded. It then makes the ONE declared cross-partition read this module allows itself (nodeType:Feedback/Feedback partitions:all, once a week — a report lives in its reporter's space or in Feedback/_Submissions), reads every eval case, builds the digest and CREATES Feedback/_Digests/{week} as System, then reads the new node once so its own hub comes up and posts; a second hub loses the create and does nothing. Both reads are asked as system BY VALUE and fail closed (a timer tick has no ambient identity), and both carry an explicit page bound (200, newest first — the surface's upper end): a full page is recorded on the digest as truncated, a lower bound said out loud rather than a silent short count. A hub ticks only while it is activated, and a per-node hub is activated only by something reaching its address — so every Feedback/Feedback hub wakes the schedule node when it activates (FeedbackDigestWake): the reports themselves guarantee that a week WITH reports has a running schedule, and a missed quiet week is caught up at the next activation with nothing to post.

What it counts (WrongAnswerDigest.Build, pure): per agent (Agent:) the SUBMITTED reports (a draft — the preview the skill files first — is not a report until Submit) by Kind:, the models (Model:), the count of the week before as the trend, the eval cases lifted from this week's reports and, over all the agent's cases with a last run, how many passed — the health line; per model, across agents, the counts by kind. A report without a labelled block is counted under the unknown agent, never lost.

What it posts (DigestSchedule.PostOnActivation). The digest's own hub, seeing itself unstamped, sends one event per agent through the same route the feedback hand-over uses (FeedbackHandover.Deliver — the configured control inbox, or the local target on the control instance) and stamps postedAt, route and the accepted identities; that stamp is what makes the post run once. The event is feedback-shaped so the same intake records it: event: digest, feedbackPath: Feedback/_Digests/{week}/{agentId} (the identity — a redelivery is one item), message the agent's digest, extraContext the same as JSON, url the digest. A week with zero reports posts nothing — the digest is still written, as the record that the week was looked at. With no route configured the digest stands and says so.

What triage does with it (Hosting/Skill/triage §5). The intake carries the standing issue forward: an item's id is instance + feedbackPath and the path carries the ISO week, so the same agent's items of the previous twelve weeks are spelled — never searched, so no page limit or ordering can hide one — and the first issueUrl found lands on the new item. The agent then reads one field: issueUrl set → CommentIssue the week's numbers onto it; empty → FileIssue it once — title Wrong answers — , labels feedback and wrong-answer-digest, in the repository that owns the agent (never the fleet's inbox Systemorph/MeshWeaver.Feedback). retrieval counts are informational: those reports were filed as bugs when they were submitted. The ledger counts a digest on its own column.

Reading it. Feedback/Digests lists every week; a digest's Digest area shows the tables the events carried and what was posted. The area is a template that declares both shapes at once and binds the node's projection into /data/digestView (DigestLayoutAreas), so neither waits on the node or the index query before it renders.

What this is not

• Not a place to fix things. The skill never edits the node, never grants access, never re-runs the search and presents that as the answer. It records and classifies; a person acts.
• Not a verdict. The agent filing the report is usually the agent being reported. Its classification is a hypothesis with its evidence attached; the reviewer decides.
• Not a substitute for /feedback. A broken preview, an error card, a timeout: /feedback.

Tests

• Feedback/Feedback/Test/FeedbackTests.cs — HandOver_BugAndIdeaAlways_PraiseNever_QuestionWhenTechnical pins that category: answer is never handed over, however technical the message reads, and that a retrieval report filed as bug is; EvalCaseStub_IsUnderTheAgentsEvalsWithOnlyItsSource pins that the Compose card creates the case typed Feedback/EvalCase, under the agent's _Evals namespace, with only its source.
• Feedback/Digest/Test/DigestTests.cs (run by the type's Tests area) — the window (the last completed ISO week, due from Monday 06:00 UTC), the counts by agent, kind and model, the trend, the cases lifted and their last run, a quiet week posting nothing, the rendering, and one feedback-shaped event per agent; Hosting/Deployment/Test/TriageIntakeTests.cs pins that a digest parses, maps to its agent's item and carries the standing-issue rule.
• Feedback/EvalCase/Test/EvalCaseTests.cs (run by the type's Tests area) — the lift from the labelled block (every field; a /feedback bug is refused; an invented value is never the expected one), the case path, the three checks (looked only on a Get/Search, digit-insensitive invention, correct by digits or n/a) and the summary (not-invented unanimous, the rest rates, zero runs never a pass).
• Feedback/EvalNight/Test/EvalNightTests.cs (run by the type's Tests area, and as ordinary tests in MeshWeaver.AI.Test) — the declared settings and their defaults, which night it is, the cap, the deterministic order, refused cases taking no slot, the stamp as whoever saved the case confirmed, the runner taking only a stamped person, and the confirmer's right checked before the run. src/MeshWeaver.AI.Test/NightlyEvalRunnerTest.cs runs a night on a real mesh: the cap in order, a faulting case that does not stop the night, the stamped confirmer as every case's identity, a second pass running nothing; a forged confirmer refused by the platform on create and on patch; and real conversations through the engine on a stand-in provider.
• Follow-up: an e2e beside e2e/feedback-skill.spec.ts that drives /wrong-answer in a thread and asserts the draft's extraContext carries every labelled line.