copilot-agent-answers-wrong

Find out why a Copilot Studio agent is answering wrongly — whether the content is missing, unreachable, or being ignored — before changing anything.

Procedure

  1. start start. Goes next to s1
  2. check Reproduce it three times, three sessions. Goes on_fail to s9; next to s2
  3. decision How often did it fail?. Goes if intermittently to s6; if default to s3
  4. check Ask it again as two other users. Goes on_fail to s9; next to s4
  5. check Search the source for a phrase from the doc. Goes on_fail to s9; next to s5
  6. decision Was the content found?. Goes if not-found to s7; if default to s6
  7. escalate Hand to whoever owns that ingestion. Terminal
  8. check Audit the instructions, citation rule first. Goes on_fail to s9; next to s8
  9. escalate Escalate to the agent's owner. Terminal
  10. action Write down the cause and its evidence, RW (reversible-write). Goes next to __end_success
  11. end success. Terminal

Open the graph on its own page — larger, printable, and an address you can send.

Walk it without running it — you answer what your environment would answer, and the path, the capabilities, the approvals and the distance back to an undo come out of the document. Nothing executes; there is no Run button here and there will not be one.

Trust

T2 · attested 2026-10-11 · runbooks.directory

Nobody has attested this record. That is the ordinary case: T2 is what the catalog can check on its own, and T3 needs somebody to run the procedure and sign for it.

  • Link liveness has not been checked: it needs a network and the catalog builds without one. Run pnpm gates --online where there is one.

Steps

  1. s1 check — Reproduce it three times, three sessions
  2. s2 decision — How often did it fail?
  3. s3 check — Ask it again as two other users
  4. s4 check — Search the source for a phrase from the doc
  5. s5 decision — Was the content found?
  6. s6 check — Audit the instructions, citation rule first
  7. s7 escalate — Hand to whoever owns that ingestion
  8. s8 action — Write down the cause and its evidence (reversible-write)
  9. s9 escalate — Escalate to the agent's owner

Capabilities

This is a privilege request. Check it against your own allowlist before running.

    Provenance

    Source
    https://github.com/microsoft/ai-agent-runbooks/blob/083f33eec889/02-patterns/Grounding-and-Response-Quality-Remediation/Grounding-and-Response-Quality-Remediation-Runbook.md
    Fetched
    2026-09-04
    Source last checked
    not since it was captured on 2026-09-04 — recorded as current
    Licence
    MIT
    How it got here
    Rewritten here from the source above, under its licence. The words and the steps are this catalog's; the procedure it describes is not this catalog's invention, and the source is where to check it.
    Highest risk in this procedure
    reversible-write

    Take it with you

    Run it under supervision

    Runs here, on your machine, under supervision: every step is proposed before it happens, an approval gate stops for you, and a failed expectation goes where the procedure says rather than on to the next step. Nothing is sent anywhere. To supervise an agent that is doing the work instead, runbooks hook sits in its tool-call path.

    npx @runbooks/cli run adapted/copilot-agent-answers-wrong@1.0.0 --as "$(whoami)" \
  --input question=<the question, in the words the complaint used>