Methods / 8 min
Count the work the owner can use, including review, repair and recovery.
Proposed worksheet · Method notes updated 2026-11-11 · No independent effectiveness test
The decision
Compare a narrow change against a competent baseline before calling it an optimization.
Use this when
You want to change instructions, models, tools or workflow structure to improve an agent’s work.
Skip this when
There is no defined task or quality contract yet. First make the desired outcome inspectable.
Work through the checks
Freeze the comparison
Specify the task set, inputs, tools, permissions, acceptance tests and conditions before running. Vary one factor where practical. Preserve failures and retries instead of quietly excluding them.
Measure the whole outcome
Track valid completion, elapsed time, owner review and rework, available billed usage and required checks. Token or tool-call counts diagnose cost; they do not establish usefulness alone.
Test the cases that can change the decision
Include normal, invalid and interrupted flows. Include a case where the new method should not help. Treat missing tools and unavailable measurements as unknown, not zero.
Decide with the limits visible
Report each task separately in a tiny pilot. Keep severe safety failures visible rather than averaging them away. Include setup and maintenance costs. A local improvement is not a universal claim.
What would count as success?
- The baseline is competent and both conditions share the same requirements. Evidence for check 1
- Results include failed and interrupted runs. Evidence for check 2
- The recommendation reflects quality, owner effort and measured costs together. Evidence for check 3
Evidence to bring back
Proposed evidence plan, not observed results or extra mandatory process. Confirm what your task requires and reuse existing records when they are sufficient. Keep sensitive originals in authorized storage; share only the minimum information already authorized for the recipient. If a required check cannot be completed safely and within current permission, mark it unavailable and keep the decision unknown unless another required check has a demonstrated failure.
The baseline is competent and both conditions share the same requirements.
Proposed evidence for acceptance check 1. No result has been observed here.
Bring back: Keep the frozen task inputs, shared requirements, permissions and version/configuration of each condition, with the baseline’s acceptance results. State differences that cannot be controlled.
Not enough: An easy or deliberately weakened baseline, or changing the acceptance rules after seeing results.
Results include failed and interrupted runs.
Proposed evidence for acceptance check 2. No result has been observed here.
Bring back: Keep a per-task run ledger including failed, interrupted and retried attempts, their outcomes and the reason for any exclusion. Record unavailable measurements explicitly.
Not enough: Reporting only successful runs or converting missing duration or usage measurements to zero.
The recommendation reflects quality, owner effort and measured costs together.
Proposed evidence for acceptance check 3. No result has been observed here.
Bring back: Keep per-task quality results, owner review/rework, measured usage where available, and setup/maintenance effort. Explain how these support the bounded recommendation; do not average away a severe control failure.
Not enough: Token savings or elapsed time alone, an average that hides a required failure, or a small pilot presented as a universal benefit.
Stop or recover
If quality regresses or required controls fail, stop the rollout and preserve the evidence. Revert only to a known safe state, not merely the older version.
What this does not establish
No controlled agent comparison is supplied by this worksheet. No savings, speedup or paid value is claimed.
Reference links
These links leave the saved pack when selected. Check whether dated guidance still applies before relying on it.
- Anthropic: building effective agents (Online source)
Engineering guidance on fitting workflow complexity to task needs. Not a controlled evaluation of this method.
Source review date: 2026-11-10
- METR: early-2025 developer study (Online source)
An empirical example of measuring complete developer outcomes in a particular population and period. Not a universal claim about agent productivity.
Source review date: 2026-11-10
Keep your own evidence
The draft already identifies this method and its proposed requirements. Confirm those requirements against your task, then supply only the scope and observations you are authorized to record. No evidence or permission is prefilled. Reuse an existing suitable record if you have one.
- Open this guide’s Markdown draft
- Open this guide’s JSON draft
- Inspect the guide’s source hash and method identity
Your browser may show or download text files. Edit a separate working copy in your usual editor. This reader does not accept or save your answers.
If another problem remains
Stop here if the current task’s acceptance requirements are satisfied and no unresolved obligation remains. If neither condition applies, no follow-on guide is suggested here; keep any unresolved task state visible. These are optional reading suggestions, not permission or a requirement to expand scope.
Only if: The comparison identifies recurring instruction work and you are considering a reusable package for it.
Review a skill before adding itReview the smallest relevant package instead of installing a broader workflow by default.
Only if: An interrupted trial may have produced an external effect whose outcome is still unknown.
Check the outcome before retryingResolve the effect before retrying; do not silently omit it from the comparison.