Learn

Why AI Agents Fail—and What to Try Next

A troubleshooting ladder separates AI agent failures into input, instruction, access, tool, model, boundary, and review layers.

Quick answer: An AI agent can fail because it received the wrong files, misunderstood the assignment, lacked access, used a broken tool, chose an unsuitable model, hit a boundary, or produced an answer nobody checked. Identify the failure layer first. Then change one thing and rerun a small test.

The tempting response is to switch to a bigger model or write a much longer prompt. Both can make the real cause harder to see. A missing file permission will not be repaired by better prose, and an unreadable scan will not become legible because the model is larger.

Start with the symptom

What happened Check first Try next
A file was skipped Was it selected, readable, and in a supported format? Test that file alone
A fact was invented Did the task require a source or allow not found? Require page or file anchors and an exception value
The format was wrong Was the output schema explicit? Give column names or headings and one short example
The agent stopped Did it hit an ambiguity, permission, or tool error? Read the exact stop message before changing anything
A tool failed Could the tool open that input and produce that output? Test the tool path with one simple file
The answer was shallow Was the relevant evidence actually available? Narrow the source set and ask for cited findings
A connection was blocked Did the chosen mode permit that model or tool path? Choose an allowed path; do not ask the model to bypass it
The result varies wildly Did the files, model, tools, or instruction change? Freeze the setup and compare two runs

This table is diagnostic, not universal. The useful question is not “Why are agents unreliable?” It is “Which observable part of this run did not behave as expected?”

1. Check the input before the intelligence

Open the source files yourself. Confirm that the text can be selected, the spreadsheet has the expected sheet, and the folder contains the version you intended to use.

A scanned PDF may contain only page images. A table may be visually clear to a person but poorly represented in extracted text. A workbook may hide the relevant data in another tab. If the input is incomplete, duplicated, or unreadable, the agent may be working from a different task than the one in your head.

For a first retry, reduce the input to one representative file. If that works, add the remaining files in small groups.

2. Turn the request into a work order

“Review these files” leaves too many decisions open. Name the deliverable and the rule for missing information:

Create contract-summary.md from the selected agreements.

For each agreement include:
- parties;
- effective date;
- termination notice;
- source filename and page;
- `Not found` when the source does not state the value.

Do not change the source files.

This is shorter than many elaborate prompts, but it defines the artifact, fields, evidence, and boundary. See how to write instructions an AI agent can follow for a reusable task-card format.

3. Separate access failures from reasoning failures

If the agent could not open a file, write to the target folder, call a model, or use a tool, the failure happened before answer quality mattered. Read the activity or error record and ask:

  • Which file, model connection, or tool was attempted?
  • Was the action unavailable, denied, or unsuccessful?
  • Did the agent stop, or did it continue without the missing input?

Do not weaken a boundary merely to make the error disappear. If a task genuinely requires a network service, use a mode and connection intended for that work. If the task should remain offline, use a local model Connection and verify that every tool, script, application request, and supporting service in the path has no external network dependency. Model location alone does not make the whole task offline.

4. Test model fit with the same task

A model can be fluent yet weak at the exact work you need. Keep the files, instruction, tools, and output checks unchanged. Run the task with another suitable model and compare:

  • source coverage;
  • factual errors;
  • format compliance;
  • time and memory use;
  • review effort.

If you change everything at once, you learn nothing about the cause. Choose a model without reading benchmark tables by testing the work you actually do.

5. Inspect the path, not just the final answer

Agent work is multi-step. A polished final document can hide a skipped source; an imperfect draft can still show a sound process with one correctable tool error. Anthropic's agent-evaluation guidance distinguishes the final outcome from the trajectory used to reach it. For an ordinary user, that means checking both:

  • Did the artifact meet the acceptance checks?
  • Which files and tools contributed to it?
  • Were blocked or failed steps visible?
  • Did the agent change anything it was meant only to read?

If the product does not expose a relevant path, record it as unknown. Absence of a visible log is not proof that an action did or did not occur.

Use a one-change retry ladder

  1. Reproduce the failure with the smallest representative input.
  2. Record the exact error or failed acceptance check.
  3. Classify it as input, instruction, access, tool, model, boundary, or review.
  4. Change one item.
  5. Rerun the same test.
  6. Keep the change only if the named failure improves without breaking another requirement.

This is different from proving that a workflow is ready for repeated use. For that, use the two-run AI agent test. For the broader control model, see how AI agents stay reliable.

Where Agenaxy fits

Agenaxy is a local-first AI agent workbench for file assignments. The useful troubleshooting unit is a visible run: selected files, chosen model Connection, available tools, Activity, and an editable Artifact.

In Vault, every model Connection must be explicitly authorized. Outbound-data tools are unavailable, and agent-run scripts are blocked from network access. A blocked step should be treated as a real boundary decision, not disguised as a prompt problem.

Try one controlled retry

Describe one non-confidential failed task and the artifact you expected in Try Agenaxy. Do not submit files, credentials, customer records, or production data through the form.

FAQ

Should I always try a bigger model first?

No. First rule out missing input, unclear output, file access, and tool failures. Then compare models on the same task.

Does a stopped agent mean the workflow failed?

Not necessarily. Stopping on a missing value or disallowed action may be the correct behavior. Judge it against the work order.

How many retries are enough?

There is no universal number. Stop when you have isolated the cause and the review cost no longer justifies the expected benefit.

Sources and Fact-Checking Notes

Was this useful?