When to Use an AI Agent — and When a Script Will Do

Two kinds of software

Traditional automation follows a path someone drew in advance. A PowerShell script, a Power Automate flow, an RPA bot clicking through a legacy screen — each one executes the same steps in the same order every time. If the input matches what the author expected, it works. If it doesn’t, the script stops.

An AI agent works differently. You give it a goal and a set of tools, and it decides which tools to use and in what order. Nobody drew the path. The agent works it out at runtime, and it may take a different route on Tuesday than it took on Monday.

Every other distinction worth knowing follows from that one.

What actually changes

The last two rows are where most projects get into trouble.

The failure modes are opposite, and this matters most

A script fails loudly. It throws an exception, the job exits non-zero, your monitoring fires an alert. The failure is obvious and it stops there.

An agent fails quietly and plausibly. It produces a confident, well-formatted, entirely wrong answer, and nothing in the pipeline notices. An invoice gets coded to the wrong cost centre. A ticket gets closed with a summary that misses the actual problem. There is no exception to catch, because from the system’s point of view nothing went wrong.

This is the single most important operational difference, and it drives the rest of the design:

• Verification has to be built in. With a script you test the code once. With an agent you need ongoing evaluation — a set of known cases you re-run whenever anything changes.
• “Anything changes” includes things you don’t control. Model providers update their models. Behaviour that was stable last quarter can shift without a deployment on your side. Pin model versions where the platform allows it, and re-run your evaluation set when you move.
• Audit trails need the reasoning, not just the result. For a script, the code is the audit trail. For an agent, you have to log which tools it called with which arguments, or you cannot reconstruct how a decision was reached.

Where traditional automation still wins

For predictable, high-frequency processes with little variation, a script remains the better and cheaper choice. That is the correct engineering decision, not a fallback.

Use conventional automation when:

• The input format is stable and structured
• The rules can be written down completely
• The same input must always produce the same output (billing, payroll, compliance reporting)
• Volume is high and margins per transaction are thin
• The process must be auditable line by line

A nightly sync that moves 40,000 records between two systems does not need a language model. It needs a well-tested script and decent error handling. Adding an agent adds cost, latency and a new class of silent failure, in exchange for flexibility the task never asked for.

Where agents earn their cost

Agents pay off where the variation lives, where writing down every rule is the hard part.

• Unstructured input. Fifty suppliers send invoices in fifty layouts. A parser per supplier is a maintenance treadmill; a model that reads them all is not.
• Long-tail exceptions. The 80% that fits the rules is already automated. The remaining 20% is why humans still touch the process, and it is too varied to encode.
• Context-dependent judgement. Triaging a support ticket, matching a bank payment to an open invoice, deciding whether two customer records are the same person.
• Multi-step work across systems. The task spans a ticketing system, a CRM and a file share, and the order of operations depends on what you find along the way.

The categories seeing real production use today are narrower than the marketing suggests: coding assistants that build and test changes, workflow agents inside back-office processes, customer-facing concierge agents for bookings and returns, and voice agents. Browser agents that drive arbitrary websites remain unreliable enough that most teams keep them out of anything that matters.

Emerging standards are making the plumbing less bespoke. Anthropic’s Model Context Protocol (MCP), open-sourced in late 2024, gives models a common way to reach tools and data sources. Google’s Agent2Agent (A2A) protocol, since donated to the Linux Foundation, addresses agents talking to each other. Both are young. Neither removes the need for the verification work above.

Human-in-the-loop is a design choice, not a limitation

There is a persistent assumption that keeping a person in the approval step means the technology isn’t ready yet, and that a mature deployment eventually removes them.

Treat it the other way round. The approval step is where you decide how much a silent failure costs you. Some decisions are cheap to get wrong and easy to reverse — let the agent run. Others are expensive, slow to detect, or regulated. Those keep a human, permanently, and that is a sound architecture rather than an interim one.

A useful way to place the boundary:

How to decide, in practice

Before committing budget, answer four questions about the process:

□ Can you write down the complete rules? → If yes, script it.
□ Is the input structured and stable?     → If yes, script it.
□ What does a silent wrong answer cost?   → Sets your approval design.
□ How many runs per month × cost per run? → Compare against the script.

If the process passes the first two, conventional automation will be faster to build, cheaper to run and easier to defend in an audit. If it fails them, an agent may be the only practical option, and the third and fourth questions tell you how to wrap it.

The most effective pattern we see is neither one nor the other: deterministic automation handling the predictable majority, with an agent taking the exceptions it kicks out. The script keeps its guarantees on the volume path. The agent is only exposed to the cases that need judgement, where its cost per run is justified and the volume is low enough to review.

If you’re weighing an agent against conventional automation for a specific process and want an independent view, get in touch.