Practical AI Agents for Business Workflows

Why the Demo Works and Production Doesn't

AI agent demos are almost always impressive. A clean input, a clear task, a well-formed output. Then you try to deploy it on real data and the model starts making decisions that weren't accounted for, the output format drifts, something upstream returns an unexpected value and the agent proceeds anyway with bad context.

The model usually isn't the problem. The problem is that production inputs don't look like demo inputs, and most agent implementations aren't built to handle that gap.

Start Narrow

The worst AI implementations try to build a general-purpose assistant that can handle anything. The best ones pick one specific, well-defined workflow and build something that works reliably on that one thing.

Before building, you need clear answers to:

• What exactly is the agent doing? One sentence. If it takes more than one sentence, the scope is too broad.
• What does a correct output look like, and how will you know when it's wrong?
• What systems does it read from and write to?
• What happens when the input is ambiguous or incomplete?

That last question is where most pilots fail. The happy path works. The edge cases (missing fields, unexpected formats, inputs that don't fit any category) are where agents go wrong and often continue confidently.

The Architecture That Works

For most business workflow automation, the pattern is:

• Trigger: scheduled job, form submission, webhook, or API call
• Context assembly: pull the relevant data from internal systems before calling the model
• Model call: structured prompt with a defined output schema
• Output validation: check that the response is in the expected format and within expected parameters
• Action: write to a system, send a notification, or route to a human reviewer
• Logging: capture inputs, outputs, and decisions

Step 4 is the one that gets skipped. Models hallucinate. They go off-format. They return plausible-sounding outputs that fail schema validation. Treating model output as trusted input to downstream systems is the most common way AI workflows create data integrity problems that are hard to trace back.

Structured Outputs Are Worth the Effort

Using a defined output schema wherever the platform supports it makes validation trivial and makes prompts easier to iterate on without breaking downstream systems.

from anthropic import Anthropic
import json

client = Anthropic()

system = """
Classify the support ticket. Respond only with valid JSON:
{
  "category": "billing | technical | access | other",
  "priority": "high | medium | low",
  "requires_human_review": true | false,
  "confidence": 0.0 to 1.0
}
"""

response = client.messages.create(
    model="claude-opus-4-7",
    max_tokens=256,
    system=system,
    messages=[{"role": "user", "content": ticket_text}]
)

result = json.loads(response.content[0].text)

# Validate before acting on the output
assert result["priority"] in ("high", "medium", "low"), f"Unexpected priority: {result['priority']}"

Human Review Gates

For any workflow where the AI output drives a consequential action (modifying records, sending communications, triggering other workflows), build a human review gate before the action runs.

This is good workflow design, not an admission that the model isn't reliable enough. The gate should show the proposed action clearly, require explicit approval rather than a timeout-based default-approve, and log who approved and when. As confidence builds over time, individual gates can be removed based on actual error rates, not just intuition.

Starting without gates and adding them retroactively after something goes wrong is much harder than removing them as trust builds.

The goal is AI handling the volume and humans handling the exceptions. Gates are how you make sure the exceptions actually get handled.

Operational Realities

Token costs compound at scale. Log usage per workflow run from the start. Budget alerts are cheap. Finding out you've spent 10x your expected API cost after three months of production traffic is not.

Prompt changes are code changes. Version prompts, test before deploying, and have a rollback path. A prompt that worked perfectly for six months can start producing unexpected outputs after a seemingly minor change.

Context window management is the real constraint. If you're pulling in too much context, the model's attention gets diluted. Be deliberate about what you include. Retrieval over a large document store is almost always better than including the whole thing.

Know what data you're sending. For enterprise deployments, "we're sending employee data to an external LLM API" is a conversation that needs to happen before you build, not after you deploy.