AI Agents in Production: Guardrails, Human Approvals, and Audit Logs That Actually Work

AI agents are non-deterministic. Given the same input twice, they may take different paths, call different tools, and produce different outputs. In a demo, this is a feature. In a production workflow handling customer data or financial transactions, it is a risk surface that requires explicit engineering.

This guide covers the four guardrail layers that separate production-grade AI agents from demo agents — and the audit infrastructure that lets you prove they are working.

Layer 1: Scope limits (what the agent can and cannot touch)

The most fundamental guardrail is the tool list. An agent can only do harm with tools you give it. Define every tool explicitly — with its allowed inputs, output schema, and rate limits — and give the agent access to nothing it does not need.

Read-only by default

Start every new agent with only read-only tool access. Add write access for specific tools only after the agent has proven accuracy in read-only mode across 100+ real tasks. This is the single most effective risk-reduction pattern in agentic systems.

Data scope restriction

If an agent processes one customer's data, it should not be able to query other customers' records. Implement row-level security in your database that the agent's credentials enforce automatically — do not rely on the agent's instructions to prevent cross-customer data access.

External scope

Define which external systems the agent can reach: which APIs, which webhooks, which outbound email domains. Block everything else at the network layer (egress allow-list). An agent that can send email to any address can be prompt-injected into exfiltrating data.

Layer 2: Cost caps

LLM API costs are variable per token. An agent that enters a reasoning loop can spend $50 on a task that should cost $0.05. Implement hard stops at two levels:

• Per-task budget: maximum dollar spend per individual agent run. Typical: $0.10–$2.00 depending on task complexity. Hard-stop and alert if exceeded.
• Per-day budget: maximum total spend across all agent runs per 24-hour period. Set at 3–5× expected daily spend. Page on-call if exceeded.
• Step limit: maximum number of tool calls or reasoning steps per run. Prevents infinite loops. Typical: 15–30 steps for most workflows.
• Time limit: maximum wall-clock seconds per run. Typical: 30–120 seconds. Longer than this usually means a loop.

Layer 3: Human approval gates

Approval gates are specific action types that the agent must pause on and request explicit human confirmation before executing. The agent drafts the action, the human reviews and approves or rejects, the agent proceeds.

What should require approval?

• Financial: any payment, refund, or credit over a defined threshold ($100–$1,000 depending on business context).
• Irreversible: file deletion, account closure, permanent data modification.
• External communication: emails or messages sent to external parties on behalf of a brand (until the agent has an accuracy track record).
• Privileged access: anything that involves admin permissions or elevated data access.
• Ambiguous intent: when the agent's confidence in the correct action is below a threshold, require a human to pick.

How to implement approval gates

Approval gates should be asynchronous — the agent generates a structured approval request (action type, rationale, proposed parameters, expected outcome, rollback plan if rejected) and pauses. The human receives a notification, reviews in a purpose-built UI, and approves or rejects. The agent resumes. Target approval response time: under 30 minutes for non-urgent, under 5 minutes for time-sensitive workflows.

Layer 4: Audit logs

Every agent run should produce a complete, queryable trace. This is your evidence layer — for debugging, compliance, and continuous improvement.

What to log

• Run ID, start/end timestamps, task type, triggering context.
• Every prompt sent to the LLM (full text, not summary).
• Every LLM response received (full text + token count + model + latency).
• Every tool call: tool name, input parameters, response, latency, cost.
• Approval gate events: action proposed, who approved/rejected, timestamp, rationale.
• Final outcome: success, failure, partial completion, cost total.
• Any errors or exceptions with full stack traces.

Log storage and retention

Store agent logs in a structured store (Postgres, BigQuery, or Datadog) — not flat text files. You need to query "show me all runs where the agent called the payment API in the last 30 days" reliably. Retain for at least 90 days for debugging; 7 years if you are in a regulated industry.

Testing your guardrails before production

1. 1Red-team your tool list: try to prompt-inject the agent into calling tools it should not have access to. Every red-team test should fail.
2. 2Trigger cost caps deliberately: run a task designed to loop. Confirm the hard stop fires and alerts.
3. 3Test approval gates end-to-end: run a task that requires approval with the approver absent. Confirm the agent waits, not acts.
4. 4Simulate a corrupt tool response: return garbage from a tool API. Confirm the agent handles it gracefully, not by hallucinating the result.
5. 5Audit log completeness: run 10 tasks, query your log store, verify all 10 appear with full traces.