When AI Starts to Act: Architecting Tool-Using Systems with Controlled Execution

An AI assistant that produces a wrong answer creates an information risk.

An AI system that calls the wrong tool can create a business event.

This is the architectural boundary that matters when we move from conversational AI to tool-using assistants and agents. The model may interpret a request, select a tool and prepare its parameters. However, it should not have unrestricted authority to decide whether an action is permitted or safe to execute.

The key design principle is simple:

The model proposes an action.

Deterministic controls authorize it.

A controlled execution layer performs it.

The platform records the outcome.

This separation allows an AI system to be useful without turning probabilistic reasoning into uncontrolled operational authority.

Tool Use Changes the Risk Model

Without tools, a language model usually produces information. The answer may be incomplete or wrong, but a person can still review it, question it or ignore it.

Tool use changes this relationship. The AI system may search private records, create a case, update a customer profile, send a message, trigger a workflow or submit a transaction. Its output now has side effects.

Once an AI-generated decision can change a business system, the architecture must answer a different set of questions:

• Whose identity is used to access the tool?
• Does the tool have more permission than the initiating user?
• Are the proposed parameters structurally and semantically valid?
• Is the action reversible or compensatable?
• Does it require confirmation, approval or separation of duties?
• How does the system prevent duplicate execution?
• What evidence explains why the action was proposed and allowed?
• How can the capability be restricted or stopped during an incident?

A prompt telling the model to “be careful” cannot answer these questions. Prompts guide model behavior. They do not replace authorization, validation, transaction controls or audit evidence.

Separate Proposal, Authorization and Execution

A reliable tool-using architecture separates three responsibilities that are often combined in early prototypes.

1. The model proposes

The model interprets the user’s request, identifies a suitable capability and produces structured arguments. It can also explain what it intends to do and what information is still missing.

2. The control layer decides

A deterministic policy and authorization layer checks identity, permissions, business rules, risk level, approval requirements and current workflow state. These decisions must not depend only on model confidence or natural-language reasoning.

3. The execution layer acts

A controlled component invokes the approved operation, handles idempotency and retries, records the result and returns a bounded response to the orchestration layer.

Consider an assistant that receives a request to change a delivery address. The model can identify the intent and extract the proposed address. Before anything changes, the surrounding system must still authenticate the customer, verify access to the relevant order, validate the address, confirm that the order has not already entered an incompatible state and apply any required confirmation step.

The model contributes interpretation. The business platform retains authority.

 

Place a Controlled Gateway in Front of Tools

Giving a model direct access to internal APIs makes policy enforcement inconsistent and incident response difficult. A stronger pattern places a controlled tool gateway between the AI orchestration layer and business systems.

The gateway becomes the common enforcement point for AI-originated actions. It can:

• Expose only approved operations.
• Enforce user and service permissions.
• Validate tool names and parameters against schemas.
• Apply value, volume, frequency and time-based limits.
• Require confirmation or approval for selected actions.
• Generate idempotency keys to reduce duplicate execution.
• Record the request, policy decision, approval and result.
• Disable or restrict a tool without redeploying every AI application.

The gateway does not replace security inside the target system. The business application must remain the final authority over its own data and processes. The gateway adds a consistent boundary for tool discovery, policy enforcement, observability and operational control.

This distinction is important. Central control improves consistency, but the architecture should not create a new privileged bypass around existing business rules.

Design Tools as Narrow Business Contracts

Broad tools such as execute_query, run_command or update_record give the model too much freedom to construct behaviour that the architecture did not intend.

A tool should represent a narrow business capability, not a generic technical primitive.

Instead of exposing a generic database update, provide an operation such as propose_customer_address_change. Define the required fields, accepted formats, length limits, authorization scope, expected response and possible failure states. The contract should describe the business action while hiding the underlying implementation.

Narrow contracts improve the system in several ways:

• The model has fewer ambiguous choices.
• Input validation becomes more predictable.
• Permissions can be assigned per capability.
• Logs describe a meaningful business event.
• Tests can cover known conditions and abuse cases.
• Contracts can be versioned without exposing internal implementation details.

Tool descriptions still matter because they help the model choose correctly. However, a description is guidance. The controls that enforce allowed behavior must remain in code, policy and the target system.

Preserve the Initiating Identity and Context

An AI application should not become a highly privileged shared account that performs actions on behalf of every user.

Where possible, each tool call should carry the initiating identity and the minimum business context required for authorization. The target system should be able to determine who requested the action, through which application, for which resource and under which delegated authority.

Some integrations require a service identity. In that case, the service account should have narrowly scoped permissions, and the execution layer should independently verify the end user’s entitlement. A valid service credential does not prove that a particular user may perform a particular action.

This prevents a common failure mode: the assistant can access or change more than the person using it.

Validate Meaning, Not Only Structure

Structured output is useful, but valid JSON is not the same as a valid business request.

The execution boundary should validate:

• Required and optional fields.
• Data types, formats and allowed values.
• Relationships between fields.
• Transaction, quantity and frequency limits.
• User permissions and resource ownership.
• Current business state.
• Whether the operation is valid at the current workflow stage.

For example, a payment amount may be a valid number and still exceed a permitted limit. A customer identifier may have the correct format and still refer to a record the user cannot access. An address change may be valid in isolation but no longer allowed after dispatch.

If validation fails, the system should ask for clarification, return a controlled error or route the request for review. It should not silently invent a replacement value or reinterpret a rejected parameter.

The same boundary helps protect against model error, prompt injection and ordinary user mistakes.

Match Human Oversight to Business Impact

Approving every tool call makes the system slow and frustrating. Automatically executing every tool call creates unnecessary risk. Human oversight should therefore be proportionate to the impact of the action.

The approval interface must show the exact proposed action and the parameters that matter. “Do you want to continue?” is not meaningful oversight if the user cannot see the recipient, amount, target resource or resulting change.

Approval must also be bound to the action that was reviewed. Authorization for one recipient, document or amount must not be reusable after the model changes the proposal.

Protect Multi-Step Workflows

Tool-using systems often perform several operations: retrieve a record, calculate a value, update one system and notify another. A failure midway through the sequence can leave the business process in an inconsistent state.

The architecture should define:

• Where each transaction begins and ends.
• Which operations are idempotent.
• How retries distinguish between failure and an uncertain response.
• How completed steps are compensated when a later action fails.
• Which state is the system of record.
• When the workflow must stop and move to manual handling.

These are familiar distributed-systems concerns. They become even more important when a probabilistic component can select the next step or alter the sequence.

Observe Both the Decision and the Effect

A conventional application log may show that an API returned a success code. For an AI-originated action, that is only part of the evidence.

A useful trace should connect:

• The initiating request and authenticated identity.
• The application, model and prompt versions.
• The relevant context supplied to the model.
• The proposed tool and normalized arguments.
• The policy and authorization decision.
• The approval record, when required.
• The request sent to the target system.
• The response and resulting business state.

Sensitive information should be minimized, masked and protected. At the same time, the organization needs enough evidence to investigate an event, reproduce the control path and assign operational responsibility.

Monitoring should also detect patterns rather than only individual failures. Repeated rejected calls, unusual tool sequences, rising costs, attempts to reach restricted operations and unexpected activity outside normal volumes can all indicate model failure, abuse or incorrect configuration.

Design a Stop Mechanism Before You Need It

When a tool or workflow behaves unexpectedly, teams must be able to reduce its capability quickly.

Useful operational controls include:

• Disabling a specific tool centrally.
• Changing an action from automatic execution to mandatory approval.
• Reducing value, rate or volume limits.
• Blocking a model, prompt or application version.
• Revoking a service identity or delegated permission.
• Routing the workflow to a manual process.

These controls are most useful when they can be applied without a full application release. A kill switch buried inside a deployment pipeline may arrive too late during an active incident.

Start with Bounded Actions

The best first tool-using use cases are rarely the most impressive demonstrations. They are the ones that help the organization learn while keeping failures contained.

Good candidates usually have:

• A well-defined user intent.
• A small set of approved tools.
• Structured inputs and outputs.
• Existing business validation.
• Low-impact or reversible effects.
• Clear success and failure states.
• A practical human escalation path.

These bounded workflows allow teams to build the identity, policy, evaluation, monitoring and incident-response capabilities needed for greater autonomy.

The goal is not to avoid tool use. It is to increase autonomy only when the controls around it are ready.

Final Takeaway

Tool use changes AI from a communication feature into an execution participant.

The model may understand the request, choose a capability and propose a useful next step. The architecture around it must preserve identity, enforce permissions, validate business meaning, manage approvals, protect transaction boundaries and record the resulting evidence.

Useful autonomy does not come from removing boundaries. It comes from designing boundaries that are explicit, enforceable and proportionate to the action.

Sources

• OWASP Top 10 for Agentic Applications for 2026.
• OWASP AI Agent Security Cheat Sheet.
• NIST AI Risk Management Framework.