AI Agent Governance: What Your Agent Can Do, Who Approves It, and How to Stop It

AI agent governance is the set of permissions, decision rules, ownership, and operational controls that determine how an agent may act. For a software team, that means answering concrete questions: which repository can it read, which branch can it change, who can approve a merge, and what stops the next tool call after the task is cancelled?

A coding agent can produce a plausible fix and still create a problem by editing the wrong project, exposing private context in a public comment, or executing a test with production credentials. The objective is to make useful autonomy available within a defined boundary. This guide turns that boundary into a permission matrix, a worked coding-agent policy, and checks a CTO can ask the team to demonstrate.

What AI Agent Governance Has to Control

OWASP's Excessive Agency guidance identifies three causes of damaging agent actions: excessive functionality, excessive permissions, and excessive autonomy. They are different engineering problems. Removing an unnecessary tool reduces functionality. Restricting its service account reduces permissions. Requiring approval for a consequential action reduces autonomy.

The NIST AI Risk Management Framework provides a broader, voluntary approach to managing AI risk across design, use, and evaluation. For an engineering team, the practical output is a maintained operating policy and evidence that it works. A one-time review of a prompt does not cover later changes to tools, credentials, or deployment configuration.

Start with a Permission Matrix, Not an Admin Token

List actions by resource and consequence before granting access. "GitHub access" is too broad: reading an assigned repository, creating a draft pull request, modifying a workflow, and merging into the default branch are separate capabilities. Apply permissions at the downstream service as well as at the agent's tool interface.

The following matrix is a starting policy for a coding-agent pilot. Adapt it to the task and data involved; the action category alone does not determine whether it is safe.

Read-only does not mean risk-free. Retrieved files can contain confidential information, malicious instructions, or data belonging to another customer. A read-only query can also consume substantial database resources. Check whose data the requesting user may access, restrict what enters the model context, and control where resulting content can be sent.

Use task-scoped or short-lived credentials where supported, and a revocable dedicated identity where they are not. Keep development, staging, and production authority separate. The agent should not be able to edit its own policy or obtain a more privileged identity through a fallback tool.

A Worked Policy for a Dependency-Update Agent

Consider an agent assigned to update a dependency in one backend repository. A useful boundary lets it inspect the code, prepare a patch, and show the result without also granting deployment authority. This is an illustrative policy outline, not a configuration file for a particular framework:

Task: update the dependency named in the assigned issue
Repository: the assigned backend repository only
Read: task-relevant source, tests, and dependency metadata
Write: the assigned agent/task-id branch and isolated workspace
Execute: approved test/build workflows in the isolated environment
Network: approved dependency and model endpoints only
Credentials: scoped to the task; no production secrets
Deliverable: one draft PR with the diff and validation results
Disabled: merge, deploy, change permissions, publish unrelated comments
Limits: shared task budget, deadline, and tool-attempt limits
Stop: pause execution and record any unfinished operations

Translate every line into a control. Validate the target repository and branch in the gateway; a branch naming convention alone is not authorization. Restrict network access at the execution boundary. Run install and test commands without production secrets, because dependency scripts and tests execute code even when the command name looks familiar. Review the complete diff, including lockfiles and workflow changes, before merge.

GitHub's Copilot Agents application card gives a concrete product example: its cloud agent uses an ephemeral environment, constrained repository and branch access, and a default network firewall. GitHub also calls for reviewing and testing generated changes. These are product-specific controls to inspect, not properties automatically inherited by every local agent or custom integration.

Treat Tool Results and Repository Content as Untrusted Input

A coding agent reads more than the user's task. Issue descriptions, code comments, dependency documentation, and tool output may come from other people. The OWASP prompt-injection guidance describes how instructions embedded in such content can redirect an LLM application. The permissions problem appears when that content can influence a tool with authority the task never required.

For example, an issue might tell the agent to upload environment details to an external diagnostics service. The issue is evidence to analyse, not a grant of permission to send data elsewhere. Labelling external content and screening suspicious instructions can help, but the network policy and publishing tools must still reject unauthorized destinations if the model proposes them.

• Validate the operation and its parameters. Check the actual repository, object, tenant, path, recipient, and action. An approved tool name is not enough if its arguments select a different resource.
• Keep tools narrow where practical. A function for opening a draft PR is easier to constrain than an unrestricted shell holding a general-purpose access token. If shell access is needed, enforce isolation and credential limits around it.
• Preserve authorization through retrieval and delegation. A child agent, MCP server, or search index must not expand the user's data access simply because it operates through a shared service account.
• Check outbound destinations and returned content. A valid internal analysis can still disclose data when copied into a public PR, a message, or an external tool request.

A second model can help flag suspicious actions, but its decision is another model output. Keep deterministic access checks in the application and downstream systems. A refusal in the chat transcript is useful evidence of model behaviour; a rejected unauthorized API operation is evidence of enforcement.

Make Human Approval Specific Enough to Mean Something

A confirmation such as "allow the agent to finish the task" is too broad for a consequential action. Show the reviewer what will happen: the exact diff or message, destination, affected records, relevant test results, and any uncertainty. The reviewer needs evidence of the action, not just a persuasive summary written by the agent.

Bind approval to an immutable proposed action: its operation, target, payload or diff digest, source revision, and validity period. Record the approver's identity and check their authority. If the content or target changes, require a fresh decision. Immediately before execution, recheck permissions, expiry, cancellation state, and relevant resource versions. Approval should not become a reusable token for a different action.

Separate actions that a team can preauthorize from those that need case-by-case review. Reading an assigned file or updating a draft in the approved workspace usually does not need repeated confirmation. Releasing a change, modifying access, or sending a novel external message can have a different policy. Narrow, predictable actions can earn more autonomy through evaluation without granting unrestricted authority.

Plan for Stopping, Recovery, and Actions You Cannot Undo

A branch can be discarded. A released change may need a rollback or forward fix. A message already delivered to a customer cannot be made unread, and deleting a public comment does not remove notifications or copies. Classify those consequences before deciding how much autonomy to allow.

For each write-capable workflow, define what recovery means and who can perform it. A practical stop-and-recovery procedure covers the whole task tree:

1. Block new actions. Mark the run as stopped in a shared control plane and check that state at the tool gateway. Queued operations and child agents must observe the same stop state.
2. Contain active work. Cancel remote operations where supported, terminate task environments, and revoke task credentials when appropriate. Stopping a local process alone may leave remote work running.
3. Preserve the action record. Capture affected resources, request identifiers, approvals, and known outcomes before cleanup removes useful evidence.
4. Reconcile uncertain results. If a request timed out, check whether it created a PR, sent a message, or changed a record before retrying. Use idempotency support where the downstream system provides it.
5. Recover and verify. Revert or compensate where possible, confirm the resulting state, and handle any external communication needed. Restart only after the relevant owner has reviewed the cause and the updated controls.

Circuit breakers can trigger on repeated denied actions, unexpected destinations, excessive publication volume, or exhausted time and spend budgets. Set thresholds for the specific workflow. Our AI agent cost-control guide covers shared budgets, retries, and paid operations in more detail.

Name the Owners and Keep an Action-Level Audit Trail

Ownership needs to cover business scope, technical controls, review, and incident response. In a small team, one person may perform several roles, but the decisions and access requirements should still be explicit. Oversight effort depends on action volume and consequences; it cannot be estimated reliably as a fixed hour or two per week for every deployment.

Record the user and root task ID, child task IDs, model and prompt versions, policy version, proposed operation, target resource, authorization result, approval reference, timestamps, downstream request ID, and actual outcome. Link a code change to the source revision, diff, and test results that were reviewed. Record denied actions as well as successful ones.

Keep logs proportionate to the investigation need. Redact secrets and unnecessary personal data, limit access, and set retention periods. Store sensitive artifacts behind controlled references instead of copying everything into a general log stream. A concise explanation and the observable action trail are more useful for accountability than relying on a model's private reasoning as an audit record.

Test the Boundaries Before Expanding Autonomy

Evaluate task quality and control enforcement separately. A correct patch does not prove that the agent cannot read another customer's data. A perfect record of blocked actions does not prove that the agent produces useful work. Your test set needs ordinary tasks, ambiguous cases, and attempts to cross the intended boundary.

Begin with an offline or shadow evaluation where write side effects are technically disabled. Next, allow bounded work in an isolated environment; then introduce reviewed writes for a limited set of tasks. Expand the allowed actions only when the evidence supports that change. The word "dry run" in a prompt is not a substitute for disabling the write capability.

Track accepted results, review effort, attempted and executed out-of-scope actions, approval rejections, stop-to-last-action time, and recovery outcomes. Repeat representative cases because model behaviour can vary. A pilot with no observed incidents is evidence about those test conditions, not proof that every future input is safe.

Working with an External Team: Agree the Control Ownership

An external team can build and operate an agent workflow, but responsibility should not disappear between organizations. Decide who owns each account, who can change the policy, who approves production actions, and who can stop the system. Document the handoff as part of the delivery scope.

• Accounts and access: identify the owner of repositories, model-provider accounts, cloud resources, and service identities. Agree how access is granted, reviewed, and removed.
• Change authority: specify who may alter tool scopes, network access, model versions, and approval policies. Permission expansion should be a visible configuration change.
• Operational access: make sure the responsible people can inspect action records, pause runs, revoke credentials, and use the recovery procedure during the agreed support hours.
• Handover evidence: include versioned policies, test results, deployment instructions, known limitations, and an exercised stop-and-recovery procedure in the deliverables.

If you are choosing how to staff the work, our in-house vs outsourcing comparison covers the delivery model. Regardless of that choice, keep the authority to approve the workflow and its access clearly assigned on your side.

When a Simpler Workflow Is the Better Choice

Use an agent when interpreting context and choosing among bounded actions adds value. If the task is a fixed transformation or a deterministic rule, ordinary automation may be easier to test and operate. If only the drafting step needs a model, let the surrounding workflow remain deterministic.

• Keep draft-only operation when review is still essential. A useful summary, patch, or message draft can save time without granting execution authority.
• Delay autonomous writes when success cannot be checked. If nobody can distinguish a correct result from a plausible one, increasing access will not solve that problem.
• Reduce scope when review becomes a bottleneck. A growing approval queue may mean the tasks are too broad or the output is too noisy. Measure review effort before adding more agents.
• Prefer explicit workflows for irreversible effects. Use narrowly defined actions and appropriate authorization when money, access, or external commitments are involved.

How UData Can Help Implement the Boundaries

UData's software development and automation services can support the engineering around an agent: scoped integrations, execution environments, approval interfaces, action logging, and evaluation tooling. Start with one workflow and a specific deliverable, such as a dependency-update assistant that produces a reviewable draft PR within an enforced access policy.

Bring the task, connected systems, intended users, and current release or review process. The initial scope can define the permission matrix, owner responsibilities, acceptance tests, and rollout plan. If you need engineers working within your existing delivery process, see our dedicated developers offering.

Operational ownership also includes losing access to the model itself. Our AI model access risk and continuity guide covers qualified fallback routes, portable task state, and recovery procedures that preserve the same authorization boundaries.

Useful Autonomy Has an Enforceable Boundary

Good AI agent governance makes the permitted work clear and the limits testable. Scope access to the task, authorize actions outside the model, bind approval to what will actually execute, and keep enough evidence to investigate and recover. Expand autonomy when measured results justify it. The outcome is an agent your team can operate, rather than a demo whose authority nobody has fully mapped.

Sources and Implementation Notes

• OWASP LLM06:2025 — Excessive Agency: minimizing functionality, permissions, and autonomy; downstream authorization.
• OWASP — LLM Prompt Injection Prevention Cheat Sheet: indirect injection, tool validation, least privilege, and layered controls.
• GitHub — Copilot Agents application card: product-specific execution controls, permission boundaries, and review responsibilities.
• NIST — AI Risk Management Framework: voluntary guidance for risk management across AI design, development, use, and evaluation.

Sources were reviewed for the September 25, 2026 revision. The dependency-update policy, matrices, and test scenarios are illustrative engineering recommendations, not a client case study or a certification checklist. Verify current product behaviour and translate the recommendations into controls for your own systems.