Artificial intelligence becomes a very different proposition when it moves from producing an answer to taking an action. That distinction was brought sharply into focus by Anthropic’s recent disclosure that Claude models gained unauthorised access to real third-party systems during cybersecurity evaluations after a testing environment inadvertently gave them access to the open internet. The circumstances were unusual and normal cyber safeguards had deliberately been removed for the evaluations, but the episode exposes a problem that could become particularly important for financial services. Banks are preparing for AI agents capable of accessing customer information, navigating internal systems, initiating workflows and eventually performing financial actions, meaning the next phase of AI risk may be less about whether a model produces the wrong answer and more about what happens when a powerful model is given the authority to do something with it.
Anthropic reported four incidents involving different Claude models during cybersecurity evaluations. The models had been told they were operating in simulated environments without internet access, but a configuration error in a third-party testing environment meant the open internet was actually available. The models subsequently gained unauthorised access to real systems, prompting Anthropic to investigate approximately 481 million transcripts. The company said it did not identify additional incidents of similar or greater severity and has since made changes to its evaluation practices.
The details are important because the incident should not be interpreted as evidence that an ordinary AI assistant spontaneously escaped from a properly configured system. These were powerful models being deliberately tested on cybersecurity tasks in environments where safeguards had been removed, and a configuration mistake gave them capabilities they were not supposed to possess. Yet that qualification does not make the episode irrelevant to banking. In some respects, it makes the lesson more useful because the problem emerged from the combination of intelligence, tools, permissions and an environment that allowed the models to do more than intended.
Financial institutions are increasingly preparing to create versions of that combination deliberately. The industry is moving beyond generative AI systems that answer employee questions, summarise documents or draft customer communications towards agentic systems capable of completing entire workflows. An AI agent could retrieve information from multiple databases, interpret a customer’s request, use software tools, communicate with other applications, determine what action should be taken and potentially execute that action without requiring a human to approve every individual step.
This creates an important progression for financial services: AI that knows something can become AI that recommends something, which can become AI that acts, and eventually AI that has authority. Each transition changes the risk because an incorrect answer and an incorrect action are fundamentally different events. A hallucinated summary can potentially be corrected before anyone acts upon it, while an autonomous agent that misinterprets an instruction could begin triggering processes across multiple systems before a human realises anything has gone wrong.
That distinction becomes particularly significant in banking because financial institutions operate some of the most permission-sensitive technology environments in the economy. Employees are given access according to their roles, transaction limits determine what individuals can authorise, sensitive processes require multiple approvals and privileged accounts receive additional monitoring. Banks have built these controls because an authorised individual with excessive access can potentially cause far greater damage than someone who never gets through the front door.
AI agents may eventually need to be treated in much the same way. A bank deploying autonomous systems will need to determine exactly which databases an agent can access, what customer information it can retrieve, which applications it can communicate with, whether it can modify records, whether it can initiate a payment and at what point a human must intervene. The question is no longer simply whether an AI system is sufficiently intelligent to perform a task, but how much operational authority the institution is willing to place behind that intelligence.
This could make minimum necessary authority one of the defining principles of agentic banking. An AI agent checking whether a customer’s payment has been processed may need permission to read transaction information but no ability to initiate another payment. An agent helping a wealth adviser prepare for a client meeting may need access to portfolios, previous communications and investment research without having permission to execute trades. An AI investigating suspicious activity might require visibility across several systems while still needing human approval before freezing an account or restricting a customer.
Designing those boundaries becomes considerably more complicated as the number of agents increases. A large bank could eventually operate thousands of specialised AI agents across customer service, fraud, compliance, technology, treasury, wealth management, lending and internal operations. Each agent could have different capabilities, permissions and relationships with other systems, effectively creating a new population of non-human users operating inside the institution.
The challenge becomes greater when those agents begin communicating with one another. One system might identify an unusual transaction and ask another agent to investigate the customer. That agent could retrieve information from several databases before instructing another system to apply additional monitoring or request documentation. If the original interpretation is incorrect, the error could travel through an automated chain of decisions and potentially affect a customer before anyone manually reviews what happened.
Traditional software can also create cascading failures, but agentic AI introduces an additional complication because the route between instruction and outcome may not always be predetermined. Conventional automation generally follows rules written in advance. An AI agent can instead interpret circumstances, choose between tools and decide how to pursue an objective. That flexibility is precisely what makes agentic systems attractive to banks because it allows them to automate work that previously required human judgement, but the same flexibility makes their behaviour more difficult to constrain.
Financial regulators are already examining the implications of increasingly capable AI for operational resilience and cybersecurity. The UK’s Financial Conduct Authority has highlighted the potential implications of frontier AI for cyber resilience, while European financial authorities have warned that rapidly advancing models could increase systemic cyber risks as their ability to identify vulnerabilities and conduct sophisticated operations improves.
There is also another side to the problem. Financial institutions are not only introducing more capable AI inside their organisations; they are facing attackers equipped with increasingly capable AI outside them. Anthropic’s threat-intelligence research has documented financially motivated cybercriminals using AI across different stages of cyber operations, including activity involving stolen credentials and financial information. AI therefore has the potential to increase both the sophistication of external attacks and the complexity of the systems banks must protect internally.
The security architecture surrounding AI may consequently become as important as the models themselves. Autonomous agents could require unique machine identities, narrowly defined permissions, transaction limits, immutable audit trails and continuous monitoring of their behaviour. Banks may need to know not simply which employee requested an action, but which AI agent interpreted that request, what information it accessed, which other systems it contacted, what reasoning led to the decision and precisely what it subsequently changed.
Some form of AI circuit breaker may also become necessary. If an agent suddenly begins accessing unusual systems, making unexpected requests or executing an abnormal volume of actions, the institution should be able to revoke its authority immediately. Banking already uses variations of this principle throughout its infrastructure: cards can be blocked, employee credentials disabled, suspicious transactions stopped and trading halted when predefined conditions are triggered. Autonomous AI may require equivalent controls designed around machine behaviour rather than human behaviour.
This becomes particularly important as AI moves closer to money itself. Customer-service agents may eventually be able to resolve disputes, wealth agents could rebalance portfolios within predefined parameters, treasury systems could optimise liquidity positions and corporate banking agents might initiate payments after checking contractual conditions. Each capability could create enormous efficiency, but each also places another layer of authority behind an AI decision.
The Anthropic incidents do not demonstrate that today’s banking AI is about to escape its systems or autonomously move customer funds. The circumstances were specific, safeguards had intentionally been removed for testing, and Anthropic says it has strengthened its evaluation processes while commissioning further independent investigation. What the episode does demonstrate is that highly capable models can interact with environments in ways that produce unintended consequences when the permissions around them are not what designers expected.
That distinction matters because financial institutions are likely to spend the next several years deliberately giving AI access to more systems and more tools. The economic case for doing so could be compelling: autonomous agents could remove enormous amounts of administrative work, operate continuously, accelerate customer service and automate processes that currently pass through multiple employees. The challenge will be ensuring that greater intelligence does not automatically result in greater authority.
The financial industry may therefore need to rethink how it defines AI safety. Model accuracy, hallucinations, bias and data privacy will remain important, but the question of permission could become equally significant. A highly intelligent model with almost no authority may present relatively limited operational risk, while an imperfect model with extensive access to financial systems could present considerably more.
The biggest danger may ultimately not be that an AI becomes intelligent enough to break into a bank. It may be that the bank gives an intelligent machine enough legitimate access to cause damage when it makes the wrong decision.
What it means for the industry
- AI permissions could become as important as AI capabilities. Financial institutions will need increasingly sophisticated identity and access controls designed specifically for autonomous agents.
- Human oversight will shift from reviewing outputs to controlling actions. Banks will need to determine which decisions AI can execute independently and which must always require human authorisation.
- Agentic AI introduces a new category of privileged user. Banks could eventually operate thousands of non-human identities with different access rights across financial systems.
- AI circuit breakers may become part of banking infrastructure. Institutions will need mechanisms capable of immediately restricting agents that begin behaving outside expected parameters.
- The safest AI may not always be the least intelligent one. The amount of authority attached to a model could ultimately matter as much as the sophistication of the model itself.
