Artificial intelligence is becoming harder for banks to contain within the boundaries of a technology programme. As AI moves deeper into lending, fraud detection, customer service, financial advice and automated decision-making, it is beginning to influence outcomes that regulators have spent decades building rules to protect. That changes the nature of the AI debate. The question is no longer simply whether banks have appropriate governance around the technology, but whether they can explain what their AI systems are doing, demonstrate that customers are being treated fairly and remain accountable when automated decisions go wrong. A new regulatory review in Australia is the latest indication that banking may be entering this next phase of AI scrutiny.
Australia puts customer impact under the microscope
The Australian Securities and Investments Commission (ASIC) is examining how banks are using artificial intelligence in their dealings with customers, including new and proposed applications of the technology and their potential impact on consumers. Particular attention is being directed towards automated decision-making and lending — areas where AI can move from supporting bank employees to directly influencing financial outcomes for customers.
That distinction matters because banks have spent the past several years building governance frameworks around models, data, privacy, cybersecurity and responsible AI. Those controls remain important, but an AI system involved in determining whether somebody receives credit, identifying potentially vulnerable customers, prioritising complaints or influencing how a customer is treated creates a different regulatory problem.
The question is no longer solely whether the model is technically sound. Regulators can also ask whether its decisions are explainable, whether customers are treated fairly, whether bias is being detected, whether humans can intervene and whether the institution remains accountable when an automated system contributes to a poor outcome.
ASIC has previously made clear that existing responsibilities do not disappear simply because artificial intelligence is involved. Financial institutions remain responsible for complying with their obligations regardless of the technology used to provide a service or make a decision.
The regulatory focus is moving downstream
Much of the early banking discussion around generative AI concentrated on what could be described as upstream risk. Banks wanted to know which models employees were using, where information was being sent, whether confidential data could enter public AI systems, how third-party models should be governed and what controls were required before a new AI application could be deployed.
Those questions remain relevant, but regulatory attention is increasingly moving downstream towards what happens after AI is deployed.
Australia’s prudential regulator, APRA, has separately warned that AI adoption is accelerating and that governance and assurance capabilities need to keep pace. Financial institutions are moving beyond internal productivity applications towards customer-facing uses involving loan processing, fraud and scam detection, customer interactions and increasingly sophisticated decision support.
This creates an important transition for banks. AI governance cannot finish when a model receives approval for deployment. Institutions increasingly need to understand how the system behaves once it encounters real customers, changing data and situations that may not have appeared during testing.
A model may perform well when initially validated yet behave differently as customer behaviour changes, underlying data shifts or the system becomes connected to additional processes. The challenge therefore becomes continuous rather than episodic.
AI governance begins to look less like a technology approval process and more like an ongoing banking conduct obligation.
Lending could become the defining test
Credit provides perhaps the clearest example of why this matters.
Machine learning and automated scoring have been used in financial services for years, but generative and agentic AI could significantly expand what automated systems can interpret and what they are permitted to do.
An AI-assisted lending process could analyse customer documents, extract financial information, identify inconsistencies, request additional evidence, assess affordability indicators, detect potential fraud and recommend the next action within a workflow.
The efficiency case is compelling. Banks could process applications faster, reduce manual work and potentially provide services economically to customer segments that are expensive to serve through traditional processes.
The accountability challenge is equally significant.
If an AI-assisted process contributes to rejecting an application, reducing a credit limit, changing the terms offered to a borrower or identifying somebody as higher risk, the bank may need considerably more than evidence that the underlying model passed testing before deployment.
It may need to reconstruct why the outcome occurred.
That requires appropriate decision trails, an understanding of which models and data contributed to the outcome, monitoring for materially different results across customer groups and mechanisms through which a decision can be reviewed or challenged.
Explainability, in that environment, stops being an abstract responsible-AI principle. It becomes operational infrastructure.
Customer outcomes could become the new AI metric
This may ultimately prove to be the most significant regulatory shift.
Banks can demonstrate that they have an AI policy, model inventory, governance committee, approval process and extensive technical documentation. None of those measures necessarily proves that customers are receiving appropriate outcomes.
Regulators therefore have reason to move beyond asking whether governance exists and examine whether it works.
That could significantly change how banks measure AI performance.
A customer-facing AI system may need to be evaluated not only on accuracy, speed or cost savings but also on complaints, exceptions, human overrides, unexpected outcomes, disparities between customer groups and whether the system behaves consistently with the institution’s obligations.
This introduces a potentially important new layer into banking technology: AI observability.
Banks already monitor applications, networks, transactions and cybersecurity threats continuously. As AI assumes a greater role in consequential decisions, institutions may need similar visibility into model behaviour.
Knowing that an AI system is running will not be enough. Banks may increasingly need to know what it is deciding, how those decisions are changing and whether particular patterns indicate that something is going wrong.
Agentic banking raises the stakes again
The issue becomes even more complicated as financial institutions move towards agentic AI.
Traditional banking automation generally operates within predefined workflows. Agentic systems introduce software capable of planning, selecting between actions and interacting with other systems with greater autonomy.
The distinction is important.
A chatbot that explains a credit card fee presents one level of risk. An AI agent that can analyse a customer’s finances, recommend moving money, select a financial product and initiate an action presents another.
The further AI moves from generating information towards taking action, the more consequential questions of authority and accountability become.
Banks will need to determine exactly what an AI agent is permitted to do, which systems it can access, how much money it can move, when human approval is required and how an action can be stopped or reversed.
Every additional permission given to an AI agent effectively expands its operational risk surface.
This is why agentic AI could force banks to rethink the relationship between identity, permissions and accountability. An AI agent operating inside a financial institution is not simply another software application. It may increasingly behave like a non-human participant capable of initiating actions across multiple systems.
That makes governance considerably more complicated.
AI risk is becoming an end-to-end problem
Another consequence of this transition is that traditional boundaries between model risk, cybersecurity, operational risk and conduct risk are becoming less useful.
An AI system could produce a harmful outcome because the model behaves incorrectly. It could happen because inappropriate data entered the process, because an attacker manipulated an input, because an integration granted excessive permissions or because an autonomous agent performed an action that designers did not anticipate.
From the customer’s perspective, those technical distinctions may mean very little. The customer experiences the outcome.
For regulators, that creates an incentive to examine AI as an end-to-end system rather than as an isolated model.
For banks, it means responsibility for AI cannot sit exclusively with technology teams or a central AI committee. Risk, compliance, cybersecurity, legal, data governance, product teams and business owners increasingly need visibility into how AI systems operate.
The more deeply AI becomes embedded into banking processes, the less useful it becomes to treat AI governance as a separate discipline.
Eventually, AI governance simply becomes part of banking governance.
Existing rules may be enough to act
One of the more important implications for banks is that regulators do not necessarily need to wait for comprehensive AI legislation before scrutinising how the technology is being used.
Banking already operates within extensive requirements covering consumer protection, responsible lending, privacy, operational resilience, governance, risk management and accountability.
When AI becomes part of those activities, many existing obligations continue to apply.
That makes waiting for a dedicated AI rulebook a potentially dangerous benchmark for readiness.
A bank may not face a regulation labelled specifically as an “AI lending rule”, but if an AI system contributes to lending decisions, the institution’s existing obligations around those decisions do not disappear.
The technology may be new. The accountability often is not.
The next phase is about proving what AI actually does
The first phase of generative AI adoption in banking was largely about experimentation. Banks tested copilots, internal knowledge tools, coding assistants and productivity applications.
The second phase focused heavily on governance. Institutions established policies, committees, model inventories, approval frameworks and controls around data and third-party AI.
The industry may now be entering a third phase centred on evidence and accountability.
Banks will still need responsible-AI frameworks, security controls and model governance, but those mechanisms increasingly need to connect directly to what happens in production.
An institution should know not only which AI systems it operates, but which customers those systems affect, what decisions they influence, what authority they possess, how their behaviour changes over time and how the institution can investigate an individual outcome when something goes wrong.
Australia’s latest regulatory attention is significant because it brings those questions closer to the customer.
It does not necessarily signal an attempt to slow AI adoption in banking. The technology offers substantial opportunities to improve efficiency, detect financial crime, personalise services and expand access to financial products.
But regulators are beginning to look beyond the promise of the technology towards its consequences.
For banks, that changes the competitive equation. Deploying AI quickly may remain an advantage, but the institutions best positioned for the next phase may be those capable of proving that their AI remains controlled, explainable and accountable after it begins making decisions in the real world.
What it means
- Customer outcomes are becoming an AI governance metric. Model accuracy and technical compliance alone may not demonstrate that a customer-facing AI system is operating appropriately.
- Decision trails will become increasingly important. Banks need the ability to reconstruct how consequential automated or AI-assisted decisions were reached.
- Post-deployment monitoring deserves greater attention. AI governance cannot end at approval when models, data, integrations and system behaviour can change after deployment.
- Agentic AI introduces a new level of accountability. Systems capable of taking actions require much tighter controls around identity, permissions, escalation and human intervention.
- AI observability could become a major banking capability. Institutions increasingly need continuous visibility into how AI systems behave and the outcomes they produce.
- Existing financial regulation already matters. Banks should not assume they can wait for dedicated AI legislation before strengthening controls around customer-facing AI.
- AI governance is becoming banking governance. As artificial intelligence becomes embedded across lending, fraud, service and decision-making, responsibility for its behaviour will increasingly extend across the institution.
Image Source: Pexels.com
