How to manage risk when finance teams use agentic AI

AI agents have become a mainstream aspect of finance and accounting. While agents are a productivity revelation, they do not change accounting fundamentals. Even if an agent performs a task, a human must be responsible for the outcome, and the more judgement required, the closer a human must be to the task. This begins with appropriate controls and oversight.

Finding the right level of involvement with AI agents is a matter of balance. Too much involvement risks giving up the time and efficiency gains agents offer professionals; too little, and the risk of mistakes increases. The crux is making sure speedier workflows do not compromise quality.

“Responsible use starts with understanding the level of risk associated with the task being performed and designing appropriate controls around that task. The more complex the task, the stronger the controls need to be,” says Nick Perrett, CEO at Yarra Lane Group.

Perrett has spent four years designing and implementing agentic AI systems and is currently exploring how human judgement, robotic execution and AI agents can work together.

AI agents act

Unlike AI tools such as ChatGPT and Claude, agents can take actions. AI tools answer questions, generate content and solve problems such as planning itineraries or producing background research. While AI tools generate these outputs, agents perform the task itself. In finance, this capability must be subject to permissions and controls.

If an AI agent is used to prepare tax research, it can be restricted to drawing on approved sources and referencing specific legislation. It can also provide links to the supporting material used to arrive at its conclusions. The agent can be instructed to identify the professional standards it has considered when producing advice and document any assumptions, risks and alternative views.

The system can then prepare the output in a document, together with supporting workpapers for review.

While major accounting platforms such as MYOB are rolling out AI agents, it is also possible to build agents through platforms like ChatGPT and other emerging tools.

Praxio AI has developed an AI app that helps with tax research. “When we built our practical tax assistant, we only used legislation and guidance from the Australian Taxation Office or their documents, nothing else,” says Praxio AI adviser and content creator, William Young FCPA.

With all the enthusiasm around AI, the risk is that employees will use it without approval or the business’s knowledge. Education is key.

“Employees need to understand the opportunities and risks. From a practical perspective, I believe finance teams should provide approved, enterprise-grade AI tools rather than simply prohibiting their use,” Perrett says. “If you ignore AI, employees will seek out their own solutions using free consumer products, which can create a much greater security and compliance risk.

The objective should not be to prevent the use of AI, but to provide secure tools, clear policies and appropriate training so it can be used responsibly.”

A named human

The way agents are overseen must be decided before they are adopted. Ideally, a strong AI governance framework supports a system where every significant action an agent takes, source it draws on and assumption or decision it makes, can be reviewed. 

“The control question has changed. It is not enough to ask whether a human reviewed the output,” says David Lee Kuo Chuen, professor in the School of Business at Singapore University of Social Sciences. “The better question is whether the agent was authorized to perform the exact action, within exact limits, using approved data, code and policy, and with a named human accountable.”

The objective is to design a governance system where any failure is immediately surfaced by the system and addressed.

This means constantly monitoring agent output and maintaining clear accountability for its actions.

“That is why I would like to move from the phrase ‘human-in-the-loop’ to a clearer ‘delegation-of-authority’ model,” Lee says.

“Every finance agent capable of material actions should have an identity, human sponsor, permitted purpose, a financial authority limit, approved tools, an expiry date and a revocation mechanism. The firm should know who the agent is, what it can do, what it cannot do and who is accountable when something goes wrong.”

A clear delineation of tasks between people and agents is also required.

“An AI agent can draft a memo, prepare a first-pass analysis or recommend a journal entry. But a named human should approve material, irreversible or external-facing actions,” Lee says. Examples include regulatory filings, impairment judgements, revenue recognition judgements, payment releases and material journals.

Ordinary system access controls may not be enough if an agent can browse, click buttons and move through systems like a human user. “The agent should operate in a sandbox with domain allow lists, session logs, action-by-action approval where needed and alerts when it steps outside its mandate,” Lee says.

 

"It is not enough to ask whether a human reviewed the output. The better question is whether the agent was authorised to perform the exact action, within exact limits, using approved data, code and policy, and with a named human accountable."

 

National regulatory postures around AI are emerging. Safe AI Australia’s Guidance for AI Adoption sets out six essential practices for responsible governance and adoption by organisations operating in the country. The checklist covers accountability, understanding impacts, managing risks, sharing essential information, testing and monitoring, and maintaining human control.

In financial services, the Australian Prudential Regulation Authority’s (APRA) Letter to Industry on AI makes it clear that AI is an operational-resilience, cyber and board-oversight issue for the entities it regulates. APRA is among many regulators looking at AI, while the Australian Securities and Investments Commission has raised concerns that AI adoption may be outpacing governance frameworks.

Singapore’s Model AI Governance Framework for Agentic AI is instructive as it was written with agents in mind. The framework recommends placing limits on agents’ autonomy and introducing checkpoints where human approval is required.

“The requirement to assess and bind risks upfront means classifying AI use cases by materiality, autonomy, reversibility and data sensitivity,” Lee says. “Making humans meaningfully accountable requires assigning a CFO sponsor, controller, agent owner, data owner and internal audit responsibility.”

Above all, the legal obligations professionals need to meet remain the same, whether an agent is involved in a finance team’s processes or not, he continues. “Boards, CFOs, auditors and lawyers still need to decide who has authority, what counts as approval and how liability is allocated.”

Perrett believes understanding and documenting how AI agents reach conclusions is now an intrinsic part of the accountant’s role.

“That principle should not change because AI is involved,” Perrett says. “If an existing process needs a manager to review work before it is provided to a client, the same review process should apply to work produced by an agentic AI system.

In many cases, the supporting documentation generated by AI may be more comprehensive than the workpapers traditionally produced by employees because references, sources and reasoning can be captured automatically.” 

This means that systems need to be implemented to verify AI output in the same way as work produced by any other team member.

“If a task currently needs to be reviewed and signed off, that will be the case even if it is performed by an agent. The reviewer should assess the quality of the work, validate the conclusions, examine the supporting evidence and determine whether the output is appropriate for the circumstances,” Perrett says.

Using AI agents responsibly is not a set-and-forget exercise.

“You need to have policies and controls in place, then you should be performing continuous testing in the organisation to make sure employees are using it properly and risks are being managed,” Young says.

For accounting and finance teams that are experimenting with AI agents, one approach is to treat a new agent like a new colleague.

“Qualify it. Understand how it does the job and test it on work where you already know the right answer before you let it touch anything live,” says Bryan Sng, co-founder and COO of AI-agent accounting platform, SimpleAI.

 

"If you ignore AI, employees will seek out their own solutions using free consumer products, which can create a much greater security and compliance risk. The objective should not be to prevent the use of AI, but to provide secure tools, clear policies and appropriate training so it can be used responsibly."

 

“Check it closely in the early months, the way you would review a new hire under probation before trusting them unsupervised. Keep a periodic control check running on the system, the same as any internal audit. To effectively implement AI agents, certain parts of your processes have to change.

“Verify by reconciling the output back to an independent source like the bank statement or the source invoice,” Sng continues. “Reduce oversight only when you have evidence the agent is reliable, not just because three months have passed. Keep a regular check on the agent’s output to ensure it is working as desired.”

Spot checks and full, regular reconciliations are also essential.

“Randomly select accounting vouchers, journal entries and report items to compare AI outputs with original source data, and reconcile total accounts to check data consistency,” says Collin Jin FCPA, Deloitte China audit and assurance innovation and digital services leader, and president of CPA Australia’s East and Central China committee.

“Then, apply rule-based cross-verification. Embed accounting standards, accounting policies and logical formulas into inspection rules to spot abnormal entries, mismatched figures and unreasonable analytical conclusions automatically.

“Two employees should be responsible for checking the output, he continues. Allow junior employees to conduct primary verification, while senior financial professionals can do a secondary audit, especially with financial statements and critical analytical results.

To make sure the system is working, track model performance continuously and record error rates regularly, then retrain and optimise AI models when deviations happen.” “Use historical and simulated business data to verify AI adaptability under complex conditions. Such multilayered checks ensure the accuracy, compliance and reliability of AI-generated financial outputs.”

The bigger picture

Any fully autonomous AI or agentic system should be approached with caution.

“While AI can deliver impressive speed and accuracy, accounting and finance professionals operate in an environment where privacy, confidentiality, regulatory obligations and professional judgement are critical,” Perrett says. “AI systems can and do make mistakes. More importantly, some of the decisions professionals make involve subjective judgement where there is no single correct answer.”

Figuring out what should be done by an agent, a human or a robot — and how they work together — is the challenge. In the long term, every finance team will have access to similar AI tools. The differentiator will be how effectively businesses use that technology.

 

_{Production credit
Banner image Francesco Carta fotografo via Getty Images / Shaumiaa Vector via Getty Images}