Manage the risks posed by AI agents

The ability of AI agents to perform multi-stage tasks is based on so-called iterative reasoning. Unlike a regular language model, which provides a single answer, an agent thinks cyclically:

• it plans the next step 
• it performs the next step 
• it evaluates the result 
• it corrects its functioning to achieve its ultimate goal.

For an AI agent to perform multi-stage tasks, it needs memory:

• Short-term memory: the ability to maintain the context of the task during the process.
• Long-term memory: the ability to use existing data or external databases, often with the help of RAG technology.

In businesses and the public sector, customer cases typically have multiple stages and involve different datasets and steps. Particularly in the public sector, cases also require legality checks. The possibilities of agentic AI to automate demanding processes entirely has created hopes for a significant increase in efficiency and productivity.

Teams formed by AI agents can, in principle, perform very demanding and complex processes. For example, agentic AI can – at least in theory – organise a relatively large music event with different stages, such as

• selecting and inviting artists
• contract negotiations
• payments to performing artists
• selecting and booking performance venues
• ticket sales.

However, agents functioning as teams or flocks does not happen on its own. Instead, the teams need to be “orchestrated”. This means implementing the interoperability of team agents. Each agent in a team has its own task.

The operations of the agents in relation to each other must therefore be clearly defined

• for stages in the process chart: what happens in each stage
• for APIs between agents: what data do agents exchange and under which protocol.

A human-in-the-loop model should be used for critical processes. In the model, an AI agent prepares the work, but it is not allowed to perform the work until a human has given final approval. This is a key risk management method.

It is important to ensure the competence – for example, through additional training – of those working with AI agents. Working with agents is not for amateurs in the world of 007 or that of AI agents, and mistakes can be expensive.

For the purposes of learning how to use agents, and agentic AI in particular, it is a good idea to set up an isolated test environment or so-called agent sandbox where mistakes can be safely made.

When moving to production, it is better to be safe than sorry. Initially, agents should be granted only the most essential authorisations, and their work should be restricted to tasks where they cannot process and share sensitive information. As the organisation’s competence increases, this use can be gradually expanded.

Just as in all AI use, system training data plays a critical role. If agents have been trained using weak-quality data – or if they have access to such data – the consequences are inevitably negative. Therefore, data quality controls are essential for agents as well.

The output of agents must be monitored and evaluated actively, and corrective action must be taken where necessary, just as in other artificial intelligence systems. There must also be up-to-date and suitable equipment for monitoring.

AI agents are not just a concern for IT and security departments. Issues related to the liability of agents also concern an organisation’s lawyers. An organisation’s management, on the other hand, must define the role of agents in the organisation’s strategy.

When ground rules are prepared in the organisation for AI agents – and, more broadly, for agentic AI – personnel working with agents must be involved. Personnel should be kept well-informed in any case. Everyone should understand and accept the rules related to agents and the instructions derived from those rules.

Not doing this causes a risk of “shadow agent activity”. This means individuals or entities within the organisation operating according to their own rules, which may cause unforeseen risks.