The S in “AI Agent” Stands for Security

To design secure agents, you need a clear mental model of the complete agent landscape. We will tackle six key concepts. For each concept, we provide practical guidelines you can use to design, govern, and implement AI agents with confidence. 

AI agents do not follow a fixed execution plan. They adapt their behaviour to each request, depending on the prompt, the intermediate results the agent generates, and the tools it has access to. 

This variability is the core security challenge. You are not securing a single predictable flow. You are securing a system that can change adapt its own execution plan at runtime. 

• The data the agent touches is dynamic, not predetermined. 

• The execution plan is difficult or impossible to predict upfront. 

• A small change in wording or context can alter what gets retrieved, which tools get called, and ultimately the result. 

Below we break down 6 core facets of an AI agent that matter most for security within their autonomous world.

A prompt is usually an instruction given by a user. It can include text, images, audio and other types of files. Often, a user’s prompt comes with additional instructions and settings that help the AI agent to interpret, process and respond to a prompt correctly. 

Security Guidelines: Treat everything placed in the prompt context as potentially visible to the user. Only include information the user is authorised to access. 

Every AI agent runs on an LLM (Large Language Model). It reads the prompt, reasons over the context it receives, and generates the response based on patterns learned during training. 

That training is based on the vendor’s training data (which you likely cannot inspect), but also includes your own fine tuning, custom training, and any feedback or learning loops you enable. Every one of those choices can shape what the model might reproduce later. 

Security Guidelines: Any data used to train the LLM should be considered as exposed. Avoid using sensitive data during training, or use effective anonymisation techniques. 

Retrieval Augmented Generation (RAG) uses vector databases to enrich an agent with additional pre-processed information. RAG enables us to add knowledge to an LLM without running expensive retraining processes while avoiding the latency and overhead you often get from calling other external data sources. 

That said, enforcing access control in vector databases is still complex, especially when you need fine grained permissions and strong guarantees about what each user is allowed to retrieve. 

Security Guidelines: The easiest way to secure a knowledge store is to ensure it is only populated with non-sensitive data. Otherwise, robust access controls must be implemented. 

Tools differ from other facets because they enable AI agents not only to access data, but also to create, change and delete it. In many setups, you can run those tools either in the user’s authentication context or in the agent’s own context. 

Luckily, most tools benefit from traditional integration security mechanisms that we are already familiar with! Leveraging these is important when setting up your tools. 

But there is an extra inherent risk with agentic systems; because the agent can decide its own steps at runtime, it may take actions you did not intend when trying to fulfil a request. That can lead to surprising side effects and operational impact. 

Security Guidelines: Leverage existing security controls, and ensure users can recover from unintended actions the AI agent takes. 

Lastly, be aware that even with strong security controls in place, AI agents may guess, infer or hallucinate information. Those inferences may be either correct or incorrect, but either way they introduce a new kind of risk. 

This becomes especially sensitive when an agent infers personal or protected attributes, or when it presents speculation as fact. 

Security Guidelines: Direct the AI agent (eg. in supplementary prompts) to avoid speculation and use only trusted sources of data. This is not foolproof, so where possible, train users to identify and cope with this risk. 

AI Agents introduce some unique security challenges, but you do not have to rely on the AI agent itself to provide that security. Above you will find concrete advice that you can incorporate into your security by design program. These pragmatic controls will help build confidence in your AI integrations, which in turn bolsters your business autonomy. 

We would love to help you with your AI integration architecture! Reach out to us for support with planning, implementing and managing your next AI-enabled journey.