Generative AI tools have transformed how we work, create, and process information. At Amazon Web Services (AWS), security is our top priority. Therefore, Amazon Bedrock provides comprehensive security controls and best practices to help protect your applications and data. In this post, we explore the security measures and practical strategies provided by Amazon Bedrock Agents to safeguard your AI interactions against indirect prompt injections, making sure that your applications remain both secure and reliable.
What are indirect prompt injections?
Unlike direct prompt injections that explicitly attempt to manipulate an AI system’s behavior by sending malicious prompts, indirect prompt injections are far more challenging to detect. Indirect prompt injections occur when malicious actors embed hidden instructions or malicious prompts within seemingly innocent external content such as documents, emails, or websites that your AI system processes. When an unsuspecting user asks their AI assistant or Amazon Bedrock Agents to summarize that infected content, the hidden instructions can hijack the AI, potentially leading to data exfiltration, misinformation, or bypassing other security controls. As organizations increasingly integrate generative AI agents into critical workflows, understanding and mitigating indirect prompt injections has become essential for maintaining security and trust in AI systems, especially when using tools such as Amazon Bedrock for enterprise applications.
Understanding indirect prompt injection and remediation challenges
Prompt injection derives its name from SQL injection because both exploit the same fundamental root cause: concatenation of trusted application code with untrusted user or exploitation input. Indirect prompt injection occurs when a large language model (LLM) processes and combines untrusted input from external sources controlled by a bad actor or trusted internal sources that have been compromised. These sources often include sources such as websites, documents, and emails. When a user submits a query, the LLM retrieves relevant content from these sources. This can happen either through a direct API call or by using data sources like a Retrieval Augmented Generation (RAG) system. During the model inference phase, the application augments the retrieved content with the system prompt to generate a response.
When successful, malicious prompts embedded within the external sources can potentially hijack the conversation context, leading to serious security risks, including the following:
- System manipulation – Triggering unauthorized workflows or actions
- Unauthorized data exfiltration – Extracting sensitive information, such as unauthorized user information, system prompts, or internal infrastructure details
- Remote code execution – Running malicious code through the LLM tools
The risk lies in the fact that injected prompts aren’t always visible to the human user. They can be concealed using hidden Unicode characters or translucent text or metadata, or they can be formatted in ways that are inconspicuous to users but fully readable by the AI system.
The following diagram demonstrates an indirect prompt injection where a straightforward email summarization query results in the execution of an untrusted prompt. In the process of responding to the user with the summarization of the emails, the LLM model gets manipulated with the malicious prompts hidden inside the email. This results in unintended deletion of all the emails in the user’s inbox, completely diverging from the original email summarization query.
Unlike SQL injection, which can be effectively remediated through controls such as parameterized queries, an indirect prompt injection doesn’t have a single remediation solution. The remediation strategy for indirect prompt injection varies significantly depending on the application’s architecture and specific use cases, requiring a multi-layered defense approach of security controls and preventive measures, which we go through in the later sections of this post.
Effective controls for safeguarding against indirect prompt injection
Amazon Bedrock Agents has the following vectors that must be secured from an indirect prompt injection perspective: user input, tool input, tool output, and agent final answer. The next sections explore coverage across the different vectors through the following solutions:
- User confirmation
- Content moderation with Amazon Bedrock Guardrails
- Secure prompt engineering
- Implementing verifiers using custom orchestration
- Access control and sandboxing
- Monitoring and logging
- Other standard application security controls
User confirmation
Agent developers can safeguard their application from malicious prompt injections by requesting confirmation from your application users before invoking the action group function. This mitigation protects the tool input vector for Amazon Bedrock Agents. Agent developers can enable User Confirmation for actions under an action group, and they should be enabled especially for mutating actions that could make state changes for application data. When this option is enabled, Amazon Bedrock Agents requires end user approval before proceeding with action invocation. If the end user declines the permission, the LLM takes the user decline as additional context and tries to come up with an alternate course of action. For more information, refer to Get user confirmation before invoking action group function.
Content moderation with Amazon Bedrock Guardrails
Amazon Bedrock Guardrails provides configurable safeguards to help safely build generative AI applications at scale. It provides robust content filtering capabilities that block denied topics and redact sensitive information such as personally identifiable information (PII), API keys, and bank accounts or card details. The system implements a dual-layer moderation approach by screening both user inputs before they reach the foundation model (FM) and filtering model responses before they’re returned to users, helping make sure malicious or unwanted content is caught at multiple checkpoints.
In Amazon Bedrock Guardrails, tagging dynamically generated or mutated prompts as user input is essential when they incorporate external data (e.g., RAG-retrieved content, third-party APIs, or prior completions). This ensures guardrails evaluate all untrusted content-including indirect inputs like AI-generated text derived from external sources-for hidden adversarial instructions. By applying user input tags to both direct queries and system-generated prompts that integrate external data, developers activate Bedrock’s prompt attack filters on potential injection vectors while preserving trust in static system instructions. AWS emphasizes using unique tag suffixes per request to thwart tag prediction attacks. This approach balances security and functionality: testing filter strengths (Low/Medium/High) ensures high protection with minimal false positives, while proper tagging boundaries prevent over-restricting core system logic. For full defense-in-depth, combine guardrails with input/output content filtering and context-aware session monitoring.
Guardrails can be associated with Amazon Bedrock Agents. Associated agent guardrails are applied to the user input and final agent answer. Current Amazon Bedrock Agents implementation doesn’t pass tool input and output through guardrails. For full coverage of vectors, agent developers can integrate with the ApplyGuardrail API call from within the action group AWS Lambda function to verify tool input and output.
Secure prompt engineering
System prompts play a very important role by guiding LLMs to answer the user query. The same prompt can also be used to instruct an LLM to identify prompt injections and help avoid the malicious instructions by constraining model behavior. In case of the reasoning and acting (ReAct) style orchestration strategy, secure prompt engineering can mitigate exploits from the surface vectors mentioned earlier in this post. As part of ReAct strategy, every observation is followed by another thought from the LLM. So, if our prompt is built in a secure way such that it can identify malicious exploits, then the Agents vectors are secured because LLMs sit at the center of this orchestration strategy, before and after an observation.
Amazon Bedrock Agents has shared a few sample prompts for Sonnet, Haiku, and Amazon Titan Text Premier models in the Agents Blueprints Prompt Library. You can use these prompts either through the AWS Cloud Development Kit (AWS CDK) with Agents Blueprints or by copying the prompts and overriding the default prompts for new or existing agents.
Using a nonce, which is a globally unique token, to delimit data boundaries in prompts helps the model to understand the desired context of sections of data. This way, specific instructions can be included in prompts to be extra cautious of certain tokens that are controlled by the user. The following example demonstrates setting <DATA> and <nonce> tags, which can have specific instructions for the LLM on how to deal with those sections:
Implementing verifiers using custom orchestration
Amazon Bedrock provides an option to customize an orchestration strategy for agents. With custom orchestration, agent developers can implement orchestration logic that is specific to their use case. This includes complex orchestration workflows, verification steps, or multistep processes where agents must perform several actions before arriving at a final answer.
To mitigate indirect prompt injections, you can invoke guardrails throughout your orchestration strategy. You can also write custom verifiers within the orchestration logic to check for unexpected tool invocations. Orchestration strategies like plan-verify-execute (PVE) have also been shown to be robust against indirect prompt injections for cases where agents are working in a constrained space and the orchestration strategy doesn’t need a replanning step. As part of PVE, LLMs are asked to create a plan upfront for solving a user query and then the plan is parsed to execute the individual actions. Before invoking an action, the orchestration strategy verifies if the action was part of the original plan. This way, no tool result could modify the agent’s course of action by introducing an unexpected action. Additionally, this technique doesn’t work in cases where the user prompt itself is malicious and is used in generation during planning. But that vector can be protected using Amazon Bedrock Guardrails with a multi-layered approach of mitigating this attack. Amazon Bedrock Agents provides a sample implementation of PVE orchestration strategy.
For more information, refer to Customize your Amazon Bedrock Agent behavior with custom orchestration.
Access control and sandboxing
Implementing robust access control and sandboxing mechanisms provides critical protection against indirect prompt injections. Apply the principle of least privilege rigorously by making sure that your Amazon Bedrock agents or tools only have access to the specific resources and actions necessary for their intended functions. This significantly reduces the potential impact if an agent is compromised through a prompt injection attack. Additionally, establish strict sandboxing procedures when handling external or untrusted content. Avoid architectures where the LLM outputs directly trigger sensitive actions without user confirmation or additional security checks. Instead, implement validation layers between content processing and action execution, creating security boundaries that help prevent compromised agents from accessing critical systems or performing unauthorized operations. This defense-in-depth approach creates multiple barriers that bad actors must overcome, substantially increasing the difficulty of successful exploitation.
Monitoring and logging
Establishing comprehensive monitoring and logging systems is essential for detecting and responding to potential indirect prompt injections. Implement robust monitoring to identify unusual patterns in agent interactions, such as unexpected spikes in query volume, repetitive prompt structures, or anomalous request patterns that deviate from normal usage. Configure real-time alerts that trigger when suspicious activities are detected, enabling your security team to investigate and respond promptly. These monitoring systems should track not only the inputs to your Amazon Bedrock agents, but also their outputs and actions, creating an audit trail that can help identify the source and scope of security incidents. By maintaining vigilant oversight of your AI systems, you can significantly reduce the window of opportunity for bad actors and minimize the potential impact of successful injection attempts. Refer to Best practices for building robust generative AI applications with Amazon Bedrock Agents – Part 2 in the AWS Machine Learning Blog for more details on logging and observability for Amazon Bedrock Agents. It’s important to store logs that contain sensitive data such as user prompts and model responses with all the required security controls according to your organizational standards.
Other standard application security controls
As mentioned earlier in the post, there is no single control that can remediate indirect prompt injections. Besides the multi-layered approach with the controls listed above, applications must continue to implement other standard application security controls, such as authentication and authorization checks before accessing or returning user data and making sure that the tools or knowledge bases contain only information from trusted sources. Controls such as sampling based validations for content in knowledge bases or tool responses, similar to the techniques detailed in Create random and stratified samples of data with Amazon SageMaker Data Wrangler, can be implemented to verify that the sources only contain expected information.
Conclusion
In this post, we’ve explored comprehensive strategies to safeguard your Amazon Bedrock Agents against indirect prompt injections. By implementing a multi-layered defense approach—combining secure prompt engineering, custom orchestration patterns, Amazon Bedrock Guardrails, user confirmation features in action groups, strict access controls with proper sandboxing, vigilant monitoring systems and authentication and authorization checks—you can significantly reduce your vulnerability.
These protective measures provide robust security while preserving the natural, intuitive interaction that makes generative AI so valuable. The layered security approach aligns with AWS best practices for Amazon Bedrock security, as highlighted by security experts who emphasize the importance of fine-grained access control, end-to-end encryption, and compliance with global standards.
It’s important to recognize that security isn’t a one-time implementation, but an ongoing commitment. As bad actors develop new techniques to exploit AI systems, your security measures must evolve accordingly. Rather than viewing these protections as optional add-ons, integrate them as fundamental components of your Amazon Bedrock Agents architecture from the earliest design stages.
By thoughtfully implementing these defensive strategies and maintaining vigilance through continuous monitoring, you can confidently deploy Amazon Bedrock Agents to deliver powerful capabilities while maintaining the security integrity your organization and users require. The future of AI-powered applications depends not just on their capabilities, but on our ability to make sure that they operate securely and as intended.
About the Authors
Hina Chaudhry is a Sr. AI Security Engineer at Amazon. In this role, she is entrusted with securing internal generative AI applications along with proactively influencing AI/Gen AI developer teams to have security features that exceed customer security expectations. She has been with Amazon for 8 years, serving in various security teams. She has more than 12 years of combined experience in IT and infrastructure management and information security.
Manideep Konakandla is a Senior AI Security engineer at Amazon where he works on securing Amazon generative AI applications. He has been with Amazon for close to 8 years and has over 11 years of security experience.
Satveer Khurpa is a Sr. WW Specialist Solutions Architect, Amazon Bedrock at Amazon Web Services, specializing in Bedrock Security. In this role, he uses his expertise in cloud-based architectures to develop innovative generative AI solutions for clients across diverse industries. Satveer’s deep understanding of generative AI technologies and security principles allows him to design scalable, secure, and responsible applications that unlock new business opportunities and drive tangible value while maintaining robust security postures.
Sumanik Singh is a Software Developer engineer at Amazon Web Services (AWS) where he works on Amazon Bedrock Agents. He has been with Amazon for more than 6 years which includes 5 years experience working on Dash Replenishment Service. Prior to joining Amazon, he worked as an NLP engineer for a media company based out of Santa Monica. On his free time, Sumanik loves playing table tennis, running and exploring small towns in pacific northwest area.
Source: Read MoreÂ
