Prompt Injection: Hackers hide malicious instructions to manipulate enterprise AI

Evelyn Carter
18 hours ago
5 min read

Attackers are testing a new approach: instead of writing emails solely to deceive employees, they are embedding hidden instructions designed to trick AI assistants that read and process email messages.

This technique is known as indirect prompt injection. Malicious instructions can be placed in the email body, subject line, quoted email threads, attachments, or hidden formatting code. When an AI assistant summarizes or processes the message, it may mistake the attacker’s instructions for a legitimate user request.

On July 8, 2026, Microsoft announced that Microsoft Defender for Office 365 could detect prompt injection instructions hidden in incoming emails before the messages reached users’ inboxes or AI assistants. The development highlights how email is becoming an increasingly important attack surface in AI-enabled enterprise environments.

prompt-injection-through-email — *Warning: Prompt Injection through email to* manipulate enterprise AI

How does Prompt Injection in email work?

Prompt injection is a technique in which instructions are inserted into data that a language model will process. The objective is to alter the AI system’s behavior, bypass its original instructions, or cause it to perform an unintended task.

In a direct prompt injection attack, a user enters manipulative instructions directly into a chatbot. With indirect prompt injection, malicious instructions are hidden in external data sources such as:

Emails and email signatures
Word or PDF documents
Websites
Images containing text
Database records
Source code repositories or technical documentation

The attacker does not necessarily need direct access to the AI system. They only need to place malicious content in a source that the AI is likely to read.

Why is email an attractive target?

Email is one of the most valuable data repositories within an enterprise. In many organizations, AI assistants are already being used to:

Summarize long email threads
Draft replies
Search for information in mailboxes
Extract tasks and deadlines
Read attachments
Connect information from calendars, documents, and collaboration platforms

Traditionally, most email-based attacks have focused on human users. Attackers attempt to persuade employees to click malicious links, download infected files, disclose passwords, or transfer money.

Prompt injection introduces a new target: the AI systems that help employees process email.

An email may appear harmless to the recipient while containing hidden instructions intended for an AI assistant. These instructions may be concealed in invisible text, HTML code, quoted content, document metadata, or attachments.

why-is-email-an-attractive-target? — *Why is email an attractive target?*

What are the potential consequences of Prompt Injection?

The potential impact depends heavily on the permissions granted to the AI system.

A chatbot that can only answer questions presents a different level of risk from an AI agent that can read emails, access SharePoint, call APIs, create tasks, or send data to external systems.

Sensitive data leakage

If an AI system is connected to email, calendars, OneDrive, SharePoint, or document management platforms, malicious instructions may attempt to make the model search for and disclose information that the user did not explicitly request.

The exposed data could include:

Customer information
Contracts
Financial reports
Meeting notes
Internal communications
Authentication details
Confidential project documents

Manipulation of AI-Generated results

Prompt injection can cause an AI system to:

Ignore the user’s original request
Produce an inaccurate summary
Hide important warnings
Prioritize attacker-controlled information
Recommend unsafe actions
Present malicious content as a trustworthy conclusion

If employees rely on AI-generated output to make decisions, manipulated responses could affect business operations, financial processes, incident response, or customer communications.

Unintended actions

The risk increases significantly when an AI system is allowed to use external tools.

An AI agent may be authorized to send emails, create support tickets, query databases, access source code repositories, or perform tasks on cloud platforms.

How does Microsoft protect email against Prompt Injection?

According to Microsoft documentation updated on July 8, 2026, Defender for Office 365 evaluates incoming emails during the mail-filtering process.

The protection model is divided into several layers:

Protection layer	Where it operates	Primary role
Defender for Office 365	In the email flow before message delivery	Detects malicious instructions hidden in incoming emails
Microsoft 365 Copilot safety systems	When the AI model processes data	Reduces the likelihood that malicious instructions will reach or manipulate the model
Microsoft Defender XDR	Across the broader security incident	Correlates signals from email, identity, endpoints, and data

What should enterprises do to reduce the risk?

No single security filter can protect an entire AI environment. Organizations need to combine email security, access control, data governance, application security, and AI agent monitoring.

1. Identify what data AI systems can access

Organizations should create an inventory of all AI assistants, copilots, and AI agents currently in use, including tools that have not been formally approved by the IT department.

The assessment should determine:

Whether the AI can read email
Which mailboxes and folders it can access
Whether it can access SharePoint or OneDrive
Whether it can call APIs
Whether it can send data externally
Whether it stores prompts and conversation history
Whether it can trigger automated workflows

A structured AI Security Risk Assessment can help organizations identify relevant data sources, access permissions, threat scenarios, and required security controls before expanding AI adoption.

AI Security Risk Assessment: A practical framework for securing enterprise AI systems

2. Apply the Principle of Least Privilege

AI systems should receive only the permissions required to complete their assigned tasks.

For example, an assistant used to summarize emails should not automatically have permission to:

Send external messages
Read every mailbox
Access all internal documents
Retrieve sensitive credentials
Trigger financial transactions
Modify account permissions

The broader the permissions granted to an AI system, the greater the potential impact if that system is manipulated.

3. Separate External Data From System Instructions

Emails, websites, uploaded documents, and third-party content should be treated as untrusted data.

AI applications should clearly distinguish between:

System instructions
User requests
Retrieved external content
AI-generated output
Tool commands

4. Require Human Approval for Sensitive Actions

High-impact actions should require independent human confirmation.

These actions may include:

Sending external emails
Uploading or exporting data
Changing access permissions
Creating new accounts
Accessing secrets or credentials
Initiating financial transactions
Executing administrative commands

AI systems may recommend these actions, but they should not perform them automatically based solely on instructions found in an email or external document.

5. Monitor AI agents and tool-usage logs

Organizations should maintain logs that record:

Input prompts
Data sources accessed by the AI
Tools called by the system
Data returned by those tools
Actions performed by the AI
Prompt injection attempts detected or blocked

6. Test AI systems before large-scale deployment

Traditional security assessments focused on infrastructure, web applications, or APIs may not fully cover the risks introduced by language models and AI agents.

Organizations should conduct AI system penetration testing to evaluate:

Direct and indirect prompt injection
Tool and plugin permissions
Sensitive data leakage
Guardrail bypass techniques
Unauthorized workflow execution
Cross-agent manipulation
Insecure retrieval-augmented generation
Improper access control
Model output handling

Penetration Testing should be performed before production deployment and repeated whenever new data sources, tools, plugins, or automation capabilities are added.