AI Agent Phishing: Proofpoint’s New Defense

Email security has always been a game of cat and mouse. Viruses were invented Antivirus software was invented to index known viruses and detect their presence in email attachments and URLs. As viruses become more sophisticated forms of malware, cybersecurity tools have adapted to be able to scan for and detect these new threats. Phishing has become the next arena, giving rise to new tools as well as an entirely new category of defense known as security awareness training. now, Bad guys attack AI agents to bypass existing security guardrails.

“AI assistants, co-pilots and agents dramatically expand the enterprise attack surface in ways that traditional security architectures were not designed to handle,” said Todd Thiemann, a cybersecurity analyst at research firm Omdia.

Enter a series of AI-based features for Proofpoint Prime Threat Protection that were introduced at the company’s Proofpoint Protect 2025 event in September. They thwart hackers’ efforts to subvert the actions of AI agents by scanning for potential threats before emails reach your inbox.

The traditional approach to email security

Most email security tools are designed to detect known bad signals such as suspicious links, fake domains that appear to be real, or attachments carrying malware. This approach works well against traditional phishing, spam, and known exploits. But cybercriminals are now going after many of the AI ​​assistants and AI agents that have become an integral part of the workplace.

They do this by making use of prompts (questions or commands in the form of text or code) that direct AI models and AI agents to either produce relevant responses or perform certain tasks. Increasingly, emails carry hidden, malicious claims that use invisible text or special formatting designed to trick generative AI tools like Microsoft Copilot and Google Gemini into taking unsafe actions, such as leaking data or bypassing security checks.

“Instant injections and other exploits targeting AI represent a new class of attacks that use text-based payloads that manipulate machine reasoning rather than human behavior,” Thiemann said.

Daniel Rapp, chief AI and data officer at Proofpoint, provided an example: The standard used for emails known as RFC-822 specifies the use of headers, plain text, and HTML. Not all of this is visible to the user. Attackers take advantage of this by embedding instructions in messages that are invisible to humans but fully readable by an AI agent. When AI processes text, embedded instructions are inadvertently executed. This can filter data, change system behavior, or corrupt it. Old filters that look for malware or malformed links don’t see anything wrong.

Daniel Rapp, Chief AI & Data Officer at Proofpoint.Proof Point

“In recent attacks, we’re seeing cases where the HTML version and the plain text version are completely different,” Rapp said. “The email client displays the HTML version while the invisible plain text has a quick injection that can be picked up and possibly handled by an AI system.”

There are two reasons why this strategy has proven effective: First, that is:If your AI assistant has access to your inbox, it can automatically act on email as it arrives. second, The literal nature of AI agents makes them vulnerable to phishing and other social engineering tricks, Rapp said. A person might think twice before sending money to a Nigerian bank account. An AI agent might execute a command to do this blindly.

What’s unique about Proofpoint’s approach is that the company scans emails before they reach inboxes. She’s had a lot of practice. The company scans 3.5 billion emails daily, a third of the global total. Additionally, it scans nearly 50 billion URLs and 3 billion attachments every day. This is done inline, that is, as the email travels from sender to recipient.

“We put detection capabilities directly into the delivery path, which means latency and efficiency are crucial,” Rapp said.

This necessary level of speed is achieved by specifically training smaller AI models for detection, based on examples and background knowledge of the Large Language Model (LLM). For example, OpenAI’s GPT-5 is estimated to contain up to 635 billion parameters. Wading through this amount of data for each email is not feasible. Proofpoint fine-tunes its models down to about 300 million parameters. It distills and compresses its models to achieve direct, low-latency performance without sacrificing detection accuracy. It also updates these templates every two and a half days so you can effectively interpret the intent of the message itself, not just look for indicators. This way, it detects fast hidden injections, malicious instructions, and other AI exploits before delivery.

“By stopping attacks before delivery, Proofpoint prevents user compromise and AI exploitation,” Rapp said. “Our secure email gateway can see emails and stop threats before they reach your inbox.”

In addition, Proofpoint uses a batch detection architecture. Instead of relying on a single detection mechanism, it combines hundreds of behavioral, reputational, and content signals to circumvent attack vectors that might make their way through a single method.

Artificial Intelligence is changing the security game

AI agents are deployed across the enterprise and consumers. Unfortunately, the rush to leverage the potential of AI often results in security being postponed to a later stage. The bad guys know this. They are AI enabling their cybercrime techniques and technologies to master the art of phishing for the age of AI agents.

“Security tools must evolve from detecting known bad indicators to interpreting intent for humans, machines and AI agents,” Thiemann said. “Approaches that identify malicious instructions or manipulative claims before delivery, ideally using distilled AI models for built-in low-latency protection, address a significant gap in today’s defenses.”

Proofpoint leads the pack in the role these capabilities play. Other cybersecurity vendors are expected to follow suit in the coming months. But by then, what other threat does AI hold?