Barracuda AI email warning reveals hidden CEO fraud risk
Modern AI assistants are helping employees work faster, but according to a new Barracuda AI email warning, the same tools can also help attackers move faster once they gain access to an employee's account. In a controlled proof-of-concept attack, Barracuda's red team demonstrated how one compromised email account could escalate into CEO account compromise and a fraudulent wire transfer worth nearly $250,000 by using legitimate business workflows.
AI Assistants reshape Email Security risks
The demonstration began with a compromised employee account. Attackers first prompted the AI assistant to create inbox rules that hid sign-in alerts and other security notifications, helping them maintain access without alerting the user.
They then used the AI assistant to analyse months of emails, attachments and calendar entries to understand the organisation's structure, identify senior executives and locate important business conversations. The research notes that the same approach could apply to other widely available AI assistants, not just Microsoft Copilot.
Copilot Exploit exposes AI-enabled email insider threat
Using the information collected, attackers instructed the AI assistant to draft a phishing email that matched the employee's writing style and reflected genuine business context. Because the message originated from a legitimate internal account, it had a much higher chance of convincing the CEO.
After compromising the CEO's account via session token theft, the attackers repeated the process to locate financial information and approval workflows. According to the proof of concept, a simple AI prompt uncovered invoices, wire transfers and a pending payment worth $247,500.
Compromised AI email accounts accelerate fraud
The Barracuda AI email warning also explains that attackers then used the CEO's legitimate mailbox to draft a request asking finance teams to change the destination bank account before the payment was approved. Since the email referenced a real transaction and matched the executive's communication style, traditional Email Security controls had little reason to identify it as suspicious.
The attackers also created forwarding rules to intercept confirmation emails and used the AI assistant to locate and remove evidence of the fraudulent activity.
According to Daniel Avulov, Senior Cybersecurity Researcher on Barracuda's red team, email history contains valuable business context, documents, attachments and organisational relationships that attackers can exploit. He noted that AI assistants can unintentionally become malicious insiders by improving both the speed and quality of an attack, adding that organisations should focus on detecting account compromise quickly by using existing security telemetry.
Read More:
Tenable Open-source AI agent powers new CyberAgents Exchange
Quantum AI isn’t enterprise-ready yet, says Gartner report
2 Min read