Your social media posts could fuel the next AI scam

by Brian Lopez, University of Texas at Arlington

edited by Swati Mestri, reviewed by Andrew Zinin

This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

Add to Preferred Sources UTA study reveals how AI uses personal details to create targeted phishing emails. Credit: University of Texas at Arlington Picture this: You post photos online from a trip to Brazil. The next day, an email arrives from the hotel where you stayed, thanking you for your visit and offering a special discount on your next stay. The email references your recent trip, the hotel's name looks right and the link looks legitimate.

But the hotel never sent it. An AI model generated the email after scraping details from your vacation posts.

Researchers at The University of Texas at Arlington have shown that widely available AI tools can gather details from people's public social media posts and use them to quickly generate highly personalized scam emails targeting large numbers of people, a tactic known as context-aware spear phishing. The emails are designed to trick recipients into clicking malicious links and handing over sensitive information, such as passwords or credit card numbers.

"Anyone can be the attacker, and that makes it scarier," said Shirin Nilizadeh, an associate professor in UTA's Department of Computer Science and Engineering who led the study. "With these AI models, they don't need to do that much. They can just tell them what they want them to do."

Spear phishing isn't new. Nilizadeh said most targeted scams have relied on static details about a person, such as their job title or employer. This study examines a more sophisticated type of attack, one that draws on details of people's everyday lives shared online.

"Now scammers can use people's social interactions and their daily lives and use it in their emails," she said. "It's more personalized, and it can make people more vulnerable."

Elham Pourabbas Vafa, a graduate student in computer science and the co-author of the study's findings, said the AI tools pulled details from people's recent social media posts, including photos, and used that information to create personalized emails. The researchers tested emails based on travel, events people were interested in, such as concerts, and even concerns they had shared online. They also tailored subject lines, sender names and links to each recipient.

"The person who receives this kind of spear-phishing email is going to find it very credible because it is referencing details specific to their recent experiences," Pourabbas Vafa said.

In a user study, participants gave the AI-generated emails high marks for quality, underscoring what researchers say makes these attacks so dangerous.

AI companies already use safety filters designed to block harmful requests. But the researchers found that these filters largely failed to detect requests to create personalized scam emails.

The team tested several widely used safety tools, including Llama Guard, ShieldGemma and WildGuard. With their default settings, the tools detected just 0% to 19% of the attack requests.

The researchers then proposed ways to stop the emails from being generated in the first place. When the safety tools were given specific instructions to recognize these attacks, their detection rates rose to as high as 99%.

"We are urging AI companies to take these kinds of attacks seriously," Nilizadeh said. "They need to put safeguards in place to prevent their tools from being misused to create highly targeted social engineering attacks."

The researchers omitted the exact instructions used to produce the scam emails from the public version of their paper to prevent others from misusing them.

The study has been accepted for presentation at the Network and Distributed System Security Symposium in Seoul, South Korea, in March 2027. Pourabbas Vafa also presented the research at the Texas Security Workshop at The University of Texas at Dallas.

Swati Mestri holds a bachelor's degree in Electronics Engineering and has worked as a content editor since 2019. She has experience editing research documents across technology, health care, and materials science, and has a particular interest in technology and space. Full profile →

Master's in physics with research experience. Long-time science news enthusiast. Plays key role in Science X's editorial success. Full profile →