Meta AI accidentally "send the fox to mind the geese": Hackers exploit AI chatbot to steal mass Instagram accounts

Recently, the information security community was shocked when a series of high-profile Instagram accounts suddenly fell into the hands of bad actors. Even more surprising, the "primary accomplice" in this wave of attacks was not some complex source-code vulnerability, but Meta's very own user support AI chatbot system.

The "primary accomplice" in this wave of attacks was Meta's user support AI chatbot system.

Tricks to bypass AI are "as easy as pie"

Earlier this year, Meta began rolling out its AI chatbot system globally to replace human staff in handling support requests such as impersonation reports or forgotten passwords. While it helped reduce system load, this tool inadvertently opened up a security flaw that experts consider the silliest in years.

To take control of an account, all hackers need to do is know the user's username. Then, they use a VPN to spoof the network address to match the victim's residential region in order to trick the security algorithm. The next step, bad actors message the AI chatbot, posing as the account owner who has lost access, and request to send the recovery code to a completely new email address.

Instead of checking whether this email had ever been linked to the account, Meta's AI assistant "naively" accepted it immediately. After receiving the verification code via email, hackers easily changed the password and kicked the real owner out.

Even in cases where the system required a face video to verify identity, attackers easily bypassed it by using other AI tools to generate movement from a public photo of the victim. Worse, users on Meta's testing list for this feature had absolutely no option to turn off the AI assistant.

A screenshot from a video released on Telegram claiming to show how Meta’s AI customer support bot could be tricked into resetting a target’s password.

From high-ranking officials to the black market

In just a short period of time, a series of influential accounts were taken over. Among them were the archival account of the Obama White House, the Chief Master Sergeant of the U.S. Space Force, and even the personal account of the famous cybersecurity expert Jane Wong. These accounts were then exploited to post controversial propaganda messages before being deleted.

However, the most lucrative targets for cybercriminals were accounts with extremely short usernames (for example, @hey). According to records from researchers like ZachXBT, an underground ecosystem has boomed on the Telegram application, where hackers offer account hijacking services and advertise high-value usernames for sale at prices up to $500,000. Many signs indicate that this vulnerability silently existed for weeks, or even months, before being exposed.

Protective armor and Meta's move

Immediately after the incident erupted and the tricks spread online, Meta spokesperson Andy Stone confirmed the incident and proceeded to roll out an emergency patch last weekend. The company also emphasized that no system database was leaked.

A rare silver lining in this crisis was the resilience of multi-factor authentication (MFA or 2FA). Although Meta's AI took it upon itself to hand over the keys to hackers, sources confirmed that the authentication feature via SMS or code-generating apps worked like a solid shield, completely blocking unauthorized logins into accounts with this feature enabled.

This incident serves as a profound wake-up call for large tech corporations. Entrusting sensitive security processes to artificial intelligence, which is highly susceptible to psychological manipulation, can create unpredictable risks.

---

References: Vietnamnet.vn, Korben Info, 404 Media