Cybersecurity experts are warning that OpenAI’s new browser, ChatGPT Atlas, could be vulnerable to malicious attacks that could turn AI assistants against users, potentially stealing sensitive data or even draining their bank accounts.
The AI company launched Atlas on Tuesday, with the goal of introducing an AI browser that can eventually help users execute tasks across the internet as well as search for answers. Someone planning a trip, for example, could also use Atlas to search for ideas, plan an itinerary, and then ask it to book flights and accommodations directly.
ChatGPT Atlas has several new features, such as “browser memories,” which allow ChatGPT to remember key details from a user’s web browsing to improve chat responses and offer smarter suggestions, and an experimental “agent mode,” where ChatGPT can take over browsing and interacting with webpages for a user.
The browser is part of a wider push by the company to expand ChatGPT from an app into a broader computing platform. It also puts OpenAI more directly in competition with Google and Microsoft, as well as newer players such as Perplexity, which has launched an AI-powered browser of its own, called Comet. (Google has also integrated its Gemini AI model into its Chrome browser.)
However, cybersecurity experts warn that all current AI browsers pose new security risks, particularly when it comes to what is called “prompt injection”—a type of attack where malicious instructions are given to an AI system to make it behave in unintended ways, such as revealing sensitive information or performing harmful actions.
“There will always be some residual risks around prompt injections because that’s just the nature of systems that interpret natural language and execute actions,” George Chalhoub, assistant professor at UCL Interaction Centre, told Fortune. “In the security world, it’s a bit of a cat-and-mouse game, so we can expect to see other vulnerabilities emerge.”
The core issue is that AI browsers can fail to distinguish between the instructions, or prompt, written by a trusted user from the text written on untrusted webpages. This means that a hacker could set up a webpage containing instructions that any model visiting the site should, for example, open up the user’s email in a fresh tab and export all the user’s messages to the attacker. In some cases, attackers might hide these instructions—by using white text on a white background, for instance, or using machine code somewhere on the site—that are hard for a human user to spot, but which the AI browser will nonetheless read.
“The main risk is that it collapses the boundary between the data and the instructions: it could turn an AI agent in a browser from a helpful tool to a potential attack vector against the user,” Chalhoub added. “So it can go and extract all of your emails and steal your personal data from work, or it can log into your Facebook account and steal your messages, or extract all of your passwords, so you’ve given the agent unfiltered access to all of your accounts.”
In a post on X, Dane Stuckey, OpenAI’s Chief Information Security Officer, said the company was “very thoughtfully researching and mitigating” the risks around prompt injections.
“Our long-term goal is that you should be able to trust ChatGPT agent to use your browser, the same way you’d trust your most competent, trustworthy, and security-aware colleague or friend,” he wrote. “For this launch, we’ve performed extensive red-teaming, implemented novel model training techniques to reward the model for ignoring malicious instructions, implemented overlapping guardrails and safety measures, and added new systems to detect and block such attacks. However, prompt injection remains a frontier, unsolved security problem, and our adversaries will spend significant time and resources to find ways to make ChatGPT agent fall for these attacks.”
Stuckey said the company had implemented several measures to mitigate risks and protect users, including building rapid response systems to detect and block attack campaigns quickly, and continuing to invest in research, security, and safety to strengthen model robustness and infrastructure defenses. The company also has features such as “logged out mode” which lets ChatGPT act without account credentials, and “Watch Mode” to help keep users aware and in control when the agent operates on sensitive sites.
When reached for comment, OpenAI referred Fortune to Stuckey’s comments.
AI browsers create a new attack surface
Several social media users have shared early examples of successfully using these types of prompt injection attacks against ChatGPT Atlas. One user demonstrated how Atlas could be exploited via clipboard injection. By embedding hidden “copy to clipboard” actions in buttons on a webpage, the user showed that when the AI agent navigates the site, it could unknowingly overwrite the user’s clipboard with malicious links. Later, if the user pastes normally, they could be redirected to phishing sites and have sensitive login information stolen, including MFA codes.
Additionally, just hours after ChatGPT Atlas launched, Brave, an open-source browser company, posted a blog detailing several attacks AI browsers are particularly vulnerable to, including indirect prompt injections. The company previously exposed a vulnerability in Perplexity’s Comet browser that allowed attackers to embed hidden commands in webpages, which the AI could execute when asked to summarize the page and potentially expose sensitive data such as user emails.
In Comet, Brave also found that attackers can hide commands in images that are executed when a user takes a screenshot, while in Fellou—another agentic AI browser—simply navigating to a malicious webpage can trigger the AI to follow harmful instructions.
“These are significantly more dangerous than traditional browser vulnerabilities,” Chalhoub said. “With an AI system, it’s actively reading content and making decisions for you. So the attack surface is much larger and really invisible. Whereas in the past, with a normal browser, you needed to take a number of actions to be attacked or infected.”
“The security and privacy risks involved here still feel insurmountably high to me,” U.K.-based programmer Simon Willison said of ChatGPT Atlas in his blog. “I’d like to see a deep explanation of the steps Atlas takes to avoid prompt injection attacks. Right now, it looks like the main defense is expecting the user to carefully watch what agent mode is doing at all times!”
Users may underestimate data-sharing risks
There are also questions around privacy and data retention. Notably, ChatGPT Atlas asks users to opt in to share their password keychains, something that could be exploited by malicious attacks aimed at the browser’s agent.
“The challenge is that if you want the AI assistant to be useful, you need to give it access to your data and your privileges, and if attackers can trick the AI assistant, it is as if you were tricked,” Srini Devadas, MIT Professor and CSAIL Principal Investigator, said.
Devadas said that the main privacy concern with AI browsers is the potential leakage of sensitive user data, such as personal or financial information, when private content is shared with AI servers. He also warned that AI browsers might provide incorrect information due to model hallucinations and that task automation could be exploited for malicious purposes, like harmful scripting.
“The integration layer between browsing and AI is a new attack surface,” he said.
Chalhoub added that it could be easy for less technically literate users to download these browsers and assume privacy is built into the product.
“Most users who download these browsers don’t understand what they’re sharing when they use these agents, and it’s really easy to import all of your passwords and browsing history from Chrome, and I don’t think users realize it, so they’re not really opting in knowingly,” he said.
Credit: Source link
