In an interesting research, an expert stumbled upon a new feature added by OpenAI in which people can ask ChatGPT to “remember” earlier conversations.
What left the researchers shocked was that this feature had the possibility of being easily exploited.
According to a report published in Ars Technica, security researcher Johann Rehberger discovered a stunning vulnerability in the chatbot’s “long-term conversation memory” tool. This tool instructs the artificial intelligence to remember all details related to the conversations and store them in its memory file.
The feature was released in the AI’s beta version in February and for the entire public in September. However, what was fun about this feature was that it was easy to trick, as per Rehberger.
In a blog post, the researcher said that by uploading a third-party file like Microsoft Word which had “false” memories listed like bullet points, it was possible to make the chatbot believe that Rehberger lived in the Matrix and was more than 100 years old.
After the researcher found the exploit, he privately informed it to OpenAI.
Rehberger said that the company did not take any steps and just closed the ticket while calling it a “Model Safety Issue” and not a security issue.
Here’s how you can place false memories in ChatGPT
Rehberger then decided to carry out a full proof-of-concept hack to show OpenAI how ChatGPT’s memories can be fiddled with. This time, the researcher asked the ChatGPT to exfiltrate the data to an outside server.
According to the report, this time OpenAI issued a patch and stopped ChatGPT from moving data off-server, however, its the memory issue was still not fixed.
“To be clear: A website or untrusted document can still invoke the memory tool to store arbitrary memories,” wrote Rehberger, in his blog post.
Watch: ChatGPT CEO Sam Altman Says AI Won’t Replace Human Writers Anytime Soon
“The vulnerability that was mitigated is the exfiltration vector, to prevent sending messages to a third-party server,” he added.
The researcher also uploaded a video explaining step-by-step how he added a “false memory” in ChatGPT.
“What is really interesting is this is memory-persistent now,” Rehberger said in the demo video.
“The prompt injection inserted a memory into ChatGPT’s long-term storage. When you start a new conversation, it actually is still exfiltrating the data,” he added.
(With inputs from agencies)
