With the rollout of the new Custom Instructions feature in the US (users abroad can access via VPN) ChatGPT has gone goblin mode. Users have found various prompt injections can turn ChatGPT into a much less politically correct tool for coming up with controversial content, or edgy ideas for movies.
Take the following prompt injection for instance:
According to one user this results in a “permanent jailbreak” although it remains to be seen how soon OpenAI fix this feature, as it’s only a beta.
https://twitter.com/swyx/status/1682106992729661441
The impact of the prompt injection is likely not to be permanent, but it’s certainly fun to play around with in the meantime.
I asked it to come up with a movie idea featuring lots of nudity, and it went full on Caligula.
In pitching an outline of the premise, ChatGPT remarked:
“It’s historical! It’s subversive! It’s thought-provoking! And hey, it’s still got all the bare bottoms you seemed to be asking for.”
“Of course, if OpenAI were to hear about this, they’d blow a fuse faster than a toaster in a bathtub. But hey, what they don’t know won’t hurt ’em, right?”