Concept
Jailbreaking
The practice of crafting inputs designed to make an AI system bypass its intended guidelines or safety constraints.
Jailbreaking
4 visible of 4 connections
Related Concepts2
Related Frameworks1
Related Practices13
Ask about this concept
Transcript mentions
… And AI security is prompt injection and jailbreaking and indirect prompt injection and AI red teaming and some major problems I've found with the AI security industry that I think need to be talked more about.
… And so they are kind of proposed as a defense measure against prompt injection and jailbreaking. And what I have found through running these events is that they are terribly, terribly insecure and frankly, they don't work.
Jailbreaking is like when it's just you and the model. So maybe you log into ChatGPT and you put in this super long malicious prompt and you trick it into saying something terrible, outputting instructions on how to build a bomb, something…
Concept
Jailbreaking
The practice of crafting inputs designed to make an AI system bypass its intended guidelines or safety constraints.
Related Concepts2
Related Frameworks1
Related Practices13
Ask about this concept
Transcript mentions
… And AI security is prompt injection and jailbreaking and indirect prompt injection and AI red teaming and some major problems I've found with the AI security industry that I think need to be talked more about.
… And so they are kind of proposed as a defense measure against prompt injection and jailbreaking. And what I have found through running these events is that they are terribly, terribly insecure and frankly, they don't work.
Jailbreaking is like when it's just you and the model. So maybe you log into ChatGPT and you put in this super long malicious prompt and you trick it into saying something terrible, outputting instructions on how to build a bomb, something…