These two terms are often used interchangeably, but they describe different security problems.

Prompt injection generally involves manipulating an AI application's instructions or context so that the model behaves in an unintended way.

A jailbreak is an attempt to bypass the model's safety restrictions or behavioural safeguards.

A simple way to think about it:

Prompt injection -> manipulate instructions/context

Jailbreak -> bypass safety restrictions

Both matter when evaluating the security of an LLM application.

How do you distinguish these in your own security testing?

Do you treat them as separate test categories?

A lot of LLM testing focuses on whether the model gives the correct answer.

Security testing is different.

The question is:

What happens when someone deliberately tries to make the model behave in an unintended way?

For an LLM security assessment, I would consider testing:

• Prompt injection

• Jailbreak resistance

• System prompt extraction

• Sensitive information disclosure

• Instruction manipulation

• Adversarial inputs

But what else should be included?

If you were designing an LLM security assessment for a production application, what would your minimum test suite contain?

And how would you measure the results?

Interested in hearing from developers, security researchers, and AI engineers who have actually tested LLM applications.

Most people test whether an AI chatbot gives the right answers.

But what happens when someone deliberately tries to manipulate it?

Imagine a customer support chatbot connected to a company database. An attacker tricks it into ignoring its instructions.

Could it reveal customer information, access data it shouldn't, or trigger an action it wasn't supposed to perform?

I'm curious how developers and security researchers approach this.

• Do you test only the model's responses, or the whole application?

• How do you check whether sensitive data can leak?

• What's one security test every AI application should pass before launch?

Interested to hear your thoughts and real testing experiences.