Red teaming is a deliberate attempt to make an AI system fail, misbehave, or leak, done before launch so your own people find the failures, not customers.
Category
AI, Data Governance
Also known as
AI red team; adversarial testing
It is testing with hostile intent: jailbreaks, injections, off-topic requests, and the questions a competitor or a journalist would ask. The output is a list of what broke and a decision on each.
In practice
Give three people an afternoon and a goal of making the bot embarrass the brand. The transcript they produce becomes the first eval set, and it is cheaper than the screenshot that would have gone viral.
Related
Something out of date?