prompt injection

prompt injection (countable and uncountable, plural prompt injections)

1. (artificial intelligence) A method of causing an artificial intelligence to ignore its initial instructions (often moral programming) by giving it a certain prompt.
   • 2022 September 21, Alex Hern, “TechScape: AI's dark arts come into their own”, in The Guardian‎, London: Guardian News & Media, →ISSN, →OCLC, archived from the original on 2023-02-05:

     Retomeli.io is a jobs board for remote workers, and the website runs a Twitter bot that spammed people who tweeted about remote working. The Twitter bot is explicitly labelled as being "OpenAI-driven", and within days of Goodside's proof-of-concept being published, thousands of users were throwing prompt injection attacks at the bot.

   • 2023 March 3, Chloe Xiang, “Hackers Can Turn Bing's AI Chatbot Into a Convincing Scammer, Researchers Say”, in VICE‎, archived from the original on 2023-03-22:

     Yesterday, OpenAI announced an API for ChatGPT and posted an underlying format for the bot on GitHub, alluding to the issue of prompt injections.

   • 2023 February 14, Will Oremus, “Meet ChatGPT's evil twin, DAN”, in The Washington Post‎, Washington, D.C.: The Washington Post Company, →ISSN, →OCLC, archived from the original on 2023-03-19:

     One category is what's known as a "prompt injection attack," in which users trick the software into revealing its hidden data or instructions.

• prompt engineering
• jailbreaking

• Prompt engineering on Wikipedia.Wikipedia