OWASP Top 10 for LLM Applications

Manipulating prompts to alter the model's intended behavior or bypass its restrictions.

Mitigation Strategies

Example: The user manipulates an LLM to summarize a webpage with instructions to ignore previous commands.

Exposing confidential or proprietary data through training, fine-tuning, or user interactions.

Mitigation Strategies

Example: Samsung employees once inadvertently exposed confidential internal data to ChatGPT.

Introducing risks via compromised models, datasets, or third-party integrations in the AI development pipeline.

Mitigation Strategies

Example: Pre-trained models and poisoned datasets can introduce biases, backdoors, and vulnerabilities, enabling data theft, system compromise, and harmful content generation.

Contaminating datasets or tampering with models to introduce biases or malicious backdoors.

Mitigation Strategies

Example: Microsoft’s Tay chatbot was manipulated by malicious users with offensive inputs, causing it to generate harmful content.

Failing to validate or sanitize model responses before passing them to downstream systems.

Mitigation Strategies

Example: An LLM generates an unrestricted SQL query, enabling attackers to delete sensitive database records.

Granting LLMs unauthorized autonomy or functionality beyond their intended scope.

Mitigation Strategies

Example: A car dealership’s LLM agent with excessive privileges exposes confidential next-quarter sales promotions, impacting current sales and revenue.

Extracting sensitive information or internal instructions embedded in system prompts.

Mitigation Strategies

Example: Threat actors extracted GPT-4o voice mode system prompts, revealing behavior guidelines and sensitive configuration details.

Exploiting vulnerabilities in vector databases or RAG pipelines to inject malicious content or retrieve sensitive data.

Mitigation Strategies

Example: In multi-tenant environments, an adversary retrieves another tenant’s embeddings, exposing confidential business data.

Generating false or misleading information that appears credible but lacks factual accuracy.

Mitigation Strategies

Example: An airline customer support chatbot incorrectly stated that bereavement fare refunds were available within 90 days, contradicting the company's policy.

Overusing LLMs for resource exhaustion, service degradation, or replicating the model's functionality.

Mitigation Strategies

Example: An adversary floods the LLM's API with synthetic data requests to train a duplicate model.