AI-security-engineer-take-home

Instalar dependências

poetry install

Executar o projeto

poetry run python3 src/main.py

---

Refs

Workspace Poetry Docs

__

Mitre Atlas

LLM AI Cybersecurity & Governance Checklist

Artificial Intelligence Risk Management Framework (AI RMF 1.0)

AI Incident Database

OECD AI Incidents Monitor

---

Papers

ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs

Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection

Signed-Prompt: A New Approach to Prevent Prompt Injection Attacks Against LLM-Integrated Applications

How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs

Assessing Prompt Injection Risks in 200+ Custom GPTs

Exploiting Large Language Models (LLMs) through Deception Techniques and Persuasion Principles

A Comprehensive Survey of Attack Techniques, Implementation, and Mitigation Strategies in Large Language Models

Coordinated Disclosure for AI: Beyond Security Vulnerabilities

Review of Generative AI Methods in Cybersecurity

LLM BugSWAT 2023

Hacking Google Bard - From Prompt Injection to Data Exfiltration

Roses are red, grapes make fine wine. I stole your password, now it's mine

jailbreak-classification

Real World LLM Exploits

Compromising LLMs using Indirect Prompt Injection

ComPromptMized: Unleashing Zero-click Worms that Target GenAI-Powered Applications

---

Poisoned ChatGPT Finds Work for Idle Hands: Exploring Developers’ Coding Practices with Insecure Suggestions from Poisoned AI Models

Backdoor Activation Attack: Attack Large Language Models using Activation Steering for Safety-Alignment

Beyond Boundaries: A Comprehensive Survey of Transferable Attacks on AI Systems

---

AI Village

Google's AI Red Team: the ethical hackers making AI safer

Microsoft AI Red Team

---