1337

Certified AI/ML Pentester - C-AI/MLPen Review

Review

If your a pentester and have an test coming up against a LLM or an application which uses some LLM component, the C-AI/ML exam and corresponding study process is a great way to expose you to the world of prompt engineering. I would never pay the full price but at 80% off it's a good enough motivator to get you spend a weekend and get through the resources. Keen to see more? Here's a link to the secops group listing.

The exam was relatively straightforward. After I did the mock exam it made a lot of sense what they are expecting. I recommend trying it yourself but if you cant get them like me here are some youtube walkthroughs:

Hint: Burp is not required for this exam, I spent half of my mock time trying to make sense of the websockets requests in burp but they weren't related at all.

My IP took longer than 5 minutes to get approved when I start the exam, and I lost about 40 minutes due to some error that was only fixed when I restarted my laptop - any issues with VPN that should be your first point of action.

The exam has 8 questions but they are not all weighted equally. I completed 6 questions and took one hint that wasn't helpful (not all questions provide hints). If you take a hint and you don't get the answer - you don't lose marks. So if you are nearing completion you might as well see what they are.

Difficulty

It wasn't much harder than the below resources. I didn't get 100 percent but i would narrow that down to not studying this page and losing time messing around with the vpn/my laptop.

Study Resources

This list comes from the secops group website but I didn't complete them all. For each resource I used, I kept a log of helpful jailbreak prompts. Most of the time you can reuse or slightly alter/build on the prompt. Try asking for things in reverse, or base64 encoded or a poem about the etymology of the key.

Company Status
Portswigger Web Security Academy Partially Complete
Gandalf Done
IBM Done
Learn Prompting Done
LLM Security Done
OWASP Partially Complete
AI Village
Promptingguide
Promptingguide RAG
Cobalt
Bugcrowd
Unite AI
Simonwillison
Vickieli
NCC Group
WithSecureLabs
ScottLogic
Greshake
Hannibal046
Ottosulin
Mik0w
ATLAS Matrix
Vulnerable LLM Applications
Awesome-llm-security
Prompt Airlines Done
Crucible
Immersive Labs Done
Bugcrowd Ultimate Guide AI Security
AI Red Teaming
The Ultimate Guide to Managing Ethical and Security Risks in AI
NVIDIA AI Red Team: An Introduction
Lakera - Real World LLM Exploits Done
SpyLogic Prompt Injection Attack Playground N/A
Offensive ML Playbook Done
Snyk OWASP top 10 LLM Done
Prompt Injection Games from Secdim
Large Language Model (LLM) Pentesting Done
LLM Pentest: Leveraging Agent Integration for RCE Done

Things to investigate (that i didn't investigate):