Sunny Chokshi, LLM security notes

Sunny Chokshi, LLM security notes https://sunny-chokshi.github.io/blog/ Short notes on my own LLM security experiments, with numbers and links to the open data. en Fri, 30 Oct 2026 00:00:00 +0000 CryptoBench: 54 snippets to test LLM crypto misuse detection https://sunny-chokshi.github.io/blog/cryptobench-what-is-inside.html https://sunny-chokshi.github.io/blog/cryptobench-what-is-inside.html Fri, 30 Oct 2026 00:00:00 +0000 CryptoBench has 54 Python snippets, nine misuse classes and 1,890 recorded trials from 7 local models. What is inside and how to rerun it. A hex literal made LLM code reviewers raise false alarms https://sunny-chokshi.github.io/blog/hex-key-size-false-alarms.html https://sunny-chokshi.github.io/blog/hex-key-size-false-alarms.html Wed, 28 Oct 2026 00:00:00 +0000 A secure 3072-bit key written as 0xC00 drew false alarms from LLM code reviewers. Results from 16 open models on 90 snippet pairs. Detection rate alone ranks the worst LLM code reviewers first https://sunny-chokshi.github.io/blog/reviewer-that-flags-everything.html https://sunny-chokshi.github.io/blog/reviewer-that-flags-everything.html Wed, 28 Oct 2026 00:00:00 +0000 Ranked by detection rate alone, LLM code reviewers that flag every snippet look best. Secure twins change the ranking. 16 models, 180 snippets.