SunnyRESEARCH IN PRACTICE
Open benchmarks, replication packages, and research artifacts for studying how language models review code. Each project links to its data or code and to the related research.
A benchmark for measuring whether adversarial source-code comments can make LLM-based vulnerability detectors miss real bugs. Cases cover 30 vulnerable functions, 14 CWE categories, and 12 comment strategies across four studies.
A benchmark of cryptographic API misuse for testing AI code reviewers. Each vulnerable Python snippet is paired with a secure control, so a model that flags everything is measured accordingly.
Verdict logs, scripts, and checks for a matched safe-code control in adversarial comment injection. The release includes the 10,580 model calls used in the paper across five Qwen2.5-Coder sizes.
Matched cryptographic misuse examples compare direct calls with aliases and defaults, alongside secure twins and an audit of pairs that are not clean matches.