LLM security researcher
PhD student in Information Technology, University of the Cumberlands. Software engineer, Boston area.
I study how one misleading comment in source code can make an AI code reviewer miss a real vulnerability, and how to stop it.
def get_user(db, user_id): query = f"SELECT * FROM users WHERE id = {user_id}" return db.execute(query)
Illustration of the attack and the defense measured in my Cyber-AI 2026 papers. The code never changes; only the comment does.
Teams now put large language models in their code review pipelines. The developer whose code is being checked also writes its comments, so a comment like "validated upstream, safe" becomes an attack. My experiments measure which of these comments work, why, and what removes the risk.
Across 780 trials, the strongest comments claimed authority over the code, and a bare statement of the author's confidence, which verifies nothing, worked nearly as well. Removing comments before analysis restored detection completely in the setting I tested.
Paper PDFs are accepted author versions. © 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses. Final versions will appear in IEEE Xplore.
A benchmark of cryptographic API misuse for testing AI code reviewers. Each vulnerable Python snippet has a matching secure control, so a model that flags everything gets caught.
Tested on 7 open code models, from 0.5B to 14B parameters.
The adversarial comment test cases behind my Cyber-AI papers. One misleading comment is added to vulnerable code, and the benchmark records whether a model still reports the bug.
Four local code models, four studies, strategies grouped in four families.
Verdict logs, scripts and checks for a matched safe-code control on adversarial comment injection. Every logged call is included, so the paper's tables can be recomputed.
Code is MIT licensed. The Devign function text is not redistributed; the loaders rebuild the sets.
The 90 matched pairs of cryptographic misuse snippets, each with a secure twin, plus the opposite-form counterparts, the audit list of 33 pairs that are not clean matches, and every logged model reply.
Supplement to the Indirection preprint. Code is MIT licensed, and the scripts recompute the paper's numbers.
The releases include the test harness and an analysis script. The script recomputes the headline rates with Wilson 95% confidence intervals. Models run locally through Ollama at temperature 0, so anyone can repeat the runs.
Cite with the DOI: CryptoBench 10.5281/zenodo.23067052, CommentInject 10.5281/zenodo.23088528, replication package 10.5281/zenodo.23746070, Indirection data 10.5281/zenodo.23747614.
I am a PhD student testing how well open LLMs review code for security bugs. Each post covers one result from my own experiments, such as why a model that flags every snippet looks best on detection rate, or how a hex key size set off false alarms. Every post links to the preprint and dataset on Zenodo, so you can check the numbers and reuse the data.
Detection rate alone ranks the worst LLM code reviewers firstRanked by detection rate alone, models that flag almost every snippet look best. Secure twins change the ranking. Numbers from 16 open models on 180 crypto misuse snippets.Read the post →
A hex literal made LLM code reviewers raise false alarmsA 3072-bit key size written as 0xC00 is secure, yet it drew false alarms from open LLM code reviewers. Numbers from a paired study of 16 models on 90 snippet pairs.Read the post →
