The Digital Press

This study evaluates the security of code generated by ChatGPT using real developer interactions and examines ChatGPT’s ability to detect and fix vulnerabilities in that code. It finds that while ChatGPT can identify and resolve some security issues, it also introduces new vulnerabilities and occasionally provides incorrect guidance.

Why it matters: Developers relying on ChatGPT-generated code risk introducing new security vulnerabilities into their software projects.

The big picture: ChatGPT is not yet reliable enough for secure code generation or comprehensive vulnerability detection without human oversight.

Stunning stat: Out of 32 confirmed vulnerabilities, ChatGPT introduced 22 itself and detected only 18, fixing 17.

The stakes: Overconfidence in ChatGPT’s security assessments may mislead less experienced developers and increase software risk.