POST: AI-generated code search?

When ChatGPT copies code and does not give credits, that's an issue.
Let's look at the new tools we have built that help uncover unknown borrowed code.

-----------------------------

## Understanding AI-Generated Code Search

As GenAI continues to integrate (or creep? or flood? :evil: ) software development, tools like GitHub Copilot, Claude and ChatGPT are becoming common to generate code, code completions, and code fragments (aka. snippets), and in some case whole apps. While these tools can enhance productivity (at least some believe it does :angel: ), they also bring new challenges and problems related to the generated code origin, licensing, and security when the generated code provenance is not well defined and lacks clarity (read: all the times). To address these issues, we started the AI-Generated Code Search project at https://github.com/aboutcode-org/ai-gen-code-search as a true open source code and open data solution to identify AI-generated code and tracing its origins back to its sources.

This project has been funded in part by the European Union's NGI Search project which is really cool :heart:

### The problem with AI-generated code

LLMs that generate code are trained on huge piles of publicly available open source code sourced from git repositories, services like GitHub, and archives like Software Heritage.
Consequently, the code they produce may eventually replicate existing FOSS code that was used in their training, because the LLMS do "memorize" the code. After all, an LLM is basically a compressed and indexed representation of its training data, and there is plenty of public evidence that LLM-backed systems can regurgitate some the texts used in their training mostly verbatim.

To deal with this, there are efforts to better trace the provenance of LLMs training code data set such with HuggingFace BigCode project where "the Stack" of code was first scanned with ScanCode to ensure that the license and provenance of the code would be used as a filter! Yeah!

But is this a problem? How often is there FOSS code that makes it mostly as-is in GenAI code? Based on anecdotal evidence, I was able to generate code highly similar to that of an existing open source project in about 25 to 30% of the cases.
You can see some details of this test at https://ai-gen-code-search.readthedocs.io/en/latest/approximate-matching-testing-2.html

If ~20% of generated code could be derived from FOSS code, this raising some issues:

- FOSS Licensing: AI-generated code might include bits from FOSS code with licenses that are incompatible with the user's project. Or include code that completely forgoes giving license notice or credits to FOSS authors where credits is due. There are a badzillion of other legal and ethical concerns raised too. And in earnest based on chats with FOSS lawyers, the jury is still out if AI-generated code is copyrightable at all, and this bring serious concerns about what FOSS means in the GenAI era if and when the bulk of the code may be AI-generated! :face_with_spiral_eyes:

- Security: If the source FOSS package contains known vulnerable code, these vulnerabilities could be carried forward into my projects with Gen-AI. Basically garbage in, garbage out, but now automated on an industrial scale. What's not to love about this? :wastebasket:

- Compliance: New regulations like the EU's Cyber Resilience Act (CRA) and requirements for Software Bill of Materials (SBOMs) demand clear documentation of software origins. What if you just cannot tell? :scales:

Identifying whether code is AI-generated or not and determining its provenance is therefore likely useful to maintain some basic level of code integrity, and software license and security compliance. And frankly, beyond compliance, this is sane and basic code and engineering hygiene.

### How we worked towards finding AI-generated code

We extended the AboutCode project with AI-Generated Code Search features and now provide tools to detect GenAI code and trace it back to potential FOSS sources. The project parts includes:

- MatchCode app: a bunch of ScanCode.io pipelines to drive the step-by-step code anslysis and identify the similarities between your code and FOSS code, down to code fragment and snippets even when these are not exactly the same.

- Indexing Tools: Utilities to create and manage indexes of FOSS codebases crawling large voulmes of FOSS code from many places, and creating indexes to support search and matching operations.

- Low Level Libraries: Various Python libraries designed to support low level code chunkling and approximate code matching, and enable identifying similarities between code fragments even when they are not identical.

- Documentation and Tutorials: Guides to assist in setting up and utilizing the tools effectively. This is always a work in progress, we never have enough of these!

You can see the project's documentation https://ai-gen-code-search.readthedocs.io/ for additional details

### How does this Work?

Traditional code matching techniques rely on exact matches of content-defined code fragments (aka. snippets), a process that is ineffective for AI-generated code that comes with slight variations, for instance when using different code generation parameters (like the "temperature"). We use instead an approximate matching approach using techniques based on locality-sensitive hashing (LSH) to detect similarities between whole code packages, whole code files, or code fragments exactly or approximately. We further "stem" the code to make the detection withstand programming variable renaming and some level of code refactoring that is typically observed with GenAI generated code.

That approach enables:

- Identification of origin(s): By comparing the scanned code with a big index of FOSS code, the tool can report potential sources based on matched similarity. The results are keyed by Package-URL (PURL), of course!

- Further license and vulnerability lookup : Understanding the origin of code helps in assessing associated open source licenses found in the original code and known vulnerabilities that may exists in the FOSS packages from which the GenAI copied code. These are tracked in the PurlDB and VulnerableCode and are looked up by the PURL returned from the step above. Using PURL also unlocks access to a vast ecosystems of tools, databases and services using PURL as a key. :candy:

### But does this work at all? Really?

Well, it works within reason. What we do is to say that code may be AI-generated if we can find a significant similarity with existing FOSS code, assuming that code was used in the backing LLM training data set.

So there are a lots of "ifs" and "buts" this brings up. If code is not similar, it will NOT be detectable with our approach. But in earnest, when generated code escapes the realm of similarity, IMHO it also escapes the realm of copyright, and it would start to be impossible to detect as such, except if some style or other metrics could be used as a proxy, or if LLMs-based generators were able to archive ALL the generated code for tracing, an unlikely feat. (but hey may be they do?)

For instance, some research on the topic talks about "striking similarity" with FOSS code as not insignificant in https://arxiv.org/abs/2408.02487v1 :

> [...] we evaluate 14 popular LLMs, finding that even top-performing LLMs produce
> a non-negligible proportion (0.88% to 2.01%) of code strikingly
> similar to existing open-source implementations. Notably, most
> LLMs fail to provide accurate license information, particularly
> for code under copyleft licenses.

So this is far from the 25 to 30% anectdotal evidence I found on a tiny scale, but still highly significant.

So here, we have some external and independent evidence that different approaches yield concurrent results and similarity is working OK for this.

### Practical usage

You can leverage these new tools to:

- Check for AI-Generated Code: Determine after the fact if code may have been generated by AI tools and if it is derived from FOSS code, and then process any associated risks. Or actually any code, not only AI-generated code. Copy and pasting code has always been an efficient and popular way to reuse code, and no AI is needed for this.

- Ensure compliance: Ensure that software code meet policy, legal, and regulatory requirements concerning code provenance.

- Track reused code origins : By understanding the origins of code at play, dev teams can make better informed decisions about whether it is OK and suitable to reuse that code in their codebase.

In all cases, whether you are a business or an open source project, you need that info to support your decisions and your policy because:

- A GPL-licensed project will not want to include proprietary code in its codebase.

- Or a commercial software development will not want to include fauxpensource "source available" code

- None likes to inject vulnerable code in a codebase.

### Getting started, slowly

To try out the AI-Generated code search in the AboutCode stack, you can:

- Follow the installation and usage instructions and docs https://ai-gen-code-search.readthedocs.io/en/latest/installation.html ...this is quite involved a process and not yet for the faint of heart. You have been warned!

- OR, drop a mail for access to a publicly accessible demo system at hello@aboutcode.org or pombredanne@aboutcode.org

We are working on a demo system to be made fully publicly accessible once we further deploy protections against abuse. We opened the access briefly and we were quickly flooded by spammers. :(

### Onward and upward!

AI-generated code becomes more prevalent, and the hype is unescapable, including with the emergence of "vibe coding" which has IMHO benefits that are still to be determined at any scale. Therefore, understanding code origin and provenance is key to maintain some sanity for software integrity, security, license, and compliance, and well, ethics such as wrt. plagiarism. This project is now integrated in the AboutCode toolchain and offers new practical FOSS tools that can work towards identifying and analyzing if the code was borrowed from FOSS, helping devs and orgs to make better informed decisions for their software development.

We still have lots of work left to do there. Come join us to kick in the tires and help make this work at scale!