NCAS-CMS / NCAS-CMS/pyfive

Policy for AI written code - please comment!

Open
#215 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
104
Forks
28
Avg merge
22h 47m
Merged PRs (30d)
6

Description

I think we need a policy for AI written code.

I do not believe "AI written code is not allowed" will work for two reasons a) a good AI will write code we can't tell isn't human, and b) I am already using it :-)

There are two big AI written code risks: 1) pull request complexity, and 2) pull request frequency. I think we need to mitigate both, as I do think whether we use AI code review or not, we still need human code review as well.

So this is my proposal for a policy:

  1. AI generated code will be accepted, provided is accompanied with a proper, reviewable issue laying out what it is trying to achieve, and the instructions which were given to the AI. If this issue is not accepted for implementation, we should not accept a pull request against it.

  2. No pull request will be accepted that is too complex for the existing team of human reviewers. If real complexity is required, each step must be first documented in an uber-issue, and in one issue per step. If there are too many steps, we will not review and accept it.

  3. The project maintainers are the arbiters of the definition of "proper, reviewable" in item 1, and of "too complex" and "too many" in item 2. We recognise that it is possible that some developers will take issue with any such decision. As always, you are free to fork if you do not like our decisions.

Comments welcome. I should say that I have expressly not reviewed the policies of other open source projects, but many of you will.

As an example of the sort of thing that I think is good practice (as in, the sort of thing I can cope with), I am in the process of doing some work on parallelism (with @zequihg50). I propose to put the work underway through this mechanism.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files or tests are named. Start by reviewing the proposal and its five comments, then determine whether maintainers agree on the policy and where it should be documented. Done means the project has a decided, reviewable policy for AI-generated code.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.