Clarification about AI

[kelson42]

Fixes #35

Simple and short rules about AI usages

[benoit74]

I'm not especially aligned. To me this is a subtle topic which deserves subtle discussions to achieve an understandable and shared rule.

I don't get why reviewers (especially given who they are today) would be granted more "freedom" than first time committers: this does not creates a welcoming environment. Out of the 3 reasons given, only the first 2 are strong ones ; both apply equally well to reviewers.

In addition to that, most of Kiwix generative AI usage today is probably not in coders roles / tasks, so I don't get why this population gets a special attention.

Also not all generative AI are equal when it comes to the reasons given, and I bet this is going to be even more true in the future, ruling out a whole technology based on some early solutions disadvantages seems strange to me.

Finally, other reasons are not given (ecological footprint, societal impact, ...) while they should at least be discussed.

Don't get me wrong, I'm not saying this is a fight we should not decide to make. Others already did (e.g. Commown in France). But this must be done at org level with org adhesion. To me this PR as-is is yet another insufficiently prepared top-down decision.

[kelson42]

I don't get why reviewers (especially given who they are today) would be granted more "freedom" than first time committers: this does not creates a welcoming environment. Out of the 3 reasons given, only the first 2 are strong ones ; both apply equally well to reviewers.

First of all we decide what we want, this is our project. Then reviewers are experienced and in charge, so they are trusted and have more responsibilities and freedom. Finally, the reviewer does not create content, and the problem is the generative AI created code.

[kelson42]

I don't get why reviewers (especially given who they are today) would be granted more "freedom" than first time committers: this does not creates a welcoming environment. Out of the 3 reasons given, only the first 2 are strong ones ; both apply equally well to reviewers.

First of all we decide what we want, this is our project. Then reviewers are experienced and in charge, so they are trusted and have more responsibilities and freedom. Finally, the reviewer does not create content, and the problem is the generative AI created code.

In addition to that, most of Kiwix generative AI usage today is probably not in coders roles / tasks, so I don't get why this population gets a special attention.

You can open an issue for the rest, I just solve the #35. It is really not necessary to make the problem bigger here.

Also not all generative AI are equal when it comes to the reasons given, and I bet this is going to be even more true in the future, ruling out a whole technology based on some early solutions disadvantages seems strange to me.

Generative AI is not new, and this CONTRIBUTING can evolve. What is sure is that we need to be clear about our contributors. It is not fair to have unclear contributing rules.

Finally, other reasons are not given (ecological footprint, societal impact, ...) while they should at least be discussed.

We are not going to make a full list, it is just a small explanation about the most important points.

Don't get me wrong, I'm not saying this is a fight we should not decide to make.

Not a fight at all, just a thing to do to make our life easier and in accordance to our values.

@benoit74 Please propose concrete changes, I personally don't want to run the xth discussion on that topic.

[benoit74]

You can open an issue for the rest, I just solve the #35. It is really not necessary to make the problem bigger here.

I think this is the root cause of our misalignment: I don't get what the problem is given description in #35 misses any description of it besides "write something on AI usage". I assumed the goal was to clarify usage of AI at Kiwix level since we are in an "overview" project and the file modified speak about contribution "in general" (not only code but also issues, design, documentation, ...). Looks like I was wrong.

@kelson42 please clarify the problem you want to solve before asking for feedback, otherwise I can't help.

[kelson42]

@benoit74 I have clarified things in the issue. I don't think this was necessary as what is written in the PR is already clear. I won't discuss here the core of the topic. I have a clear opinion on the topic based on observation of dozens of PRs. What I expect is patches on my PR to improve the phrasing.

[rgaudin]

I think this new section in CONTRIBUTING is important because we are already affected by external contributors submitting LLM-assisted PRs. We owe to our contributors to explain beforehand that those won't be accepted.

It's not a general answer to the topic but I'm afraid we won't have an answer soon enough.

What @kelson42 settled for at the moment is that we can accept LLM-assistance if it's not visible. If it looks like AI, it probably didn't receive enough care and thus we shouldn't waste our sparse review resources on it.

Because it's a difficult exercice and because we want to limit LLM-usage, we are limiting (the wording is more permissing) it to known contributors. I don't see this are not being contributors friendly. It's understood that we don't share secret nor run sensible code from PRs automatically. Same applies here.

I've made some changes to the wording.

@kelson42 do you consider this closes kiwix/overview#128?

I also think a general (not just code) discussion should document/police our own AI usage

[rgaudin]

Good enough

[Jaifroid]

As a maintainer on the receiving end of these PRs: what I keep seeing is "the contributor never ran the code" rather than "AI was used". As I mentioned in kiwix/overview#128 (comment), asking "How did you test this, and what were the results?" will often result in silence and the vibe coder disappearing. So then I close the PR.

My worry about the current phrasing is enforceability. I've had solid PRs from non-native English speakers that read as LLM-written, but have been tested and work well, and sloppy human PRs that look human but haven't been tested (granted, mostly pre-AI).

So I'd gently suggest leading with the standard you want PR contributors to reach rather than the tool they used to get there. Something like this as a replacement for the first two lines, keeping the reasons below as they are:

Contributors must understand every line they submit, be able to explain their code in review,
and have tested it thoroughly themselves. PRs that fail this standard will be closed,
whether or not AI was involved. Using generative AI to produce code or PR descriptions
without having done the hard work of grappling with the codebase is not welcome,
particularly from new contributors.

In my experience, explaining why gets better results than prohibiting. That's the approach I've taken in kiwix-js-pwa's CONTRIBUTING, just in case it's useful as a reference.

[kelson42]

My worry about the current phrasing is enforceability.

There is nothing to enforce in my mind because there is no strict interdiction!

[Jaifroid]

OK, I just felt the way it's currently phrased sounds rather like an interdiction! Hence my suggestion to lead with the standard we want PRs to meet. But it's just my tuppence!

[benoit74]

I feel the same than @Jaifroid regarding interdiction and clearly prefer his wordings to this PR by an order of magnitude.