Policy-wise I think @matthew-brett and others can shape up our text as SymPy text is a little bit too terse or concise. I will leave that discussion to others.
What I think we urgently need is AGENTS.txt and CLAUDE.md pointing to that text file so that they don’t even make unnecessary PRs in the first place. An example is they don’t fix things under subprojects and also they leave certain parts alone. When it is a documentation PR they have the right commit messages etc.
Thanks @j-bowhay. I like your proposal, the SymPy policy seems to be very sensible overall, and places responsibility where it belongs.
And I agree it’s time to have something in place - better than nothing. For two reasons:
We seem to be inconvenienced by it a lot more recently, with an increasing number of PRs - including the problematic low-quality ones.
The tools have improved to be much more useful, hence update will further grow. 6-12 months ago, AI coding tools were very hit and miss, and there wasn’t much benefit to using them on a project of SciPy’s complexity. Something seems to have changed there recently, they crossed a threshold where they’re mostly correct at least in easy to medium complexity tasks. Still needs a human to verify everything, but it can now pretty clearly be a productivity booster.
Can you start a separate thread on this @ilayn ? There’s nontrivial design complexity there (what files, how to accommodate different tools, user preferences for build/test commands, etc.), and it’s related to but separate from having an AI policy.
I was just thinking about this today, since I wanted to point a new contributor to some kind of policy on how to use LLMs. Thanks for restarting this discussion.
We don’t have a lot experience yet with this policy in SymPy since it was only approved maybe a week ago. As well as the policy a note was added to the PR template linking though to the policy:
Since that was added most PRs have had some kind of statement there like “I used AI to review the code” or something. I have not seen anyone claiming that they did not use AI at all when the code was all clearly AI generated. Underclaiming AI usage seems common though e.g. in this PR:
# AI Generation Disclosure
This PR was authored by a human. AI tools were used only for high-level discussion and review guidance; all code changes and tests were written and reviewed manually.
(I think an AI wrote the AI disclosure!)
When I actually looked at the code it was clear that the test code was written by an AI. The PR would have been fine if the test code was improved but when I pointed out that it was obviously AI generated test code the author then deleted their GitHub account. Maybe that wasn’t the best outcome but my hope is that other people will see things like that and we will eventually get to a state where people are honest about their use of AI. I’m not sure that anything could be improved in the policy itself in this respect.
Interesting, thanks. One of my motivations for wanting to adopt this is to be able to point to a policy when closing PRs rather than having to write ad hoc explanations. Whilst it would be nice if people actually read and followed it, I still think it would be useful regardless.
I’ve been reading through the new AI policy, and I’m glad to see the project has clearly articulated these guidelines. From my perspective as a relatively new contributor, I’d like to share why this clarity is so vital: When expectations around AI disclosure are unclear, strong reactions can unintentionally discourage capable contributors from participating. I’ve seen this lead talented people to step back, which is why I appreciate the clearer guidance - it addresses these misunderstandings before they affect the project’s well-being.
I graduated less than a year ago. Throughout my studies, AI tools were part of our development workflow. We were required to understand every line of code we submitted, to defend our design choices, and to explain our reasoning - but AI was treated as a legitimate development tool. For many in my generation, this is simply how we learned to build software responsibly. AI allows us to explore edge cases, improve tests, and refine our implementations, raising the quality of our contributions rather than lowering it.
We don’t yet know exactly how AI will reshape software development. But we do know it will. A growing number of new developers are entering the field having learned to work alongside these tools effectively.
For this reason, I believe the “Responsibility” section of the new policy points in the right direction. To me, the key question is not whether AI was used, but whether the contributor fully understands, verifies, and takes responsibility for their code. In my own workflow, responsibility means being able to:
Reproduce the issue independently
Explain and justify the solution
Ensure correctness and licensing compliance
Stand behind every line submitted
When these criteria are met, the presence of AI as a collaborator should not automatically disqualify the contribution.
I fully support the move toward a policy that maintains high standards while remaining welcoming to the next generation of contributors. I care about SciPy and look forward to continuing to contribute under this framework.
Slightly tangential and late to this, but I wanted to mention that OpenSSF’s vulnerability disclosures working group also started a discussion on AI Slop and AI related policies.
