SciPy limit the number of open PRs

Contributor & Development Discussion scipy
1

Hi,

Due to recent surge of automated bulk PRs batches, we’re experimenting with limiting the number of open PRs from a single user in SciPy [1].

This is strictly an experimental anti-spam measure, and we very much do not intend to anyhow limit or otherwise inconvenience actual human contributors. Therefore, if you are contributing to SciPy and are being hit by this limit

  • we’re sorry about it!
  • please let us know. Note that there’s no absolutely no need to justify why you want to open one more PR, just a quick note “I am being limited by GH” is perfectly fine. Either here in this thread, or, if you don’t feel like saying this in public, just shoot me an email.

Cheers,

Evgeni

[1] How pull request limits are cutting down the noise - The GitHub Blog

2

I’m interested to know how this worked out because I am considering if a similar policy makes sense for SymPy.

What number of open PRs did you set the limit to?

Is this limit still in place?

Does it seem to have been helpful?

3

We ended up limiting it to 7 PRs per contributor, to account for the number of open PRs of structural contributors without write access. We also added several people to the bypass list, so maybe just maybe we could have applied a more agressive limit.

The limit is still in place, and, in fact, it’s been mostly invisible. I’m not aware of any active harm or even significant friction from enabling it. At a guess, SciPy is lucky to not have this problem too be acute enough.

Anecdotally, and strictly as a personal reaction, seeing a limit of 1 PR in a project where I’m not a frequent contributor, it does send a loud and a rather unpleasant signal. While I totally understand the reason, a very aggressive limit does give out an unfriendly and “you’re not welcome” vibe, for a well-meaning occasional contributor.

SciPy I suppose will just keep its lax limit until and unless there’s a reason to revisit it, one way or another. For projects with a more severe need to put a lid on slop, I could only handwave that maybe a more aggressive limit of >1 is a reasonable middle ground, but do not have any non-anecdotal evidence for it.

4

Does that also mean that you are not aware of any significant benefit from having the limit?

5

Does that also mean that you are not aware of any significant benefit from having the limit?

Exactly.

This is however a strictly N=1 study: the upper cap of 7 PRs only fires for most egregious cases, and is invisible otherwise. A tighter limit is probably more effective—if you can place it without affecting your normal well-meaning contributors.
FWIW, I’d be interested in hearing how this goes for other projects.