How are you handling benchmark regression reporting on PRs?

Contributor & Development Discussion
1

I’m helping add benchmarking regression reporting to a Python project.

I want to know how you guys are reporting regression results in your PRs, so people can see if their PR caused a regression.i.e., do you post comments on the PR showing whether performance improved or regressed?

1 Like
2

I believe much of the numerical python community uses ASV. We’re beginning to use ASV in pysindy.

3

We are also using ASV. I’m mainly interested in how you report results to users in pull requests.for example, whether you post PR comments indicating that performance regressed or improved, or if anyone posts benchmark output directly as PR comments.

1 Like
4

That’s a good question; we haven’t cracked that nut.