Recommend best practices for continuous integration:
1 Like
This is coming out of our sprint at the 2022 NumFocus summit.
Idea for Actions:
- Recommend project to understand security risks and guard against them when possible. See Security hardening for GitHub Actions - GitHub Docs
- Use hash of release over tag name because the latter can be silently overwritten. Though maybe tag name is acceptable if your Action is from a very reputable source?
1 Like
Re: Nightly jobs – I would go a step further and recommend you run it for all PRs but allow it to fail to make sure changes are forward-compatible (e.g., not using a feature that has been dropped in unreleased upstream code). Allowing it to fail will not block PRs if the failure is unrelated to the PR changes (since dev is unstable by definition). Maintainers do have to do the due diligence to check it even though it is allowed to fail or this will not accomplish its purpose.
1 Like
Hi all, thanks for the really important work you’re doing here.
@dopplershift @jarrodmillman is there more discussion I can read somewhere to understand the context for this SPEC?
Something like what @matthewfeickert provides here
Interest in GitHub Action for scipy-wheels-nightly uploads and removals for SPEC 004?
I went to the links but they just have recommendations and then they link back here.
2 Likes
This is a placeholder from a BoF at the recent NumFOCUS project summit. I have additional details, but I haven’t had a chance to write things down yet. I should have time this weekend or early next week. But there will still be a lot to discuss and work out for this SPEC. I will comment here once I’ve added some additional details from our discussion.
Understood, thank you @jarrodmillman!
I guess I’m not clear on the process yet – I thought “draft” meant it was already accepted?
Just want to make sure I can understand and relay what you all are doing
I will remain calm 
and look forward to the discussion.
1 Like
On the idea of trying to get some simple things set up (and maybe this folds under SPEC 8 as well), how about recommend projects use whatever built in security tooling is provided by their git provider? So GitHub projects get Dependabot, secret scanning, and CodeQL enabled. GitLab has its own tooling (Get started securing your application | GitLab) for security scanning and dependency checks. BitBucket (are there any projects on Bitbucket? I haven’t seen any) seems to have Snyk integration (Bitbucket Snyk Integration to Enable DevSecOps | Atlassian).
On the infrastructure side of things the NumFOCUS Infrastructure Committee is working on a guide for CI/CD set-up (Paying for CI/CD with numFOCUS — numFOCUS Project Infrastructure documentation). Perhaps for the SPEC we will want to focus on what to do in CI and how often to do it, rather than specifics of how to set things up.
1 Like
The idea is definitely about best practices rather than technical how-tos. E.g. test with the declared minimum requirements, test with development version of the requirements, have cron jobs, etc.
From the examples above the secrets scanning could fall into these recommendations, dependabot OTOH caused some not-nice experiences for some projects (and anyway is more on the convenience side of things than necessities), so I would steer away from recommending it, as well as to go into the depth of weeds of any configs and setups, especially for outside of github.
1 Like
This item could also deal with best practices for how to maintain and develop CI systems. E.g. develop in fork before making PR against main project. see Debugging CI guidelines · numpy/numpy Wiki · GitHub