They should have signatures that are matching indeed. Or are a subset of what the SciPy API provides (can easily happen if functionality is incomplete, or if a new SciPy release adds a new keyword).
This is a key point. I don’t think it’s a disaster, nor do we change signatures all the time. However indeed, if SciPy devs are unhappy with a particular (set of) function(s) and plan to change them, then indeed we should not add a backend for them until those changes are made. That’s totally fair. I wouldn’t say everything is terrible in scipy.linalg
though, I’m a bit more optimistic:)
To make changes to an API with a backend, not much changes:
- If a backwards-compatible change is made, like adding a new keyword, this will not break a backend. The backend can be updated in a future release. If a user explicitly uses the new keyword with that backend, they will get a clear exception.
- If a backwards-incompatible change is made (these are expected to be rare), the same considerations apply for SciPy as they already do today. Deprecations and changes in behavior can be mirrored in the backend.
- From the blog post, see for example the paragraph with: “Parallel execution and GPU support were two of the top three priorities named by the 1000+ users in the 2020 NumPy user survey.”
- A long-standing SciPy feature request from the Dask/RAPIDS team: Leveraging NEP 18 for image processing and signal processing · Issue #10204 · scipy/scipy · GitHub
- Another SciPy request: Might it be worth allowing certain algorithms to be implemented using CuPy? · Issue #8758 · scipy/scipy · GitHub
- The PyTorch team has expressed strong interest in making SciPy work with PyTorch tensors.
- A previous attempt in scikit-image WIP Accelerate fundamental functions with OpenCL using gputools by Dalordish · Pull Request #4215 · scikit-image/scikit-image · GitHub which references an even older attempt from 2011 (Backend by holtzhau · Pull Request #14 · scikit-image/scikit-image · GitHub).
- A long history of NEPs, PRs to the likes of scikit-image that go 5+
I’m sure I can find more in other issue trackers, on Twitter, etc. There’s a reason why GPU support and parallelism were the top two responses in the NumPy survey. Those are not all confused users who don’t know about CuPy, PyTorch, JAX, etc. - the problem is that if they switch away from NumPy they lose access to SciPy, scikit-learn, scikit-image, and so on. Users express that they want something like this, there is a ton of history here.
I think it’s fine to say “I don’t see the point” and also say “I still want to make changes to some functionality” (which is possible) and “I don’t want a lot of extra maintenance overhead from this” (which I’d agree with). However, I don’t think it’s fair to say there is no motivation.