Continuing the discussion from A proposed design for supporting multiple array types across SciPy, scikit-learn, scikit-image and beyond:
Trying to split this out, and now continue with a bit more in-depth discussion about type dispatching. Also to throw in some concepts (sorry if I over-read these concepts already being introduced differently).
- Type dispatching is implicit, the end-user does nothing, but gets an implementation that works with whatever they passed in (
torch) – so long one is available.
- We want “generic implementations”. That is implementations that may work for multiple array-objects.
- Generic implementations have to take care to return the expected output type (in case they don’t get the expected inputs). We could try to provide helpers for this, but it is probably not generically possible (e.g. whether an operation returns a sparse or dense array is non-trivial.)
Let me jump straight into
uarray vs. multiple-dispatchers here and introduce a (maybe) new term. In the other post I intentionally called this step “type dispatching” and not multiple-dispatching.
- multiple-dispatching: A type dispatcher which uses the type hierarchy to find the best implementation
- Last come first serve (LCFS): the
__array_function__design which finds an implementation by asking all “candidates”. If one says they can handle it they get it. Since we have to assume that later registered backends are more specific, we would ask them in reverse order. So the last registered backends gets it if they want it.
- Usual multiple-dispatchers register directly with a specific function
- These LCFS dispatchers have a concept of “domains”. For
__array_function__the “domain” is all of NumPy.
uarrayextends this to arbitrarily nested/name-spaced domains.
The introduction of domains means that it is possible to override a function without registering with it specfically. To make this clear:
Dask does not have an implementation for all of NumPy, but it always provides a fallback. This fallback issues a warning and converts the Dask array to a NumPy array.
Dask “supports” functions that it does not even know about.
One thing I am not sure about: Does
uarray have some other magic, such as “registration” without actually importing the library that is overriden? (which would use the “domains” as an implementation)
uarray currently(?) also has
with statement to disable/enable versions. This is targeted at backend-selection but spills into type-dipatching. It is hard to discuss though, if this is a feature that is considered for removal.
For now, I wish to ignore this, but to me that was one of the main points of why the
uarray discussion stalled the last time. So after nailing down what backend-selection API we want, we need to look at how it plays together with type dispatching.
I think the above points basically describe our design space with respect to “type dispatching”, let us introduce “backend selection” later. Backend selection is an argument why multiple-dispatchers may not work for us, but it is not an argument for
uarray's design choices!
There are two points above, the easier one IMO is “domains”. To me it seems unimportant to provide function generic override capability (i.e. I can override multiple functions with a single implementation). For now, for all practical purposes to me “domains” are are currently only an implementation detail.
This is a difficult discussion, and I am not yet sure how much it matters in practice, but let me put forward two points:
This point I strongly disagree with: A multiple-dispatcher can insert a
DuckArrayinto the type hierarchy. This could be based on an ABC checking for dunders or based on registration. If it helps, I can provide a proof of concept impelementation.
The disadvantage of LCFS is that it has no notion of a “best” implementation. This means that registration order is important. As an example:
skimage.functionhas a generic “CPU” implementation
pytorchadds a specific
cupyadds a generic “GPU” implementation to
If the user now passes a pytorch GPU array in, the
cupyimplementation will provide the implementation even though presumably the
pytorchone is better.
Smaller points are maybe:
- LCFS is simpler to reason about for the implementer
- multiple-dispatching (to me!) seems actually easier to reason about for the person writing the functions. They just list the types that they want to handle
MyArray. The only step they really need to understand is that you probably want to use
MyArray, and we need to provide an easy way to get
- multiple-dispatching may sometimes be tedious/bloated to register the right combinations of types (although I doubt this is a huge problem). If no implementation is found, multiple-dispatching may also be harder to debug/understand.