steppi
December 19, 2025, 6:31pm
1
I’ve created an issue here:
opened 06:26PM - 19 Dec 25 UTC
Documentation
RFC
array types
In #24130 I've worked on adding documentation on `xp_capabilities` (as well as a… dding strict checks that it's used correctly). I wrote in that thread that I plan to follow-up with an additional PR on proper documentation of the use of `xp_capabilities` for classes, but first, there are some decisions that need to be made. Currently, at least two approaches have been taken in the SciPy codebase to add `xp_capabilities` to classes.
#### xp_capabilities added to entire class
In `interpolate`, `xp_capabilities` is applied directly to the class like this:
https://github.com/scipy/scipy/blob/a4756d53490bb13c59fb8d0684e60ec02458d61d/scipy/interpolate/_rbfinterp.py#L57-L69
#### xp_capabilities added to methods separately
whereas in `spatial`, `xp_capabilities is applied separately to each method:
https://github.com/scipy/scipy/blob/a4756d53490bb13c59fb8d0684e60ec02458d61d/scipy/spatial/transform/_rotation.py#L374-L380
#### Some pros and cons
There are pros and cons to each approach and either choice will require some additional work.
When applying the decorator to the class itself, one pro is that we get a nice table in the main class docstring. If the decorator is applied separately to each method, the current situation is that there is no information about array API support at all in the main class docstring.
One pro of applying the decorator individually to each method is that it allows for more granularity in declaring backend support; I think this is a double-edged sword though.
When reviewing the PR to add array API support to `scipy.spatial.Rotation`, @crusaderky commented (https://github.com/scipy/scipy/pull/23249#issuecomment-3092580228):
> Instead of answering directly it I'm going to phrase it as user story and let you decide:
>
> _As a final scipy user, I want to open the main documentation for Rotation and be clearly informed if a backend is (mostly) unusable throughout the whole class_.
>
> You'll need to render the HTML docs to decide if the current state satisfies this requirement or not.
I think this is exactly right. What users would primarily be interested in is seeing at glance if the class is mostly usable on a given backend. A well designed class isn't just a bag of methods, but should have a conceptual unity, and rather than just documenting support for each method separately and expecting users to figure thing out from there, SciPy developers should use their best judgment to decide whether the class is mostly usable in a holistic sense for each backend. As an example `scipy.spatial.Rotation` is essentially completely unusable with Dask due to missing `linalg` support, so there's not really much value in documenting that `__iter__` works in Dask like this:
https://github.com/scipy/scipy/blob/a4756d53490bb13c59fb8d0684e60ec02458d61d/scipy/spatial/transform/_rotation.py#L2648-L2658
So I think having `xp_capabilities` on the class is nice because it puts the docs table in a central place, and allows us to document whether the class is broadly usable. There is a question however, of what to do about particular methods that are not supported? We see in `scipy.spatial.Rotation` that some methods don't work in CuPy for instance:
https://github.com/scipy/scipy/blob/a4756d53490bb13c59fb8d0684e60ec02458d61d/scipy/spatial/transform/_rotation.py#L1292-L1298
I think what can be done in such cases is simply to list method specific caveats in the `extra_note` of the `xp_capabilities` applied to the class. However, when testing, this means that we currently couldn't do something nice like `make_xp_test_case(Rotation.as_davenport)` like what is currently being done in `spatial`. Instead one would need to add the relevant `@skip_xp_backends` marker directly. I think its feasible to add an API for expressing method specific limitations that would allow things to still work nicely though.
#### Docs tables
Note that currently, classes do not appear in the [big docs tables showing array API standard coverage](https://scipy.github.io/devdocs/dev/api-dev/array_api.html#api-coverage) (as distinct from the small tables that go in individual docstrings), but @ev-br has a draft PR to add them: https://github.com/scipy/scipy/pull/23960. If the decorators are applied to the classes themselves, then I think it's natural that the big tables should list classes, but if the decorators are applied to individual methods, does this mean that each individual method should be listed in these tables separately? That appears unreasonable to me. Some classes, like [csc_array](https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.csc_array.html), have boatloads of methods. Including all methods in the tables would:
* Bias the counts of what is supported. When calculating percentage of API surface covered, I don't think so much weight should be given to `csc_array`.
* Be visually confusing to readers of the array API support tables.
But if the big tables are going to only include classes and not their methods, then we still need to come up with a small table for the class that gives a summary of what mostly works and what doesn't. I don't think there's a better place to declare such a table then in the `xp_capabilities` decorator for the entire class. One could work on automatically constructing such a summary table from the individual method decorators, but I think that path is folly. There aren't that many classes in the public API, and I think it is much simpler and more reliable to defer to human judgment in this matter.
It was also pointed out to me by @rgommers that if we were to add a small table to the docstring of each individual method, this would bloat the docs even more.
#### Shouldn't methods say something about array API support too?
I agree. I think a sentence could automatically be added to the notes section of the method docstring that the class has experimental array API support with a link back to the appropriate section of the main docstring.
#### A caveat: Testing on lazy backends
There is one current problem with applying the decorator to the entire class. The machinery in `array_api_extra` for testing on lazy backends does not currently work in this case. I created an issue here https://github.com/data-apis/array-api-extra/issues/488 about this, but had not yet thought through how this should work, which @crusaderky rightly pointed out. There's still some work that needs to be done to think of the proper way to handle this. What I'm thinking of is maybe to have a system where if `xp_capabilities` to an entire class, you have to also add a different decorator to the methods that should be given the `lazy_function` treatment. I'm working on prototyping something like this, but wanted to get the ball rolling first by posting this issue.
#### tldr
I recommend that:
* `xp_capabilities` be applied to entire classes and not methods.
* That the decision for which backends are and aren't supported for classes should be made based on human judgment of whether the class is useful with a given backend.
* The docs tables showing array API standard support should include classes but not individual methods.
* Caveats about unsupported methods should go in the `extra_note` of the class level `xp_capabilities`.
* There should be a sentence added in the notes section of each public method mentioning array API standard support with a link back to the section on array API standard support in the main docstring.
* Some work is still needed on figuring out what to do about testing on lazy backends when applying `xp_capabilities` at the class level, but this is a solvable problem.
* For now, tests involving methods with specific limitations can just be skipped with direct `@skip_xp_backends` markers, but it should be straightforward to find a nicer way to specify additional limitations with a separate decorator that doesn't add an unwanted small table to the docstring of each method.
Looking forward to your feedback.
to discuss how xp_capabilities should work and be used for classes. There are currently two approaches being used in different places in SciPy. Applying xp_capabilities to the entire class, or to methods individually. I think it should be applied to the entire class, but would be happy to hear feedback for either option. Feel free to join the discussion at #24196 . Looking forward to hearing feedback.
2 Likes