Purpose of `scipy.special.cython_special`?

Contributor & Development Discussion scipy
1

In scipy.special, there is now a public module scipy.special.cython_special. It appears to contain functions that are Python wrappers of the elementary operations for some ufuncs. Why is this module public? In fact, why does it even exist? The functions duplicate functions in the special namespace, but they are not ufuncs–they accept only scalars.

I’d like to at least rename this to _cython_special, if not remove it completely.

The module is actually documented here:

https://scipy.github.io/devdocs/reference/special.cython_special.html#module-scipy.special.cython_special

but I really don’t see the point of it. The public API for these functions should be the ufuncs.

2

This has in fact been documented since v0.19.0 - the original PR was in 2016 by Josh Wilson. It looks like the original mailing list post received no replies.

3

Thanks! All these years, and I’ve been oblivious…

1 Like
4

I suppose the first step, if you would like to look at deprecating it, is to check for usage across GitHub?

5

No, after seeing its history, I’m OK with letting it be.

1 Like
6

As someone who has been working a lot on special recently, I’m glad someone brought up cython_special…

@steppi and I have pushed to make everything in special a ufunc or gufunc, and it’s worked pretty well for the special submodule as a whole. We’re largely on the road to achieving it. Throughout the refactoring, we’ve kept cython_special working, but it’s been work. And there are many special functions - namely those that operate on subarrays - that aren’t supported in cython_special and likely will never be.

I’ve never used cython_special. Are there people who use it? The only thing it could be valuable for is avoiding overhead from a call to a NumPy ufunc. But, if you are already in Cython, is there not a more standard way to achieve that given a ufunc? Cython has all kind of tricks to make accessing NumPy arrays fast, is there something equivalent for a scalar call to a NumPy ufunc? If not, maybe we should talk to the Cython developers to see if it can happen.

I’d actually be happy to see cython_special go.

Anyway, those are my thoughts. I’m mostly curious what the consensus is for what should happen to cython_special going forward.

7

I’ve used it. Cython 3 makes it easy to create ufuncs with a decorator, and it’s helpful to have access to special’s scalar kernels within the scalar kernels you write in Cython. If the Cython developers add a way to call the scalar kernel of a ufunc (evaluations on scalar input within a cdef function get compiled to C that calls the scalar kernel?) I agree that it might make cython_special unnecessary, but if so it would have to stick around until whatever version of cython that is becomes the minimum supported, and through whatever deprecation period is needed.

8

There is one occurrence of cython_special.xlogy in scikit-learn. It would not be enough to access the ufunc because xlogy is used as part of a scalar kernel (poisson deviance of a decision tree node). I guess this is typical usage pattern, but I don’t know its spread.

Given the one occurrence that is trivial to replace, a deprecation would be ok for scikit-learn.

The new envisioned C++ library for the (scalar) special functions would be the right replacement, right?

9

@lorentzenchr, I think @izaid is asking if Cython could get an enhancement that makes calls to a ufunc in cdef functions which receive only C scalar arguments get compiled into calls to the scalar kernel. But yes, I think doing cdef extern fromthe new envisioned library might become the preferred way for calling scalar special functions from Cython.

1 Like