ANN: SciPy 1.16.0rc2 -- please test

Announcements
1

Hi all,

On behalf of the SciPy development team, I’m pleased to announce the pre-release SciPy 1.16.0rc2 – please help us test this version.

Sources and binary wheels can be found at:
https://github.com/scipy/scipy/releases/tag/v1.16.0rc2
and at:
https://pypi.org/project/scipy/1.16.0rc2/

One of a few ways to install this release with pip:
pip install scipy==1.16.0rc2

SciPy 1.16.0 Release Notes

Note: SciPy 1.16.0 is not released yet!

SciPy 1.16.0 is the culmination of 6 months of hard work. It contains
many new features, numerous bug-fixes, improved test coverage and better
documentation. There have been a number of deprecations and API changes
in this release, which are documented below. All users are encouraged to
upgrade to this release, as there are a large number of bug-fixes and
optimizations. Before upgrading, we recommend that users check that
their own code does not use deprecated SciPy functionality (to do so,
run your code with python -Wd and check for DeprecationWarning s).
Our development attention will now shift to bug-fix releases on the
1.16.x branch, and on adding new features on the main branch.

This release requires Python 3.11-3.13 and NumPy 1.25.2 or greater.

Highlights of this release

  • Improved experimental support for the Python array API standard, including
    new support in scipy.signal, and additional support in scipy.stats and
    scipy.special. Improved support for JAX and Dask backends has been added,
    with notable support in scipy.cluster.hierarchy, many functions in
    scipy.special, and many of the trimmed statistics functions.
  • scipy.optimize now uses the new Python implementation from the PRIMA package for COBYLA.
    The PRIMA implementation fixes many bugs in the old Fortran 77 implementation with
    a better performance on average.
  • scipy.sparse.coo_array now supports n-D arrays with reshaping, arithmetic and
    reduction operations like sum/mean/min/max. No n-D indexing or
    scipy.sparse.random_array support yet.
  • Updated guide and tools for migration from sparse matrices to sparse arrays.
  • Nearly all functions in the scipy.linalg namespace that accept array
    arguments now support N-dimensional arrays to be processed as a batch.
  • Two new scipy.signal functions, scipy.signal.firwin_2d and
    scipy.signal.closest_STFT_dual_window, for creation of a 2-D FIR filter and
    scipy.signal.ShortTimeFFT dual window calculation, respectively.
  • A new class, scipy.spatial.transform.RigidTransform, provides functionality
    to convert between different representations of rigid transforms in 3-D
    space.

New features

scipy.io improvements

  • scipy.io.savemat now provides informative warnings for invalid field names.
  • scipy.io.mmread now provides a clearer error message when provided with
    a source file path that does not exist.
  • scipy.io.wavfile.read can now read non-seekable files.

scipy.integrate improvements

  • The error estimate of scipy.integrate.tanhsinh was improved.

scipy.interpolate improvements

  • Batch support was added to scipy.interpolate.make_smoothing_spline.

scipy.linalg improvements

  • Nearly all functions in the scipy.linalg namespace that accept array
    arguments now support N-dimensional arrays to be processed as a batch.
    See :ref:linalg_batch for details.
  • scipy.linalg.sqrtm is rewritten in C and its performance is improved. It
    also tries harder to return real-valued results for real-valued inputs if
    possible. See the function docstring for more details. In this version the
    input argument disp and the optional output argument errest are
    deprecated and will be removed four versions later. Similarly, after
    changing the underlying algorithm to recursion, the blocksize keyword
    argument has no effect and will be removed two versions later.
  • Wrappers for ?stevd, ?langb, ?sytri, ?hetri and
    ?gbcon were added to scipy.linalg.lapack.
  • The default driver of scipy.linalg.eigh_tridiagonal was improved.
  • scipy.linalg.solve can now estimate the reciprocal condition number and
    the matrix norm calculation is more efficient.

scipy.ndimage improvements

  • A new function scipy.ndimage.vectorized_filter for generic filters that
    take advantage of a vectorized Python callable was added.
  • scipy.ndimage.rotate has improved performance, especially on ARM platforms.

scipy.optimize improvements

  • COBYLA was updated to use the new Python implementation from the
    PRIMA package.
    The PRIMA implementation fixes many bugs
    in the old Fortran 77 implementation. In addition, it results in fewer function evaluations on average,
    but it depends on the problem and for some
    problems it can result in more function evaluations or a less optimal
    result. For those cases the user can try modifying the initial and final
    trust region radii given by rhobeg and tol respectively. A larger
    rhobeg can help the algorithm take bigger steps initially, while a
    smaller tol can help it continue and find a better solution.
    For more information, see the PRIMA documentation.
  • Several of the scipy.optimize.minimize methods, and the
    scipy.optimize.least_squares function, have been given a workers
    keyword. This allows parallelization of some calculations via a map-like
    callable, such as multiprocessing.Pool. These parallelization
    opportunities typically occur during numerical differentiation. This can
    greatly speed up minimization when the objective function is expensive to
    calculate.
  • The lm method of scipy.optimize.least_squares can now accept
    3-point and cs for the jac keyword.
  • The SLSQP Fortran 77 code was ported to C. When this method is used now the
    constraint multipliers are exposed to the user through the multiplier
    keyword of the returned scipy.optimize.OptimizeResult object.
  • NNLS code has been corrected and rewritten in C to address the performance
    regression introduced in 1.15.x
  • scipy.optimize.root now warns for invalid inner parameters when using the
    newton_krylov method
  • The return value of minimization with method='L-BFGS-B' now has
    a faster hess_inv.todense() implementation. Time complexity has improved
    from cubic to quadratic.
  • scipy.optimize.least_squares has a new callback argument that is applicable
    to the trf and dogbox methods. callback may be used to track
    optimization results at each step or to provide custom conditions for
    stopping.

scipy.signal improvements

  • A new function scipy.signal.firwin_2d for the creation of a 2-D FIR Filter
    using the 1-D window method was added.
  • scipy.signal.cspline1d_eval and scipy.signal.qspline1d_eval now provide
    an informative error on empty input rather than hitting the recursion limit.
  • A new function scipy.signal.closest_STFT_dual_window to calculate the
    scipy.signal.ShortTimeFFT dual window of a given window closest to a
    desired dual window.
  • A new classmethod scipy.signal.ShortTimeFFT.from_win_equals_dual to
    create a scipy.signal.ShortTimeFFT instance where the window and its dual
    are equal up to a scaling factor. It allows to create short-time Fourier
    transforms which are unitary mappings.
  • The performance of scipy.signal.convolve2d was improved.

scipy.sparse improvements

  • scipy.sparse.coo_array now supports n-D arrays using binary and reduction
    operations.
  • Faster operations between two DIA arrays/matrices for: add, sub, multiply,
    matmul.
  • scipy.sparse.csgraph.dijkstra shortest_path is more efficient.
  • scipy.sparse.csgraph.yen has performance improvements.
  • Support for lazy loading of sparse.csgraph and sparse.linalg was
    added.

scipy.spatial improvements

  • A new class, scipy.spatial.transform.RigidTransform, provides functionality
    to convert between different representations of rigid transforms in 3-D
    space, its application to vectors and transform composition.
    It follows the same design approach as scipy.spatial.transform.Rotation.
  • ~scipy.spatial.transform.Rotation now has an appropriate __repr__ method,
    and improved performance for its scipy.spatial.transform.Rotation.apply
    method.

scipy.stats improvements

  • A new function scipy.stats.quantile, an array API compatible function for
    quantile estimation, was added.
  • scipy.stats.make_distribution was extended to work with existing discrete
    distributions and to facilitate the creation of custom distributions in the
    new random variable infrastructure.
  • A new distribution, scipy.stats.Binomial, was added.
  • An equal_var keyword was added to scipy.stats.tukey_hsd (enables the
    Games-Howell test) and scipy.stats.f_oneway (enables Welch ANOVA).
  • The moment calculation for scipy.stats.gennorm was improved.
  • The scipy.stats.mode implementation was vectorized, for faster batch
    calculation.
  • Support for axis, nan_policy, and keepdims keywords was added to
    scipy.stats.power_divergence, scipy.stats.chisquare,
    scipy.stats.pointbiserialr, scipy.stats.kendalltau,
    scipy.stats.weightedtau, scipy.stats.theilslopes,
    scipy.stats.siegelslopes, scipy.stats.boxcox_llf, and
    scipy.stats.linregress.
  • Support for keepdims and nan_policy keywords was added to
    scipy.stats.gstd.
  • The performance of scipy.stats.special_ortho_group and scipy.stats.pearsonr
    was improved.
  • Support for an rng keyword argument was added to the logcdf and
    cdf methods of multivariate_normal_gen and multivariate_normal_frozen.

Array API Standard Support

Experimental support for array libraries other than NumPy has been added to
multiple submodules in recent versions of SciPy. Please consider testing
these features by setting the environment variable SCIPY_ARRAY_API=1 and
providing PyTorch, JAX, CuPy or Dask arrays as array arguments.

Many functions in scipy.stats, scipy.special, scipy.optimize, and
scipy.constants now provide tables documenting compatible array and device
types as well as support for lazy arrays and JIT compilation. New features with
support and old features with support added for SciPy 1.16.0 include:

  • Most of the scipy.signal functionality
  • scipy.ndimage.vectorized_filter
  • scipy.special.stdtrit
  • scipy.special.softmax
  • scipy.special.log_softmax
  • scipy.stats.quantile
  • scipy.stats.gstd
  • scipy.stats.rankdata

Features with extended array API support (generally, improved support
for JAX and Dask) in SciPy 1.16.0 include:

  • many of the scipy.cluster.hierarchy functions
  • many functions in scipy.special
  • many of the trimmed statistics functions in scipy.stats

SciPy now has a CI job that exercises GPU (CUDA) support, and as a result
using PyTorch, CuPy or JAX arrays on GPU with SciPy is now more reliable.

Deprecated features

  • The unused atol argument of scipy.optimize.nnls is deprecated and will
    be removed in SciPy 1.18.0.
  • The disp argument of scipy.linalg.signm, scipy.linalg.logm, and
    scipy.linalg.sqrtm will be removed in SciPy 1.18.0.
  • scipy.stats.multinomial now emits a FutureWarning if the rows of p
    do not sum to 1.0. This condition will produce NaNs beginning in SciPy
    1.18.0.

Expired Deprecations

  • scipy.sparse.conjtransp has been removed. Use .T.conj() instead.
  • The quadrature='trapz' option has been removed from
    scipy.integrate.quad_vec, and scipy.stats.trapz has been removed. Use
    trapezoid in both instances instead.
  • scipy.special.comb and scipy.special.perm now raise when exact=True
    and arguments are non-integral.
  • Support for inference of the two sets of measurements from the single
    argument x has been removed from scipy.stats.linregress. The data
    must be specified separately as x and y.
  • Support for NumPy masked arrays has been removed from
    scipy.stats.power_divergence and scipy.stats.chisquare.
  • A significant number of functions from non-public namespaces
    (e.g., scipy.sparse.base, scipy.interpolate.dfitpack) were cleaned
    up. They were previously already emitting deprecation warnings.

Backwards incompatible changes

  • Several of the scipy.linalg functions for solving a linear system (e.g.
    scipy.linalg.solve) documented that the RHS argument must be either 1-D or
    2-D but did not always raise an error when the RHS argument had more the
    two dimensions. Now, many-dimensional right hand sides are treated according
    to the rules specified in :ref:linalg_batch.
  • scipy.stats.bootstrap now explicitly broadcasts elements of data to the
    same shape (ignoring axis) before performing the calculation.
  • Several submodule names are no longer available via from scipy.signal import *,
    but may still be imported directly, as detailed at 👽️ scipy 1.16 changes for `signal.__all__` by jorenham · Pull Request #549 · scipy/scipy-stubs · GitHub.

Build and packaging related changes

  • The minimum supported version of Clang was bumped from 12.0 to 15.0.
  • The lowest supported macOS version for wheels on PyPI is now 10.14 instead of
    10.13.
  • The sdist contents were optimized, resulting in a size reduction of about 50%,
    from 60 MB to 30 MB.
  • For Cython>=3.1.0, SciPy now uses the new cython --generate-shared
    functionality, which reduces the total size of SciPy’s wheels and on-disk
    installations significantly.
  • SciPy no longer contains an internal shared library that requires RPATH support,
    after sf_error_state was removed from scipy.special.
  • A new build option -Duse-system-libraries has been added. It allows
    opting in to using system libraries instead of using vendored sources.
    Currently Boost.Math and Qhull are supported as system build
    dependencies.

Other changes

  • The internal dependency of scipy._lib on scipy.sparse was removed,
    which reduces the import time of a number of other SciPy submodules.
  • Support for free-threaded CPython was improved: the last known thread-safety
    issues in scipy.special were fixed, and pytest-run-parallel is now used
    in a CI job to guard against regressions.
  • Support for spin as a developer
    CLI was added, including support for editable installs. The SciPy-specific
    python dev.py CLI will be removed in the next release cycle in favor of
    spin.
  • The vendored Qhull library was upgraded from version 2019.1 to 2020.2.
  • A large amount of the C++ code in scipy.special was moved to the new
    header-only xsf library. That library was
    included back in the SciPy source tree as a git submodule.
  • The namedtuple-like bunch objects returned by some SciPy functions
    now have improved compatibility with the polars library.
  • The output of the rvs method of scipy.stats.wrapcauchy is now mapped to
    the unit circle between 0 and 2 * pi.
  • The lm method of scipy.optimize.least_squares now has a different behavior
    for the maximum number of function evaluations, max_nfev. The default for
    the lm method is changed to 100 * n, for both a callable and a
    numerically estimated jacobian. This limit on function evaluations excludes
    those used for any numerical estimation of the Jacobian. Previously the
    default when using an estimated jacobian was 100 * n * (n + 1), because
    the method included evaluations used in the estimation. In addition, for the
    lm method the number of function calls used in Jacobian approximation
    is no longer included in OptimizeResult.nfev. This brings the behavior
    of lm, trf, and dogbox into line.

Authors

  • Name (commits)
  • h-vetinari (4)
  • aiudirog (1) +
  • Anton Akhmerov (2)
  • Thorsten Alteholz (1) +
  • Gabriel Augusto (1) +
  • Backfisch263 (1) +
  • Nickolai Belakovski (5)
  • Peter Bell (1)
  • Benoît W. (1) +
  • Evandro Bernardes (1)
  • Gauthier Berthomieu (1) +
  • Maxwell Bileschi (1) +
  • Sam Birch (1) +
  • Florian Bourgey (3) +
  • Charles Bousseau (2) +
  • Richard Strong Bowen (2) +
  • Jake Bowhay (126)
  • Matthew Brett (1)
  • Dietrich Brunn (53)
  • Evgeni Burovski (252)
  • Christine P. Chai (12) +
  • Gayatri Chakkithara (1) +
  • Saransh Chopra (2) +
  • Omer Cohen (1) +
  • Lucas Colley (91)
  • Yahya Darman (3) +
  • Benjamin Eisele (1) +
  • Donnie Erb (1)
  • Sagi Ezri (58) +
  • Alexander Fabisch (2) +
  • Matthew H Flamm (1)
  • Karthik Viswanath Ganti (1) +
  • Neil Girdhar (1)
  • Ralf Gommers (158)
  • Rohit Goswami (4)
  • Saarthak Gupta (4) +
  • Matt Haberland (325)
  • Sasha Hafner (1) +
  • Joren Hammudoglu (9)
  • Chengyu Han (1) +
  • Charles Harris (1)
  • Kim Hsieh (4) +
  • Yongcai Huang (2) +
  • Lukas Huber (1) +
  • Yuji Ikeda (2) +
  • Guido Imperiale (104) +
  • Robert Kern (2)
  • Harin Khakhi (2) +
  • Agriya Khetarpal (4)
  • Kirill R. (2) +
  • Tetsuo Koyama (1)
  • Jigyasu Krishnan (1) +
  • Abhishek Kumar (2) +
  • Pratham Kumar (3) +
  • David Kun (1) +
  • Eric Larson (3)
  • lciti (1)
  • Antony Lee (1)
  • Kieran Leschinski (1) +
  • Thomas Li (2) +
  • Christian Lorentzen (2)
  • Loïc Estève (4)
  • Panos Mavrogiorgos (1) +
  • Nikolay Mayorov (2)
  • Melissa Weber Mendonça (10)
  • Michał Górny (1)
  • Miguel Cárdenas (2) +
  • Swastik Mishra (1) +
  • Sturla Molden (2)
  • Andreas Nazlidis (1) +
  • Andrew Nelson (209)
  • Parth Nobel (1) +
  • Nick ODell (9)
  • Giacomo Petrillo (1)
  • Victor PM (10) +
  • pmav99 (1) +
  • Ilhan Polat (73)
  • Tyler Reddy (96)
  • Érico Nogueira Rolim (1) +
  • Pamphile Roy (10)
  • Mikhail Ryazanov (6)
  • Atsushi Sakai (9)
  • Marco Salathe (1) +
  • sanvi (1) +
  • Neil Schemenauer (2) +
  • Daniel Schmitz (20)
  • Martin Schuck (1) +
  • Dan Schult (33)
  • Tomer Sery (19)
  • Adrian Seyboldt (1) +
  • Scott Shambaugh (4)
  • ShannonS00 (1) +
  • sildater (3) +
  • Param Singh (1) +
  • G Sreeja (7) +
  • Albert Steppi (133)
  • Kai Striega (3)
  • Anushka Suyal (2)
  • Julia Tatz (1) +
  • Tearyt (1) +
  • Elia Tomasi (1) +
  • Jamie Townsend (2) +
  • Edgar Andrés Margffoy Tuay (4)
  • Matthias Urlichs (1) +
  • Mark van Rossum (1) +
  • Jacob Vanderplas (2)
  • David Varela (2) +
  • Christian Veenhuis (3)
  • vfdev (1)
  • Stefan van der Walt (2)
  • Warren Weckesser (5)
  • Jason N. White (1) +
  • windows-server-2003 (5)
  • Zhiqing Xiao (1)
  • Pavadol Yamsiri (1)
  • Rory Yorke (3)
  • Irwin Zaid (4)
  • Austin Zhang (1) +
  • William Zijie Zhang (1) +
  • Zaikun Zhang (1) +
  • Zhenyu Zhu (1) +
  • Eric Zitong Zhou (11) +
  • Case Zumbrum (2) +
  • ਗਗਨਦੀਪ ਸਿੰਘ (Gagandeep Singh) (45)

A total of 124 people contributed to this release.
People with a “+” by their names contributed a patch for the first time.
This list of names is automatically generated, and may not be fully complete.

The complete issue list, PR list, and list of release asset hashes are available in the README.txt release artifact.

1 Like