Hi ,
I am currently trying to execute the solve method from Scipy.linalg by using OpenBLAS built from source.
Build steps followed:
Attempt 1 With building scipy:
Openblas:
-
Downloaded prebuilt openblas zip
-
Created a wheel file scipy_openblas32 with the following structure
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 11/12/2024 9:51 AM include
d----- 11/12/2024 9:54 AM lib
d----- 11/12/2024 9:51 AM __pycache__
-a---- 11/12/2024 10:19 AM 4589 __init__.py
-a---- 11/12/2024 8:01 AM 124 __main__.py
include directory consists of:
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 8/9/2024 11:44 AM 56088 cblas.h
-a---- 8/9/2024 11:44 AM 49852 f77blas.h
-a---- 8/9/2024 11:44 AM 747568 lapack.h
-a---- 8/9/2024 11:44 AM 853868 lapacke.h
-a---- 8/9/2024 11:44 AM 4858 lapacke_config.h
-a---- 8/9/2024 11:44 AM 474 lapacke_mangling.h
-a---- 8/9/2024 11:44 AM 35243 lapacke_utils.h
-a---- 8/9/2024 11:44 AM 4580 openblas_config.h
lib directory consists of :
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 11/12/2024 8:01 AM cmake
-a---- 8/9/2024 11:44 AM 50987903 libscipy_openblas.dll
-a---- 8/10/2024 11:42 AM 1694488 libscipy_openblas.lib
init.py and main.py taken from pypi version of scipy_openblas32
- Installed scipy-openblas32 wheel file inside scipy repository venv
4)Added openblas.pc package config file inside the lib/site-packages/scipy-openblas32 with the following content and added PKG-CONFIG-PATH pointing to this scipy-openblas32
libdir=C:/abc/scipy/venv/Lib/site-packages/scipy_openblas32/lib
includedir=C:/abc/scipy/venv/Lib/site-packages/scipy_openblas32/include
openblas_config=USE_64BITINT= NO_CBLAS= NO_LAPACK= NO_LAPACKE= DYNAMIC_ARCH=0 DYNAMIC_OLDER=OFF NO_AFFINITY=1 USE_OPENMP=0 ARMV8 MAX_THREADS=0
Name: OpenBLAS
Description: OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version
Version: 0.3.27.dev
URL: https://github.com/OpenMathLib/OpenBLAS
Libs: -L${libdir} -llibscipy_openblas
Cflags: -I${includedir}
4)Built scipy and generated wheel file for scipy from the build-install directory. installed scipy on the virtual environment
python dev.py build
5)On compilation
>>> import numpy as np
>>> from scipy.linalg import solve
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\abc\scipy\venv\Lib\site-packages\scipy\linalg\__init__.py", line 203, in <module>
from ._misc import *
File "C:\abc\scipy\venv\Lib\site-packages\scipy\linalg\_misc.py", line 3, in <module>
from .blas import get_blas_funcs
File "C:\abc\scipy\venv\Lib\site-packages\scipy\linalg\blas.py", line 213, in <module>
from scipy.linalg import _fblas
ImportError: DLL load failed while importing _fblas: The specified module could not be found.
>>>
KeyboardInterrupt
>>> ^Z
6)On benchmarking
(venv) PS C:\abc\scipy\benchmarks> asv run --dry-run --show-stderr --python=same --bench "linalg.Bench.time_solve"
Couldn't load asv.plugins._mamba_helpers because
No module named 'libmambapy'
· Unknown environment type 'mamba'. Allowed values based on existing plugins are ['existing', 'conda', 'virtualenv']. If you are trying to use `mamba`, you may need to install `conda-build`.
· Discovering benchmarks
·· Error running C:\abc\scipy\venv\Scripts\python.exe C:\abc\scipy\venv\Lib\site-packages\asv\benchmark.py discover C:\abc\scipy\benchmarks\benchmarks C:\Users\mcwsv\AppData\Local\Temp\tmpsn0d02hf\result.json (exit status 1)
STDOUT -------->
STDERR -------->
Traceback (most recent call last):
File "C:\abc\scipy\venv\Lib\site-packages\asv\benchmark.py", line 80, in <module>
main()
File "C:\abc\scipy\venv\Lib\site-packages\asv\benchmark.py", line 72, in main
commands[mode](args)
File "C:\abc\scipy\venv\Lib\site-packages\asv_runner\discovery.py", line 310, in _discover
list_benchmarks(benchmark_dir, fp)
File "C:\abc\scipy\venv\Lib\site-packages\asv_runner\discovery.py", line 278, in list_benchmarks
for benchmark in disc_benchmarks(root):
^^^^^^^^^^^^^^^^^^^^^
File "C:\abc\scipy\venv\Lib\site-packages\asv_runner\discovery.py", line 144, in disc_benchmarks
for module in disc_modules(root_name, ignore_import_errors=ignore_import_errors):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\abc\scipy\venv\Lib\site-packages\asv_runner\discovery.py", line 109, in disc_modules
yield from disc_modules(name, ignore_import_errors)
File "C:\abc\scipy\venv\Lib\site-packages\asv_runner\discovery.py", line 98, in disc_modules
module = importlib.import_module(module_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\mcwsv\AppData\Local\Programs\Python\Python312\Lib\importlib\__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 995, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "C:\abc\scipy\benchmarks\benchmarks\cython_special.py", line 3, in <module>
from scipy import special
File "C:\abc\scipy\venv\Lib\site-packages\scipy\__init__.py", line 134, in __getattr__
return _importlib.import_module(f'scipy.{name}')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\mcwsv\AppData\Local\Programs\Python\Python312\Lib\importlib\__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\abc\scipy\venv\Lib\site-packages\scipy\special\__init__.py", line 807, in <module>
from . import _ufuncs
ImportError: DLL load failed while importing _ufuncs: The specified module could not be found.
·· Failed to build the project and import the benchmark suite.
(venv) PS C:\abc\scipy\benchmarks>
Attempt 2 Without building scipy:
I uninstalled the default pip install scipy-openblas32 version of openblas
installed the wheel file i generated above and the python code was able to run the linalg.solve method.
Are there any build steps for generating the libs and dlls of openblas present here and integrating them with scipy for windows x64.
https://github.com/OpenMathLib/OpenBLAS/releases/tag/v0.3.28
I have seen the steps for arm64
(for which the ctests are failing and scipy.linalg’s solve method is getting timed out on benchmarking and infinite load on importing solve method from ‘scipy.linalg’ in a python program on arm64 device)
I cannot generate dlls and libs for x64 after modifying cmake flags the command executed
cmake .. -G Ninja -DCMAKE_C_COMPILER=clang -DNOFORTRAN=1 -DTARGET=generic -DARCH=x64 -DBINARY=64 -DUSE_OPENMP=0 -DCMAKE_SYSTEM_PROCESSOR=X64 -DCMAKE_SYSTEM_NAME=Windows -DBUILD_SHARED_LIBS=ON
Kindly help with the same