Idea: support virtual table of LAPACK functions

Open
#413 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
30/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Active
Tech stack
python, rust
Domain
hpc

Research direction

Start by reviewing the proposed lax::init_from_python(py) entry point, SciPy's scipy.linalg.cython_blas and scipy.linalg.cython_lapack interfaces, and the working approach linked in lenskit/lkpy#960. Define how a PyO3 module would obtain and install the function pointers so lax and ndarray-linalg can reuse SciPy's BLAS and LAPACK without a separate copy.

Written by the indexing model from the issue text.

Description

This might more properly belong in lapack-sys, but it would be very useful to be able to reuse BLAS and LAPACK from SciPy when using lax and ndarray-linalg to build Python extension modules with PyO3.

SciPy exposes routines from the BLAS and LAPACK it was linked against in a manner that can be imported with Cython's cimport, in the modules scipy.linalg.cython_blas and scipy.linalg.cython_lapack, to allow other Python extension modules implemented using Cython to reuse BLAS and LAPACK instead of needing to supply their own copy. I am not sure the best way to identify and properly link against these functions; I have one working method at lenskit/lkpy#960, fishing a pointer to sposv out of Python at runtime (the Cython modules expose them in a Python dictionary).

I imagine an interface where a PyO3 extension module can do something like use lax with a python feature, then call a method like lax::init_from_python(py) in a module initializer to grab the proper function pointers and set them up for lax to use, after which the lax methods (and therefore ndarray-linalg) will work without needing to compile or link against a separate BLAS and LAPACK.

In shared-library settings, like Pixi/Conda or system packages, all of the different modules can directly link against the same "system" BLAS. However, Python binary wheels do not make that easy, and the general solution is for each package to statically link against BLAS, or to use SciPy's export if it depends on SciPy.

Dominant language
Rust
Stars
452
Forks
95
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from rust-ndarray/ndarray-linalg

All issues in rust-ndarray/ndarray-linalg

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.