Repository navigation
Roadmap: Next milestone for numpy-quaddtype (post v1.0.0) #68
Description
For the next iteration we can focus on upgrading the dtype API at NumPy and other performance related issues. Listing down some known ones to track progress.
Dtype API upgrades (requires changes at NumPy)
- string parsing of user-defined dtypes
- Tracking Issues: ENH: add a hook to the DType API for parsing dtype strings numpy#30699
- PR:
- Exposing hook for
np.dot& fixingnp.einsumsupport for user-defined dtypes - Paths depending on
copyswaporcopyswapnleading to segfault - Using string form in array printing for user-defined dtypes
- Tracking Issue: QuadPrec scalar repr isn't ideal for display in arrays #29 , BUG: Array printing bypasses user dtype's formatting silently corrupts precision. numpy#30796
- PR:
- Extra context: Subclassing dtype to the native dtype (like
quaddtype<np.floating) seems to give good formatting but not upto the entire precision, that has to be explicitly set vianp.printoptions(...)which might not be ideal as we want the printed output to be in native precision (or any other properties) of the dtype by default
- Allowing
np.linalgsubmodule to allow user-defined dtypes routine registration- Tracking Issue:
- PR:
Updates/Fixes in numpy-quaddtype
- Supporting WASM build
- casting to quaddtype emits uneccesary warnings
- Tracking Issues: casting to
quaddtypeemits uneccesary warnings #25 - PR:
- Tracking Issues: casting to
Activity
SwayamInSync commentedon Jan 27, 2026
I think we can create separate tracking issues for the numpy API fixes in numpy repo and initiate some forward discussion on each and the potential fixes, so that contributors from both numpy and user-dtypes side can come forward as per the interests.
cc: @ngoldbaum @seberg
ngoldbaum commentedon Jan 27, 2026
Please do open numpy issues for improvements for the user dtype system.
mdhaber commentedon Feb 3, 2026
Thanks for working on this! I'm excited about the potential for use in SciPy, especially the elementwise iterative methods like
scipy.differentiate.derivativeandscipy.integrate.tanhsinh(quadrature), which are written in terms of the array API. How feasible is support throughout NumPy's array API functions? I thoughtdifferentiate.derivativemight "just work", but I ran into trouble with input validation -np.isdtype(..., "real floating")didn't recognize aQuadPrecDTypeyet. If it would be helpful, I can try to find pain points in some representative SciPy features.ngoldbaum commentedon Feb 3, 2026
That would be great, if only to figure out where to focus effort in NumPy.
I think there are open issues over there about better fleshing out DType hierarchies and adding ways to register dtypes in a hierarchy, but right now there isn't anything like that.
SwayamInSync commentedon Feb 3, 2026
Thanks @mdhaber for looking into it and definitely it'll be great to get know of more limitations.
I checked
isdtypeand the docstring stays "This function only supports built-in NumPy's data types". One recommendation might be here to useissubdtypebut I feel we should also update
isdtypefunction as well inside NumPymdhaber commentedon Feb 3, 2026
Ah, but that is not part of the array API. I could add a special case temporarily as a workaround, but it's best to minimize that.
mdhaber commentedon Feb 5, 2026
It was not too hard to get
scipy.differentiate.derivativeandscipy.integrate.tanhsinhworking with aQuadPrecDType; the only real incompatibility wasxp.isdtypein a few places (as discussed above). A few other things I noticed:strof a quad dtype does not (at least by default) use scientific notation? (e.g.print(np.finfo(QuadPrecDType()).smallest_normal)prints all ~5000 digits, mostly leading zeros.)np.log(np.asarray(1e-300, dtype=QuadPrecDType())**2))Otherwise, I think it will mostly "just work" with these and other features that could really benefit from very high working precision. Thanks!
SwayamInSync commentedon Feb 5, 2026
Thanks @mdhaber . This might be an important issue, I just checked and
np.linalgmodule only supports hardcoded_real_typesand_complex_typesand directly calls the LAPACK routines via_umath_linalggufuncs.This is inconsistent with how ufunc works, we need to make
_umath_linalggufuncs support the same loop registration for dtypes that regular ufuncs do. This might require big refactoring of gufunc infrastructure (but should be good plan for long term)mdhaber commentedon Feb 5, 2026
Yeah, I imagine linalg support is a tall order, but thought
solvewas worth mentioning. If we ever get a special function extension into the array API standard (data-apis/array-api#725), that would be the next thing I'd ask for.@steppi any chance that xsf could have quad precision implementations of some functions?
SwayamInSync commentedon Feb 5, 2026
I think linalg support is worth solving in NumPy as well, I will by this weekend raise all the found issues over NumPy and we can start working towards them.
The external implementations of routines might not work with our dtype as to ensure cross-compatibility the quaddtype is defined differently over different platforms but this won't be a pain point as its straightforward to implement such operations within the package and then register them in NumPy to use.
steppi commentedon Feb 5, 2026
From a numerical perspective, I know that the main thing we'd need to do that would take more than minimal work is to fit some quad specific mini-max polynomial approximations. We could use Boost's or Cephes' implementation of Remez-exchange for this. I don't know enough about
numpy-quaddtypeyet to know how defining ufuncs would work for the quad dtype, but expanding dtype support inxsfby fitting more minimax approximations is planned.juntyr commentedon Sep 8, 2026
Now that PyEmscripten packages can be published to PyPi, it would also be good to publish them as part of the next release
ngoldbaum commentedon Sep 8, 2026
Is it helpful to do that if NumPy itself doesn't do that yet?
juntyr commentedon Sep 9, 2026
Do you know why NumPy doesn't publish PyEmscripten wheels yet?
I still think that it is useful. Pyodide still provides NumPy wheels by default since it provides a special build env for downstream packages that link to numpy. But Pyodide now also doesn't really want to accept new recipes that they need to maintain. So I think publishing numpy-quaddtype wheels would be useful.