Skip to content

Run on numpy 2 and current TensorFlow, without changing what main's users get - #17

Merged
jries merged 4 commits into
mainfrom
claude/friendly-cerf-qt2sfj
Oct 7, 2026
Merged

jries merged 4 commits into
mainfrom
claude/friendly-cerf-qt2sfj

Conversation

@jries

@jries jries commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Makes uiPSF run on numpy 2 and current TensorFlow (tried: Python 3.12, numpy 2, TensorFlow 2.21, CPU, including Apple-silicon-capable versions), so it can be installed on current Macs and driven from SMAPpy (ries-lab/SMAPpy#5). The aim is to leave existing users exactly where they are: the same installs where they were pinned, and the same results.

Changes (11 files, +50 / −23)

  • io/h5.py: load reads the result with h5py instead of hdfdict, which does not import with numpy ≥ 2 (numpy.string_). It returns the same nested DottedDict, and closes the file.
  • numpy 2:
    • np.product → np.prod.
    • The fitter casts its target ROIs to float32. Under NEP 50, subtracting a numpy float64 scalar no longer keeps float32 arrays float32, and TensorFlow refuses the mixed dtypes.
    • The multi-channel PSFs' variable weights are float32.
    • iterlearn_psf takes element 0 of the one-element stage position before float().
  • tkinter: the unused imports are removed (every use was already commented out). tkinter is missing from many Python builds, and on macOS it crashes off the main thread.
  • setup.py: the TensorFlow requirement now depends on platform and Python version:
    • Python 3.7 (environment.yml): exactly tensorflow==2.9.1 and tensorflow-probability==0.17, as before.
    • Windows, Python 3.8–3.10: tensorflow<2.11 (2.10 is the last with native-Windows GPU support), tensorflow-probability<0.19, numpy<1.24.
    • Everything else (Linux, macOS, Windows on 3.11+, which TensorFlow 2.10 does not support): tensorflow>=2.9 and tensorflow-probability[tf]>=0.17. The [tf] extra brings tf-keras, which tensorflow-probability needs with Keras 3.
    • hdfdict is dropped; h5py and pillow are declared; find_packages() so a non-editable install includes the subpackages.

No parameter, config file or result format changes.

Checks

  • Same results on the old stack. Python 3.9 (no 3.7 could be installed in the test environment), TensorFlow 2.9.1, tensorflow-probability 0.17, numpy 1.23.5. main and this branch learnt the same three jobs from the same inputs, 15 iterations each: zernike_vector beads in one channel, in two channels, and insitu_zernike from blinking frames. Every array of every result file is bit-identical: I_model, zernike_coeff, pos, T, stagepos, zoffset, the fitted ROIs and the localizations. Reading each result back with io.h5.load (hdfdict on main, h5py here) gives the same entries, values and types.

  • What pip installs. Resolved with uv pip compile:

    Python, platform TensorFlow tensorflow-probability numpy
    3.7 Linux, Windows 2.9.1 0.17.0 1.21.6
    3.9 and 3.10 Windows 2.10.1 0.18.0 1.23.5
    3.11 Windows 2.21.0 0.25.0 2.4
    3.12 Linux, macOS (arm64) 2.21.0 0.25.0 2.5
  • Current stack, end to end. In a fresh Python 3.12 venv, pip install -e . gives a working uiPSF. Learning from simulated beads (one and two channels) and from simulated blinking frames runs end to end on TensorFlow 2.21. A spline fit with the learnt bead models recovers the simulated z with slope 1.03 (one channel) and 1.015 (two channels).

Not tried: Windows, GPUs, 4Pi, field-dependent, voxel and LLS models, and multi-channel in situ on the new stack.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GiNdULxyiqQEqWHGKTbVzV

claude added 4 commits October 6, 2026 08:55
- io.h5.load: read the result with h5py; hdfdict does not import with
  numpy >= 2 (numpy.string_)
- drop the unused tkinter imports: tkinter is missing from many Python
  builds and crashes when used off the main thread on macOS
- np.product -> np.prod (removed in numpy 2)
- Fitter: cast the target rois to float32; under NEP 50 subtracting a
  numpy float64 scalar no longer keeps them float32, and TensorFlow
  refuses the mixed dtypes
- iterlearn_psf: float() of a one-element stage position array
- setup.py: tensorflow>=2.9 instead of the 2.9.1 pin, h5py and pillow
  declared, subpackages included

Tested with Python 3.12, numpy 2, TensorFlow 2.21 (CPU): bead learning
(zernike_vector, single channel) and in situ learning run end to end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GiNdULxyiqQEqWHGKTbVzV
…hift

- PSFMultiChannel*: the variables' weights are float32; under numpy 2 a
  float64 numpy scalar times a float32 tensor is refused by TensorFlow
- channel_shift (config_base/config_user, default []): a known [y, x] shift
  per channel, target minus reference, used instead of the search from bead
  coordinates, which on a regular grid of beads can pair each bead with its
  neighbour

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GiNdULxyiqQEqWHGKTbVzV
…ing uiPSF

tensorflow-probability needs tf-keras with TensorFlow >= 2.16 (Keras 3); the
[tf] extra installs it.  Checked in a fresh Python 3.12 venv: pip install -e .
then learning a bead PSF end to end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GiNdULxyiqQEqWHGKTbVzV
…hannel_shift

- setup.py: Python 3.7 (environment.yml) installs exactly tensorflow 2.9.1
  and tensorflow-probability 0.17, as before; Windows on Python 3.8-3.10
  stays below TensorFlow 2.11, the last with native GPU support, with
  numpy < 1.24; only elsewhere (Linux, macOS, Windows on 3.11+, which
  TensorFlow 2.10 does not support) does it allow current TensorFlow
- channel_shift removed again (psflearninglib, config_base, config_user):
  smappy sets the shift from its own worker instead, so uiPSF needs no
  new parameter

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GiNdULxyiqQEqWHGKTbVzV
@jries jries changed the title Fix numpy 2.0+ compatibility and remove deprecated hdfdict dependency Run on numpy 2 and current TensorFlow, without changing what main's users get Oct 6, 2026
jries pushed a commit to ries-lab/SMAPpy that referenced this pull request Oct 7, 2026
The page's install line clones uiPSF's main rather than the branch of
ries-lab/uiPSF#17, which is to be merged first; NOTES.md speaks of #17
rather than of the branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GiNdULxyiqQEqWHGKTbVzV
@jries
jries merged commit 7677072 into main Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants