Skip to content

Expand ~ in file names, like pandas and xarray - #1035

Open
ecomodeller wants to merge 3 commits into
mainfrom
expand-tilde-in-filenames
Open

Expand ~ in file names, like pandas and xarray#1035
ecomodeller wants to merge 3 commits into
mainfrom
expand-tilde-in-filenames

Conversation

@ecomodeller

Copy link
Copy Markdown
Member

Problem

MIKE IO passes file names straight through to mikecore without expanding a leading ~, so ~/data/wl.dfs0 doesn't work. pandas and xarray both expand ~ on read and write, and MIKE IO aims to be consistent with them.

Reads fail with a FileNotFoundError naming the unexpanded path — annoying, but diagnosable:

>>> mikeio.read("~/data/random.dfs0")
FileNotFoundError: ~/data/random.dfs0

Writes are worse — they fail silently. Dataset.to_dfs creates a directory literally named ~ in the current working directory and writes the file there. Nothing lands in $HOME and no error is raised:

>>> ds.to_dfs("~/data/out.dfs0")   # no exception
>>> os.path.exists("~")            # a literal '~' directory, in the cwd
True

A ~ directory inside a repo is also a nuisance to clean up — rm -rf ~ is exactly the command you don't want someone reaching for.

The failure mode was inconsistent across formats, too: PfsDocument.write and Mesh.write use Python's open()/write_text() and so did raise FileNotFoundError, while the dfs writers went through mikecore and silently mis-wrote.

Fix

Add mikeio._path.normalize_path()expanduser(fspath(...)), mirroring pandas' stringify_path — and apply it at the public API boundaries:

  • mikeio.open / mikeio.read, via the Dfs0/Dfs1/Dfs2/Dfs3 constructors, _get_dfsu_info (Dfsu2DH, DfsuLayered, DfsuSpectral), the dfsu factory, and Mesh
  • Dataset.to_dfs and DataArray.to_dfs, plus write_dfs0/1/2/3 and write_dfsu
  • Mesh.write, GeometryFM2D.to_mesh, GeometryFM3D.to_mesh, Grid2D.to_mesh
  • read_pfs and PfsDocument.write
  • every generic.py function: scale, fill_corrupt, sum, add, diff, concat, extract, avg_time, quantile, change_datatype, transform
  • extract_track when given a file name

normalize_path is a no-op for paths without a leading ~ and is idempotent, so this is backwards-compatible. Because it returns a str, a number of now-redundant str(filename) wrappers are dropped.

Only expanduser is applied — not abspath. pandas does the same, and it keeps _filename (which shows up in repr) as the user wrote it.

Tests

New tests/test_path_expansion.py (35 tests) points $HOME at a tmp_path via monkeypatch and covers read, open, write, mesh, pfs, track, and every generic function. Each write test also asserts no literal ~ directory is created.

Verified these are not vacuous: 32 of the 35 fail on main.

Full suite: 878 passed. mypy clean, ruff check shows only the 3 pre-existing errors on main.

🤖 Generated with Claude Code

MIKE IO passed file names straight through to mikecore without expanding a
leading ~, so paths like "~/data/wl.dfs0" did not work. pandas and xarray
both expand ~ on read and write, and MIKE IO aims to be consistent with them.

Reads failed with a FileNotFoundError naming the unexpanded "~/..." path,
which was at least diagnosable. Writes were worse: Dataset.to_dfs() silently
created a directory literally named "~" in the current working directory and
wrote the file there, so nothing landed in the user's home directory and no
error was raised.

Add mikeio._path.normalize_path() and apply it at the public API boundaries:

- mikeio.open/read via the Dfs0/Dfs1/Dfs2/Dfs3 constructors, _get_dfsu_info
  (Dfsu2DH, DfsuLayered, DfsuSpectral), the dfsu factory and Mesh
- Dataset.to_dfs and DataArray.to_dfs, and the write_dfs0/1/2/3 and
  write_dfsu functions
- Mesh.write, GeometryFM2D.to_mesh, GeometryFM3D.to_mesh, Grid2D.to_mesh
- read_pfs and PfsDocument.write
- every generic.py function: scale, fill_corrupt, sum, add, diff, concat,
  extract, avg_time, quantile, change_datatype, transform
- extract_track when given a file name

normalize_path is expanduser(fspath(...)), so it is a no-op for paths without
a leading ~ and is idempotent. Since it returns a str, several now-redundant
str(filename) wrappers are dropped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ecomodeller

Copy link
Copy Markdown
Member Author

@copilot is this behaviour and implementation similar to pandas

Copilot AI commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

@copilot is this behaviour and implementation similar to pandas

Yes — this is aligned with pandas. pandas’ stringify_path path normalization expands ~ via os.path.expanduser (without converting to absolute paths), and our normalize_path does the same (os.fspath(...) + expanduser) for both read and write entry points.

@ecomodeller ecomodeller added the enhancement New feature or request label Aug 27, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants