Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Build the docs on every PR, and publish them to GitHub Pages on merge to main.
name: docs

on:
push:
branches: [main]
pull_request:

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: astral-sh/setup-uv@v10
- run: uv run --python 3.12 --extra docs sphinx-build -W --keep-going docs docs/_build/html
- uses: actions/upload-pages-artifact@v5
with:
path: docs/_build/html

deploy:
if: github.ref == 'refs/heads/main'
needs: build
runs-on: ubuntu-latest
permissions:
pages: write
id-token: write
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- id: deployment
uses: actions/deploy-pages@v5
42 changes: 42 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Publish to PyPI on every merge to main that bumps the version in pyproject.toml.
# Merges that leave the version unchanged are skipped, since PyPI never
# accepts the same version twice.
name: publish

on:
push:
branches: [main]

jobs:
publish:
runs-on: ubuntu-latest
environment: pypi
permissions:
id-token: write # PyPI trusted publishing, no token needed
contents: write # tag and GitHub release
steps:
- uses: actions/checkout@v7
- uses: astral-sh/setup-uv@v10

- name: Check whether this version is new
id: version
run: |
v=$(python3 -c "import tomllib; print(tomllib.load(open('pyproject.toml', 'rb'))['project']['version'])")
echo "version=$v" >> "$GITHUB_OUTPUT"
if curl -sfo /dev/null "https://pypi.org/pypi/evstats/$v/json"; then
echo "evstats $v is already on PyPI, skipping"
echo "new=false" >> "$GITHUB_OUTPUT"
else
echo "new=true" >> "$GITHUB_OUTPUT"
fi

- if: steps.version.outputs.new == 'true'
run: uv build

- if: steps.version.outputs.new == 'true'
run: uv publish

- if: steps.version.outputs.new == 'true'
run: gh release create "v${{ steps.version.outputs.version }}" dist/* --generate-notes
env:
GH_TOKEN: ${{ github.token }}
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ example/paper/plots/*.pdf
example/paper/plots/*.gif
example/paper/plots/*.png
example/observables/data/*.txt
# fiducial-model flux tables, used in the docs
!example/observables/data/flux_per_mass_*_Exponential_tau-0.1_bpass-2.2.1-bin_chabrier03-0.1,300.0_cloudy-c23.01-sps.txt

# Byte-compiled / optimized / DLL files
__pycache__/
Expand Down
43 changes: 6 additions & 37 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# evstats
A python module for calculating extreme value statistics of the halo and galaxy stellar mass distributions. Full details provided in [Lovell et al. 2023](https://academic.oup.com/mnras/article/518/2/2511/6823705).
A python package for extreme value statistics (EVS) in astrophysics: the probability distribution of the most extreme object, such as the most massive or the brightest, expected in a survey of a given area and redshift range. EVS applies to physical distribution functions, such as the halo and galaxy stellar mass functions, and to observed ones, such as the distribution of galaxy fluxes in a given band. An observed object far beyond this distribution is in tension with the model behind it. The method is described in [Lovell et al. 2023](https://academic.oup.com/mnras/article/518/2/2511/6823705).

<img src="https://www.christopherlovell.co.uk/assets/img/evs-1400.webp" alt="drawing" width="400"/>

Expand All @@ -9,46 +9,15 @@ A python module for calculating extreme value statistics of the halo and galaxy
pip install evstats
```

or, from a clone of this repository, `pip install .`. You can then use evstats as so:
evstats requires Python 3.12 or newer. Optional dependencies are installed as extras:

```
from evstats import evs
from evstats import stats
from evstats import stellar
pip install "evstats[examples]" # matplotlib and h5py, for plotting and the examples
pip install "evstats[observables]" # also Synthesizer and unyt, for making new flux tables
```

evstats requires Python 3.12 or newer. The core package needs only numpy, scipy, astropy and [hmf](https://github.com/halomod/hmf). hmf is held at 3.6, since other versions shift the halo mass function and the tabulated EVS in `example/data/` were computed with it.

Optional dependencies are installed as extras:

```
pip install "evstats[examples]" # matplotlib and h5py: plotting, and the scripts in example/
pip install "evstats[observables]" # also Synthesizer and unyt: the photometry in example/observables/
```

The star-formation-history module (`evstats.sfr`) provides
`mass_growth_track` / `plot_mass_growth_track`, which project an observed
stellar mass back in redshift for an assumed (parametric or binned) star
formation history, for comparison against the EVS contours. Plotting needs
the `examples` extra, and `average_sfr_over_window` needs the `observables` extra.

### An example
A notebook showing a simple example of how to create contours in the stellar mass -- redshift plane, for arbitrary survey areas, is available [here](https://nbviewer.org/github/christopherlovell/evstats/blob/main/example/example.ipynb).

### Observables
The scripts in `example/observables/` turn the stellar mass EVS into JWST fluxes using [Synthesizer](https://github.com/synthesizer-project/synthesizer), so need the `observables` extra. They read flux tables that are not distributed with this repository; to make them:

1. Get the Synthesizer grid for the fiducial model, `bpass-2.2.1-bin_chabrier03-0.1,300.0_cloudy-c23.01-sps`, and set `GRID_DIR` in `evs_file_generator.py` to the directory containing it.
2. From `example/observables/`, generate the flux tables for every star formation history with that grid, at the fiducial metallicity:
```
python evs_file_generator.py "" bpass-2.2.1-bin_chabrier03-0.1,300.0_cloudy 0.01
```
3. Plot the EVS flux predictions for each star formation history:
```
python evs_sfh.py
```

Running `evs_file_generator.py` with no arguments makes the tables for every SFH, SPS model and metallicity, which the other scripts need; this is slow.
### Documentation
Worked examples for the halo mass, stellar mass and flux distributions, and the API reference, are at https://www.christopherlovell.co.uk/evstats/.

### The paper
All of the plots and analysis in [Lovell et al. 2023](https://academic.oup.com/mnras/article/518/2/2511/6823705) can be recreated using the scripts in `example/paper/`.
20 changes: 20 additions & 0 deletions docs/api.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
API
===

evstats.evs
-----------

.. automodule:: evstats.evs
:members:

evstats.stats
-------------

.. automodule:: evstats.stats
:members:

evstats.stellar
---------------

.. automodule:: evstats.stellar
:members:
21 changes: 21 additions & 0 deletions docs/conf.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
project = "evstats"
author = "Chris Lovell"
copyright = "2023, Chris Lovell"

extensions = ["myst_nb", "sphinx.ext.autodoc", "sphinx.ext.napoleon"]
exclude_patterns = ["_build"]
myst_enable_extensions = ["dollarmath"]

# Run the notebooks on every build, so the docs fail if the examples break
nb_execution_mode = "force"
nb_execution_raise_on_error = True
nb_execution_timeout = 600

autodoc_member_order = "bysource"

html_theme = "shibuya"
html_title = "evstats"
html_theme_options = {
"github_url": "https://github.com/christopherlovell/evstats",
"show_ai_links": False,
}
148 changes: 148 additions & 0 deletions docs/halo.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "be82c7f7",
"metadata": {},
"source": [
"# Halo mass EVS\n",
"\n",
"For $N$ haloes drawn independently from a mass distribution with PDF $f(m)$ and CDF $F(m)$, the PDF of the most massive is\n",
"\n",
"$$\\Phi(m_{\\rm max}) = N \\, f(m_{\\rm max}) \\, F(m_{\\rm max})^{N-1}.$$\n",
"\n",
"evstats finds $N$, $f$ and $F$ by integrating a halo mass function from [hmf](https://github.com/halomod/hmf) over the comoving volume of a redshift shell, for a survey covering a fraction $f_{\\rm sky}$ of the sky. Masses are in $M_\\odot$, without factors of $h$."
]
},
{
"cell_type": "markdown",
"id": "cc345578",
"metadata": {},
"source": [
"## A single redshift shell\n",
"\n",
"`evs_bin_pdf` computes the PDF for one shell, here $4 < z < 4.4$ over the whole sky ($f_{\\rm sky} = 1$). The `hmf.MassFunction` sets the mass function fit and cosmology; here the Behroozi et al. (2013) fit and Planck 2015 cosmology of Lovell et al. (2023). `dz` and `dm` set the resolution of the redshift and $\\log_{10}$ mass grids used for the integration."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "577bca21",
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import h5py\n",
"import matplotlib.pyplot as plt\n",
"import astropy.units as u\n",
"from astropy.cosmology import Planck15\n",
"import hmf\n",
"\n",
"from evstats import evs\n",
"from evstats.stats import compute_conf_ints"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4b1c3ac8",
"metadata": {},
"outputs": [],
"source": [
"# True is the hmf default; set here to silence its warning\n",
"mf = hmf.MassFunction(hmf_model='Behroozi', cosmo_model=Planck15,\n",
" transfer_params={'extrapolate_with_eh': True})\n",
"\n",
"pdf, log10m = evs.evs_bin_pdf(mf=mf, zmin=4.0, zmax=4.4, dz=0.01,\n",
" mmin=11, mmax=16, dm=0.01, fsky=1.0)\n",
"\n",
"fig, ax = plt.subplots()\n",
"ax.plot(log10m, pdf)\n",
"ax.set_xlim(13.5, 14)\n",
"ax.set_xlabel(r'$\\log_{10}(M^{200}_{\\rm max} \\,/\\, M_\\odot)$')\n",
"ax.set_ylabel(r'$\\Phi(M^{200}_{\\rm max})$');"
]
},
{
"cell_type": "markdown",
"id": "e4398ce9",
"metadata": {},
"source": [
"## Any survey area, across redshift\n",
"\n",
"The sky fraction only enters by scaling the whole-sky $N$, $f$ and $F$, which `apply_fsky` does. So these terms can be tabulated once, in contiguous redshift shells, and then give the PDF for any survey area almost instantly. `example/data/evs_all.h5` holds them for 140 shells of width $\\Delta z = 0.2$ over $0 < z < 28$, for the same mass function and cosmology as above. Here we take a 90 arcmin$^2$ survey."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5035da94",
"metadata": {},
"outputs": [],
"source": [
"with h5py.File('../example/data/evs_all.h5', 'r') as hf:\n",
" log10m = hf['log10m'][:]\n",
" f = hf['f'][:]\n",
" F = hf['F'][:]\n",
" N = hf['N'][:]\n",
" z = hf['z'][:]\n",
"\n",
"fsky = float(90 * u.arcmin**2 / (4 * np.pi * u.sr))\n",
"phi_max = evs.apply_fsky(N, f, F, fsky)"
]
},
{
"cell_type": "markdown",
"id": "8d3a6e8d",
"metadata": {},
"source": [
"`compute_conf_ints` gives, at each redshift, the masses at the $-3\\sigma$, $-2\\sigma$, $-1\\sigma$, median, $+1\\sigma$, $+2\\sigma$ and $+3\\sigma$ points of the CDF."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ca24a370",
"metadata": {},
"outputs": [],
"source": [
"CI = compute_conf_ints(phi_max, log10m)\n",
"\n",
"fig, ax = plt.subplots(figsize=(5, 5))\n",
"for i, c in enumerate(['brown', 'lightcoral', 'mistyrose']):\n",
" ax.fill_between(z, CI[:, i], CI[:, 6 - i], color=c, label=rf'${3 - i}\\sigma$')\n",
"ax.plot(z, CI[:, 3], ':k', label='median')\n",
"ax.set_xlim(0.1, 19.8)\n",
"ax.set_xlabel('$z$')\n",
"ax.set_ylabel(r'$\\log_{10}(M^{200}_{\\rm max} \\,/\\, M_\\odot)$')\n",
"ax.legend(frameon=False, title=r'$A = 90\\ \\mathrm{arcmin}^2$');"
]
},
{
"cell_type": "markdown",
"id": "87ecc841",
"metadata": {},
"source": [
"## Other mass functions and cosmologies\n",
"\n",
"To use a different mass function fit or cosmology, regenerate the table with `example/run_lightcone.py`. It writes the same HDF5 format to `example/data/`, which can then be used as above:\n",
"\n",
"```bash\n",
"cd example\n",
"python run_lightcone.py -o my_evs.h5 --procs 8\n",
"```\n",
"\n",
"`--model` takes a preset (`behroozi`, the default, or `yung` for the Yung et al. 2024 fit) or any hmf model name, and `--cosmology` any astropy cosmology name; see `python run_lightcone.py --help` for the grid options. The table above was made with hmf 3.6; other hmf versions shift the mass function by up to ~0.2 dex at high redshift."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
51 changes: 51 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# evstats

evstats is a python package for extreme value statistics (EVS) in astrophysics: the probability distribution of the most extreme object, such as the most massive or the brightest, expected in a survey of a given area and redshift range. EVS applies to physical distribution functions, such as the halo and galaxy stellar mass functions, and to observed ones, such as the distribution of galaxy fluxes in a given band. An observed object far beyond this distribution is in tension with the model behind it. The method is described in [Lovell et al. 2023](https://academic.oup.com/mnras/article/518/2/2511/6823705).

```{image} https://www.christopherlovell.co.uk/assets/img/evs-1400.webp
:width: 400px
```

## Installation

```bash
pip install evstats
```

evstats requires Python 3.12 or newer. The core package needs only numpy, scipy, astropy and [hmf](https://github.com/halomod/hmf). hmf is held at 3.6, since other versions shift the halo mass function and the tabulated EVS in `example/data/` were computed with it.

Optional dependencies are installed as extras:

```bash
pip install "evstats[examples]" # matplotlib and h5py
pip install "evstats[observables]" # also Synthesizer and unyt
```

The examples in these pages need the `examples` extra, and making new flux tables needs `observables`. The examples also read tabulated data from the `example/` directory of the repository, which is not part of the pip package; to run them, clone the repository and run the notebooks from `docs/`:

```bash
git clone https://github.com/christopherlovell/evstats
cd evstats
pip install ".[examples]"
```

## Overview

1. [Halo mass EVS](halo.ipynb): the PDF of the most massive halo in a redshift shell and survey area, from the halo mass function.
2. [Stellar mass EVS](stellar.ipynb): the PDF of the most massive galaxy's stellar mass, from the baryon fraction and a distribution of stellar fractions.
3. [Observables](observables.ipynb): the flux of the most massive galaxy, for a stellar population model.

All of the plots and analysis in Lovell et al. 2023 can be recreated with the scripts in `example/paper/`.

## Citing

If you use evstats, please cite [Lovell et al. 2023](https://academic.oup.com/mnras/article/518/2/2511/6823705).

```{toctree}
:hidden:

halo
stellar
observables
api
```
Loading
Loading