Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# The kit's own CI: unit tests (Python + JS), both bundle builds, the example
# course built with the dev bundle plus the browser smoke test, and the
# reusable course-check workflow run against the example course.
# The kit's own CI, on pull requests: unit tests (Python + JS), both bundle
# builds, the example course built with the dev bundle plus the browser smoke
# test, and the reusable course-check workflow run against the example course.
name: CI

# Pull requests only. Branch protection requires PRs to be up to date with
Expand Down Expand Up @@ -41,7 +41,11 @@ jobs:
- name: Dev build + tests
run: npm run build:dev && npm test

# The slowest job. A release PR only changes version strings and the
# changelog, so it is skipped there (a skipped job counts as passing for
# required checks).
example-course:
if: github.head_ref != 'release-please--branches--main--components--course-kit'
needs: [python, js]
runs-on: ubuntu-latest
steps:
Expand Down
14 changes: 10 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,11 +155,17 @@ downloads); CI retries failed tests once.

## CI

- `ci.yml`: pytest; JS release + dev builds with tests; example course build +
e2e; the reusable `course-check.yml` run against `example-course` with
`kit-source: local`.
- `ci.yml` runs on pull requests only (branch protection requires PRs to be up
to date with `main`): pytest; JS release + dev builds with tests; example
course build + e2e (skipped on the release-please PR, which only changes
version strings); the reusable `course-check.yml` run against
`tests/example-course` with `kit-source: local`.
- `release.yml`: release-please PR -> tag `vX.Y.Z` -> npm (provenance) and
PyPI (trusted publishing) -> move `vX.Y` and `vX`.
PyPI (trusted publishing) -> move `vX.Y` and `vX`. release-please uses the
`RELEASE_PLEASE_TOKEN` secret (a PAT or app token with Contents, Pull
requests and Workflows access), because it edits `KIT_VERSION` in
`.github/workflows/` and the default token may not; the token also makes CI
run on the release PR. Setup: README, "Releasing".
- `course-publish.yml` job order: check -> notebooks -> publish-notebooks ->
render-scrolly -> prepare-site -> myst build -> Pages deploy. One run per
repository at a time (notebook pushes never race). No CNAME step: with the
Expand Down
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Maintained by release-please from conventional commit messages.
* don't force provenance in package.json (CI passes --provenance) ([5199e87](https://github.com/neuromatch/course-kit/commit/5199e875c476013ebcf573ce721e4460382fce07))
* reorganize repo, add release manifest and package metadata ([b1a96c7](https://github.com/neuromatch/course-kit/commit/b1a96c725a5c94ceefd08cf8d1ac61e180a3c381))

## 0.1.0 (unreleased)
## 0.1.0

First version, extracted from `neuromatch/course-template`:

Expand Down
27 changes: 15 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,9 +105,9 @@ See [AGENTS.md](AGENTS.md) for the layout, the Python/JS twin rules and the
test commands. In short:

```bash
cd python && uv venv && uv pip install -e '.[test]' -r ../tests/example-course/requirements.txt && uv run pytest -q ../tests/python
cd js && npm ci && npm run build && npm test && npm run build:dev && npm test
tests/scripts/build-example.sh # build example-course with the dev bundle
(cd python && uv venv && uv pip install -e '.[test]' -r ../tests/example-course/requirements.txt && uv run pytest -q ../tests/python)
(cd js && npm ci && npm run build && npm test && npm run build:dev && npm test)
tests/scripts/build-example.sh # build tests/example-course with the dev bundle
```

To try a kit change in a course without releasing, point the course's
Expand All @@ -120,16 +120,19 @@ Commits on `main` use conventional messages (`fix:`, `feat:`, `feat!:`).
release-please keeps a release PR open; merging it tags `vX.Y.Z`, publishes
npm and PyPI, and moves the `vX.Y` and `vX` tags.

One-time setup (after this repo exists as `neuromatch/course-kit`):

1. **npm**: create the free npm organization `neuromatch`. Publish the first
version by hand (`cd js && npm run build && npm publish --access public`),
because npm trusted publishing can only be configured on an existing
package. Then add a trusted publisher: repository `neuromatch/course-kit`,
workflow `release.yml`.
2. **PyPI**: add a *pending publisher* for project `neuromatch-course-kit`:
The release flow is already set up. This is what it depends on, in case it
has to be rebuilt:

1. **npm**: the `neuromatch` organization owns `@neuromatch/course-kit`. The
first version was published by hand
(`cd js && npm run build && npm publish --otp=<code>`), because npm trusted
publishing can only be configured on an existing package. The package has a
trusted publisher: repository `neuromatch/course-kit`, workflow
`release.yml`, no environment, direct publishing allowed (the workflow runs
a plain `npm publish --provenance`), dist-tag management off.
2. **PyPI**: the project `neuromatch-course-kit` has a trusted publisher:
repository `neuromatch/course-kit`, workflow `release.yml`, environment
`release`. The first release creates the project.
`release`.
3. Create the `release` environment in the repo settings.
4. Add a second owner/maintainer on npm and PyPI.
5. Create a fine-grained personal access token (or GitHub App) for this repo
Expand Down
92 changes: 92 additions & 0 deletions js/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# @neuromatch/course-kit

The [MyST](https://mystmd.org) plugin for Neuromatch courses: one bundled
file plus the browser assets it uses. It is one half of the
[Neuromatch course kit](https://github.com/neuromatch/course-kit); the other
half is the [`nma-kit`](https://pypi.org/project/neuromatch-course-kit/)
command line, which checks content and generates notebooks. Both ship under
the same version.

You normally don't install this package. A course's `myst.yml` loads it from
jsDelivr:

```yaml
version: 1
extend: https://cdn.jsdelivr.net/npm/@neuromatch/course-kit@0.1/dist/nma.yml
project:
github: https://github.com/<org>/<course>
```

`dist/nma.yml` adds the plugin and turns on `project.jupyter.lite`. In CI,
`nma-kit prepare-site` rewrites `@0.1` to the exact version of the CLI, so the
plugin, its assets and the CLI always match. See the
[kit README](https://github.com/neuromatch/course-kit#using-the-kit-in-a-course)
for the full course setup.

## What it adds to MyST

| Directive / feature | What it does on the site |
|---|---|
| `{nma-video}`, `{nma-slides}` | Embedded YouTube, Bilibili or OSF videos and slide decks |
| `{nma-solution}`, `{nma-explanation}` | A "Click for solution" link; the solution text itself is never on the page |
| `{nma-feedback}` | A feedback widget that posts to datatops from the browser |
| `layout: scrolly`, `{step}`, `{stage}` | Scrollytelling pages with live Python (Pyodide) |
| Editable cells | Code cells become editable and runnable in the browser on pages with a `kernelspec`; `runtime: colab` pages get a Colab banner instead |

Examples:

```markdown
:::{nma-video} Video 1: Linear dynamical systems
:youtube: vPYQPI4nKT8
:bilibili: BV1up4y1S7wj
:::

:::{nma-feedback} Linear_Dynamical_Systems_Video
:::
```

Directives must be at the top level of a page. Their options are checked by
`nma-kit check`.

Settings live in the `nma:` block of `myst.yml` (all optional):

```yaml
nma:
paths:
tutorials: tutorials
notebooks:
branch: notebooks-branch
feedback: { url: https://..., project: ..., user_key: ... }
scrolly:
pyodide_url: https://cdn.jsdelivr.net/pyodide/v314.0.7/full/
```

One rule belongs in the course's own CSS, because a plugin can't add
site-wide CSS: `.myst-jp-nb-toolbar { display: none !important; }`.

## What's in the package

```
dist/nma.mjs the plugin (all features, one file)
dist/nma.yml the config courses `extend:`
dist/assets/<name>/ browser widgets and CSS, one folder per feature
dist/build-info.json {mode, version}
```

Asset URLs inside the bundle point at this exact version on jsDelivr. MyST
downloads them at build time and serves them from the course's own site, so
readers' browsers don't contact the CDN for them. The one exception is
the Python runtime: editable cells and scrolly pages load Pyodide from
`cdn.jsdelivr.net`, and micropip installs packages from PyPI.

## Versions

The kit is on 0.x. A breaking change bumps the minor version, so `@0.1` only
receives compatible fixes. Don't use `@latest` in `myst.yml`: jsDelivr caches
alias URLs for up to 7 days and `@latest` crosses major versions.

## Development

Source, tests and release process: <https://github.com/neuromatch/course-kit>.

BSD-3-Clause.
65 changes: 62 additions & 3 deletions python/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,64 @@
# neuromatch-course-kit

The `nma-kit` command line tool for Neuromatch MyST courses: notebook
generation, content checks and site preparation. See the
[course kit repository](https://github.com/neuromatch/course-kit) for docs.
The `nma-kit` command line for Neuromatch courses written in
[MyST](https://mystmd.org) Markdown. It is one half of the
[Neuromatch course kit](https://github.com/neuromatch/course-kit); the other
half is the MyST plugin on npm,
[`@neuromatch/course-kit`](https://www.npmjs.com/package/@neuromatch/course-kit).
Both ship under the same version.

Courses usually don't install this by hand: the reusable GitHub workflows in
the kit repository do. To use it locally:

```bash
uv venv && uv pip install -r requirements.txt "neuromatch-course-kit==0.1.*"
uv run nma-kit check
uv run myst start
```

Requires Python 3.10 or newer. Run every command from the course root (the
folder with `myst.yml`).

## Commands

| Command | What it does |
|---|---|
| `nma-kit check` | Validate the content rules and the `nma:` block of `myst.yml` |
| `nma-kit notebooks [--dry-run]` | Generate student and instructor notebooks and the solution files into `notebooks/` |
| `nma-kit render-scrolly` | Run each scrolly page's instructor notebook and save outputs to `_build/scrolly/` |
| `nma-kit prepare-site [--dry-run] [--kit-version V]` | Add badges and fallback cells to the pages and pin the kit URL in `myst.yml`. **CI only: it edits files in place, never commit the result** |
| `nma-kit publish-notebooks [--remote R] [--message M]` | Push `notebooks/` to the notebooks branch (created if missing) |

Exit codes: `0` ok, `1` content problems or a failed publish, `2` usage or
configuration errors.

## Settings

Read from `myst.yml`: `project.github` gives the repository, and the optional
top-level `nma:` block holds the rest.

```yaml
nma:
paths:
tutorials: tutorials # where the .md pages live
notebooks:
branch: notebooks-branch # where generated notebooks are published
feedback: # required if any page uses {nma-feedback}
url: https://...
project: ...
user_key: ...
```

Values in `nma:` must be plain strings in one level of sections.

## Versions

The kit is on 0.x. A breaking change bumps the minor version, so
`==0.1.*` only receives compatible fixes. The CLI and the npm plugin always
share a version number, and `prepare-site` pins the plugin URL to the CLI's
own version.

Source, docs and release process:
<https://github.com/neuromatch/course-kit>.

BSD-3-Clause.
4 changes: 3 additions & 1 deletion python/src/nma_kit/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@
with string values. js/src/shared/config.mjs reads the same block with a
YAML parser too; both sides are tested against tests/fixtures/config/settings.json.

New settings (all optional, defaults are the template's conventions):
Settings read here (all optional; the defaults are the course template's
conventions). The feedback and scrolly sections are read by nma_kit.feedback
and the plugin (`nma.scrolly.pyodide_url`):

nma:
paths:
Expand Down
8 changes: 4 additions & 4 deletions python/src/nma_kit/solutions.py
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ def close_block(block, end):
stop = end if end != -1 else len(lines)
body = lines[block.start + 1:stop]
block.options, block.body_lines = _split_options(body)
if block.kind == "code-cell" and any(EXERCISE_RE.match(l) for l in block.body_lines):
if block.kind == "code-cell" and any(EXERCISE_RE.match(line) for line in block.body_lines):
block.kind = "exercise"

for i, line in enumerate(lines):
Expand Down Expand Up @@ -184,7 +184,7 @@ def validate(text: str) -> list[str]:
for b in blocks:
where = f"line {b.start + 1}:"
if b.kind in ("code-cell", "exercise"):
if any(TO_REMOVE_RE.match(l) for l in b.body_lines):
if any(TO_REMOVE_RE.match(line) for line in b.body_lines):
errors.append(
f"{where} code cell has a `# to_remove` marker. Move the solution into "
f"an {{nma-solution}} or the explanation into an {{nma-explanation}} directive"
Expand Down Expand Up @@ -267,7 +267,7 @@ def exercise_pairs(text: str) -> list[tuple[str, str]]:
"""(exercise source, solution code) for each exercise. Assumes validate() passed."""
blocks = [b for b in find_blocks(text) if not b.nested]
pairs = []
for ex, sol in zip(blocks, blocks[1:]):
for ex, sol in zip(blocks, blocks[1:], strict=False):
if ex.kind == "exercise" and sol.kind == SOLUTION:
exercise = "\n".join(ex.body_lines).strip("\n")
pairs.append((exercise, solution_code(sol) or ""))
Expand Down Expand Up @@ -328,7 +328,7 @@ def split_notebook(nb: dict, stem: str, rel_dir: str, repo: str,
lines = text.splitlines(keepends=True)
base = cell.get("id", "cell")[:50]
cursor = 0
for k, (block, item) in enumerate(zip(directive_blocks, items)):
for k, (block, item) in enumerate(zip(directive_blocks, items, strict=False)):
prose = "".join(lines[cursor:block.start]).strip("\n")
if prose.strip():
fragment = _markdown(prose, f"{base}-m{k}")
Expand Down
2 changes: 1 addition & 1 deletion tests/python/test_release.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ def test_release_please_bumps_every_file():

def test_generic_files_carry_the_marker():
for path in [*(WORKFLOWS / n for n in REUSABLE), KIT / "python" / "src" / "nma_kit" / "__init__.py"]:
line = next(l for l in path.read_text().splitlines() if re.search(r"\d+\.\d+\.\d+", l) and "VERSION" in l.upper())
line = next(ln for ln in path.read_text().splitlines() if re.search(r"\d+\.\d+\.\d+", ln) and "VERSION" in ln.upper())
assert MARKER in line, path


Expand Down
2 changes: 1 addition & 1 deletion tests/scripts/compare_builds.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ def first_difference(a, b, path="$"):
return found
return None
if isinstance(a, list):
for i, (x, y) in enumerate(zip(a, b)):
for i, (x, y) in enumerate(zip(a, b, strict=False)):
found = first_difference(x, y, f"{path}[{i}]")
if found:
return found
Expand Down
Loading