From 4d5d9b7a3d555b589ca6d788572380d9814d85ab Mon Sep 17 00:00:00 2001 From: Alexis Gamelin Date: Wed, 23 Sep 2026 17:26:30 +0200 Subject: [PATCH 01/57] docs: introduce pyAML, its structure and the configuration rule - Add explanation pages: "What is pyAML?", "pyAML Structure" (with an object hierarchy diagram) and "Control Modes" (with planned modes). - State the "one field = one constructor argument" rule in the configuration explanation, and fix a broken link and an unclosed block. - Rewrite the "Create and Load Configuration" how-to around writing a YAML file by hand. - Add tutorial 00 (concepts, how to run the tutorials locally, learning path); introduce the test lattice in tutorials 01 and 02; use class: paths and help() in the tutorials. - Enable MyST definition lists and heading anchors. Co-Authored-By: Claude Opus 5.5 --- docs/source/_static/fodo-cell.svg | 66 ++++ docs/source/_static/pyaml-hierarchy.svg | 86 ++++++ docs/source/conf.py | 2 + docs/source/explanation/about.md | 82 +++++ docs/source/explanation/architecture.md | 113 +++++++ docs/source/explanation/configuration.md | 82 ++++- docs/source/explanation/control-modes.md | 93 ++++++ docs/source/explanation/index.md | 4 +- .../configuration/create-configuration.md | 288 +++++++++++++++++- docs/source/index.md | 2 + docs/tutorials/GALLERY_HEADER.rst | 4 + docs/tutorials/config.yaml | 21 +- .../functionality/00_introduction.py | 155 ++++++++++ .../functionality/01_create_accelerator.py | 157 +++++++--- .../functionality/02_inspect_accelerator.py | 69 ++++- docs/tutorials/functionality/config.yaml | 21 +- 16 files changed, 1153 insertions(+), 92 deletions(-) create mode 100644 docs/source/_static/fodo-cell.svg create mode 100644 docs/source/_static/pyaml-hierarchy.svg create mode 100644 docs/source/explanation/about.md create mode 100644 docs/source/explanation/architecture.md create mode 100644 docs/source/explanation/control-modes.md create mode 100644 docs/tutorials/functionality/00_introduction.py diff --git a/docs/source/_static/fodo-cell.svg b/docs/source/_static/fodo-cell.svg new file mode 100644 index 0000000..3dedfab --- /dev/null +++ b/docs/source/_static/fodo-cell.svg @@ -0,0 +1,66 @@ + + One cell of the fodo_1gev_6d test lattice + + + + + + + + + + + + + + + + + + + + + + + + QF + SF + BPM + COR + B + QD + SD + B + + + QF_0cc → ANcc-AR/EM-QP/QF.01/magnetic_strength + SF_0cc → ANcc-AR/EM-SX/SF.01/magnetic_strength + COR_0cc → ANcc-AR/EM-COR/CH.01 (H), ANcc-AR/EM-COR/CV.01 (V) + BPM_0cc → ANcc-AR/DG-EPOS/BPM.01/x, /y + QD_0cc → ANcc-AR/EM-QP/QD.01/magnetic_strength + SD_0cc → ANcc-AR/EM-SX/SD.01/magnetic_strength + + + + + 0 + 1 + 2 + 3 + 4 + 4.8 m + + diff --git a/docs/source/_static/pyaml-hierarchy.svg b/docs/source/_static/pyaml-hierarchy.svg new file mode 100644 index 0000000..133ae64 --- /dev/null +++ b/docs/source/_static/pyaml-hierarchy.svg @@ -0,0 +1,86 @@ + + pyAML object hierarchy + + + + + + + + + + Accelerator + Accelerator.load("config.yaml") + + + + + control modes + + + live — ControlSystem + accelerator.live + + + design — Simulator + accelerator.design + + + + identical content in every control mode + + + Arrays + magnets.get("QForTune") + bpms.get("BPM") + + + Elements + magnet.get("QF_001") + bpm.get("BPM_001") + + + Tuning tools + tune, orbit, + chromaticity, ... + + + + + + + + + Attributes + .strength (physics units) · .hardware (hardware units) + .positions · .get() · .set(value) + + + + + backends + + + Control system (TANGO, EPICS, ...) + tango-pyaml, pyaml-cs-oa + DeviceAccess + catalog + + + Simulation code + pyAT lattice + (lattice elements) + diff --git a/docs/source/conf.py b/docs/source/conf.py index 8cd6f7a..cc9cb49 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -30,7 +30,9 @@ myst_enable_extensions = [ "attrs_inline", + "deflist", ] +myst_heading_anchors = 3 sphinx_gallery_conf = { "examples_dirs": ["../tutorials"], diff --git a/docs/source/explanation/about.md b/docs/source/explanation/about.md new file mode 100644 index 0000000..e5e0e37 --- /dev/null +++ b/docs/source/explanation/about.md @@ -0,0 +1,82 @@ +# What is pyAML? + +The Python Accelerator Middle Layer (pyAML) is a Python library that sits between the people who operate or study a particle accelerator and the many systems that make up that accelerator: control systems, simulation codes, archivers, databases, and so on. It is developed by a collaboration of accelerator facilities as a common platform for the design, commissioning, and operation of particle accelerators. + +## Why a Middle Layer? + +Many accelerator facilities have for years relied on a *Middle Layer*, the MATLAB Middle Layer (MML) being the best-known example. A middle layer gives physicists a uniform, physics-oriented way to access a machine: they read "the orbit at all BPMs" or set "the strength of quadrupole QF1" without having to know which control system variable holds that value, which units it is in, or how a current is converted into a magnetic strength. + +pyAML carries that idea forward with a few important changes: + +- **Python instead of MATLAB.** Python is open, free and widely used in the scientific community, and it gives access to a large ecosystem of scientific and machine-learning tools. +- **Shared between facilities.** Measurement and correction tools written for pyAML (orbit correction, tune correction, response matrices, etc.) should run at any facility that has configured pyAML, instead of being rewritten at each laboratory. +- **Simulation as a first-class citizen.** The same script can act on the real machine or on a simulated one. Tools can then be developed and tested without using expensive and limited beam time. + +## Goals + +The collaboration has identified the following key features for pyAML: + +- An agnostic interface between an accelerator control system (TANGO, EPICS, ...), a virtual accelerator and a digital model. +- A base for developing and sharing beam measurement tools, such as orbit, trajectory, linear and non-linear optics corrections. +- A virtual accelerator / digital twin which allows testing tuning tools in real-life conditions without the need for beam time. +- Handling of both physics and hardware units, with a flexible unit-conversion interface. +- The possibility to configure different types of accelerators: transfer lines, linear and circular accelerators, and ramped accelerators. +- Configuration and measurement data managed in a standardized manner. +- A set of standard measurement tools in a modular structure. +- Long-term maintainability, by following modern software practices. +- Easy integration of facility-specific functionality as separate packages. + +## Layers of the Project + +The software is organized in layers: + +**Core** +: The features needed to configure a machine and communicate with the different backends: abstraction of devices (magnets, BPMs, tune monitors, ...), grouping of devices in arrays, abstraction of the control system, connection to simulators, and conversion between hardware and physics units. This is the `pyaml` package. + +**Common high-level applications** +: Tools shared between facilities, built on top of the core: tune and chromaticity correction, response-matrix measurements, orbit correction, dispersion measurement, beam-based alignment, LOCO, etc. + +**Facility-specific applications** +: Code developed by a single facility. If it follows the same standards as the rest of pyAML, it can be used together with the core, shared with other facilities, or later moved into the common applications. + +See [pyAML Structure](architecture.md) for how these layers map to Python packages and objects. + +## Guiding Principles for the Configuration + +A facility adopts pyAML by writing a *configuration* describing its machine. The configuration follows these principles: + +- It is **completely separated from the source code**. It describes what should be built, it does not contain code. +- It is **easy to extend**. Facility-specific devices can be added without modifying pyAML. +- It is possible to use **only a subset** of it, for example to use a single tool without configuring the whole machine. +- A facility only has to **install the packages it needs**, for example an EPICS facility does not need to install TANGO. + +There is one simple rule behind the configuration: each item names a Python class, and each of its other fields is an argument of that class's constructor. See [Configuration Structure and Syntax](configuration.md). + +## Glossary + +Accelerator +: The top-level pyAML object describing one machine (a storage ring, a booster, a transfer line, ...). It holds all the control modes, arrays and devices. + +Control mode +: One way of accessing the accelerator, for example `live` (the real machine) or `design` (a simulation). All control modes offer the same interface. See [Control Modes](control-modes.md). + +Element +: A single object of the accelerator that can be read or set: a magnet, a BPM, an RF plant, a tune monitor, ... + +Array +: A named group of elements, for example all BPMs or all the quadrupoles used for tune correction, that can be read or set in one call. + +Tuning tool +: A high-level measurement or correction tool (tune correction, orbit correction, response matrix measurement, ...) configured like any other device. + +Backend +: The system that a control mode talks to: a control system (through control-system bindings such as `tango-pyaml` or `pyaml-cs-oa`) or a simulation code (pyAT). + +Catalog +: The backend-specific description of how a key used in the configuration maps to a signal of the control system. See [Control System Catalogs](catalog.md). + +Virtual accelerator +: A simulated machine exposed through a real control system (for example TANGO devices backed by a simulation), so that it can be used exactly like the real machine. + +Digital shadow / digital twin +: A simulation that follows the real machine. In a shadow, changes on the real machine are reflected in the simulation, but not the other way around. In a twin, changes are reflected both ways. diff --git a/docs/source/explanation/architecture.md b/docs/source/explanation/architecture.md new file mode 100644 index 0000000..8f29ee3 --- /dev/null +++ b/docs/source/explanation/architecture.md @@ -0,0 +1,113 @@ +# pyAML Structure + +This page explains how pyAML is organized: which packages it is made of and how the objects you work with are related to each other. + +## Packages + +pyAML is not a single package but a small ecosystem. You only install what your facility needs. + +| Package | Role | +| --- | --- | +| `pyaml` (PyPI: `accelerator-middle-layer`) | The core: accelerator, control modes, elements, arrays, unit conversion, configuration loading and validation, tuning tools. It includes the simulator backend based on [pyAT](https://atcollab.github.io/at/p/index.html). | +| `tango-pyaml` | Control-system bindings for TANGO. | +| `pyaml-cs-oa` | Control-system bindings based on [ophyd-async](https://blueskyproject.io/ophyd-async/), supporting EPICS (Channel Access and PV Access) and TANGO. | +| Facility packages | Optional packages containing classes specific to one facility (special magnet models, devices, applications, ...). | +| `pyaml-test-lattice` | A test lattice with ready-made configurations, used in the tutorials. | + +The core never imports a control system directly. A control system is selected in the configuration by naming the class of its bindings. Only the bindings you use have to be installed. See [User Installation](../how-to/installation/user-installation.md) and the [API Reference](../reference/index.md). + +## Object Hierarchy + +```{figure} /_static/pyaml-hierarchy.svg +:alt: Hierarchy of pyAML objects, from the Accelerator to the backends +:width: 100% + +The pyAML object hierarchy. Every control mode contains the same arrays, elements and tools, connected to a different backend. +``` + +### Accelerator + +The `Accelerator` is the entry point. It represents one machine (for example a storage ring) and is usually created by loading a configuration file: + +```python +from pyaml.accelerator import Accelerator + +accelerator = Accelerator.load("config.yaml") +``` + +It holds general information such as the facility name, the machine name and the energy, together with the control modes, arrays and devices. + +### Control Modes + +A control mode is one way to access the machine. Two kinds are implemented: + +- A `ControlSystem` reads and writes values through a control system. It is typically named `live` and gives access to the real machine or to a virtual accelerator. +- A `Simulator` reads and writes values in a pyAT lattice model. It is typically named `design`. + +Each control mode is available as an attribute of the accelerator, named after the mode: + +```python +live = accelerator.live +design = accelerator.design +``` + +All control modes provide exactly the same interface, so code written for one mode works in any other. See [Control Modes](control-modes.md) for details. + +### Elements + +Elements are the individual objects of the machine: magnets (`Quadrupole`, `Sextupole`, `HCorrector`, combined-function magnets, ...), `BPM`, `RFPlant`, `BetatronTuneMonitor`, etc. They are declared **once** in the `devices` section of the configuration. + +When the accelerator is created, each element is attached to every control mode. Each mode receives its own copy, connected to its own backend. This is why the same magnet can be reached from any mode: + +```python +qf_live = accelerator.live.magnet.get("QF_001") # connected to the control system +qf_design = accelerator.design.magnet.get("QF_001") # connected to the pyAT lattice +``` + +### Attributes + +Elements expose attributes that can be read and written with `get()` and `set()`. For example a magnet has: + +- `strength`: the value in physics units (for example `1/m` for a quadrupole), +- `hardware`: the value in hardware units (for example `A` for a power-supply current). + +The conversion between the two is done by the *magnet model* given in the configuration (for example `IdentityMagnetModel` or `LinearMagnetModel`). A BPM provides `positions`, `offset` and `tilt`. + +Using explicit `get()` and `set()` methods instead of plain assignment is a deliberate choice: `quad.strenght.set(0.5)` (note the typo) raises an error, whereas an assignment `quad.strenght = 0.5` would silently create a new Python attribute and leave the magnet unchanged. + +### Arrays + +Arrays are named groups of elements, declared in the `arrays` section of the configuration. They allow reading or setting all elements of a group in a single, synchronized call: + +```python +quads = accelerator.design.magnets.get("QForTune") +k = quads.strengths.get() # numpy array, one value per magnet +quads.strengths.set(k * 1.001) + +orbit = accelerator.design.bpms.get("BPM").positions.get() +``` + +Arrays are also how high-level tools know which elements to use: a tune correction tool, for example, is configured with the *name* of the quadrupole array it should act on. + +### Tuning Tools + +Measurement and correction tools (tune, chromaticity, orbit, dispersion, response matrices, ...) are configured in the `devices` section like elements, and attached to every control mode in the same way. They refer to arrays and diagnostics by name. As a result, a tool configured once can be run on the simulator and on the real machine without any change. + +### Backends and Device Access + +At the bottom of the hierarchy, attributes talk to a backend: + +- In a `ControlSystem`, each attribute uses a `DeviceAccess` object, which represents one control-system signal (a TANGO attribute, an EPICS PV, ...). The control-system bindings create these objects from the keys written in the configuration, with the help of a [catalog](catalog.md). +- In a `Simulator`, attributes read and write the corresponding elements of the pyAT lattice. The link is made by name (`lattice_names`, defaulting to the element name) or by a *linker* matching an attribute of the lattice elements. + +## Discovering What Is Available + +The accelerator provides a `yellow_pages` object listing everything that is configured (arrays, tools, diagnostics) and in which control modes it is available: + +```python +print(accelerator.yellow_pages) +``` + +## Where the Configuration Fits + +Every object described on this page is constructed from the configuration. Each configuration item names a class and gives the arguments of its constructor. The configuration therefore mirrors this hierarchy directly: an `Accelerator` with `controls`, `simulators`, `arrays` and `devices`. See [Configuration Structure and Syntax](configuration.md). diff --git a/docs/source/explanation/configuration.md b/docs/source/explanation/configuration.md index bd2f7f3..ad324b4 100644 --- a/docs/source/explanation/configuration.md +++ b/docs/source/explanation/configuration.md @@ -9,7 +9,7 @@ class: pyaml.accelerator.Accelerator facility: pyAML_facility machine: storage_ring data_folder: '' -energy: 1e6 +energy: 1e9 controls: simulators: arrays: @@ -20,9 +20,76 @@ The configuration is organised as a description of a nested Python object tree. The syntax has been chosen to allow configuration and construction of objects for both pyAML classes and third party classes. This is to allow integration of facility specific implementation in pyAML, but also to simplify future development where it might be desirable to replace old classes with newer versions without breaking compatibility. +## One Field = One Constructor Argument + +The whole configuration follows a single rule: + +```{important} +Each configuration item names a Python class in its `class` field. **Every other field of the item is an argument of that class's constructor**, with the same name. +``` + +When pyAML reads an item, it imports the class given by `class` and calls it with the remaining fields as keyword arguments. The configuration below and the Python code next to it build exactly the same object: + +`````{grid} 2 +:gutter: 2 + +````{grid-item} +**Configuration** + +```yaml +class: pyaml.magnet.quadrupole.Quadrupole +name: QF_001 +model: + class: pyaml.magnet.identity_model.IdentityMagnetModel + unit: 1/m + physics: AN01-AR/EM-QP/QF.01/magnetic_strength +``` +```` + +````{grid-item} +**Python** + +```python +from pyaml.magnet.quadrupole import Quadrupole +from pyaml.magnet.identity_model import IdentityMagnetModel + +Quadrupole( + name="QF_001", + model=IdentityMagnetModel( + unit="1/m", + physics="AN01-AR/EM-QP/QF.01/magnetic_strength", + ), +) +``` +```` +````` + +Consequences of this rule: + +- **Nested objects are nested items.** If an argument expects an object (here `model` expects a magnet model), the field contains another item with its own `class` field. Lists of objects (such as `devices` or `simulators` of the `Accelerator`) are lists of items. +- **Optional arguments are optional fields.** Arguments with a default value can be left out. +- **Unknown fields are rejected.** A field which is not an argument of the constructor, for example a misspelled one, raises an error when the configuration is loaded. +- **Any class can be used.** Nothing is specific to pyAML classes: a facility-specific class from your own package can be used in the same way, as long as it can be imported. + +### Finding the Accepted Fields + +Since fields are constructor arguments, the documentation of a class tells you what to write in the configuration. You can: + +- read the [API documentation](https://pyaml.readthedocs.io/en/stable/) of the class, +- use `help()` in Python, which shows the signature of the constructor: + + ```python + from pyaml.magnet.quadrupole import Quadrupole + help(Quadrupole) + # Quadrupole(name: str, model: MagnetModel | None = None, + # lattice_names: str | None = None, description: str | None = None) + ``` + +- use the schema registry, whose `describe()` method lists the fields of a registered class with their types. See [Use the Schema Registry](../how-to/configuration/use-schema-registry.ipynb). + ## Configuration Items -Each configurable item is represented by a mapping which describes the attributes and values needed to construct one Python object. The field `class` or `class_path` identifies the type to construct. It should be written as a fully qualified Python class path, consisting of the module and class name. For example: +Each configurable item is represented by a mapping which describes the attributes and values needed to construct one Python object. The field `class` (or its alias `class_path`) identifies the type to construct. It should be written as a fully qualified Python class path, consisting of the module and class name. For example: ```yaml class: pyaml.magnet.quadrupole.Quadrupole @@ -41,6 +108,10 @@ model: physics: AN01-AR/EM-QP/QF.01/magnetic_strength ``` +```{note} +Older configurations, including the ones of the `pyaml-test-lattice` package, use the legacy `type` field instead of `class`. It contains the path of the **module** instead of the class, for example `type: pyaml.magnet.quadrupole`. The class is then found from the module. This form is still supported, but `class` with the full class path is recommended for new configurations. The two forms cannot be mixed in the same item. +``` + ## Separation between Configuration and Source Code The configuration describes what should be constructed; it does not contain executable Python code. This keeps configuration readable, reviewable, and usable by tools such as JSON Schema editors. @@ -57,7 +128,7 @@ The configuration can be written and loaded in different formats: ## Configuration Root -The configuration root is the directory used to resolve relative configuration paths. It applies to the file passed to `Accelerator.load()` and paths used by [resolvers](#resolvers. Relative paths are resolved against this directory. +The configuration root is the directory used to resolve relative configuration paths. It applies to the file passed to `Accelerator.load()` and paths used by [resolvers](#resolvers). Relative paths are resolved against this directory. By default, the root is the current working directory when pyAML is imported. It can be changed before loading a configuration with `ROOT.set()`: @@ -71,7 +142,7 @@ After setting the root, a file written as `devices/quadrupole.yaml` in the confi Setting the root makes it possible to keep a configuration and its included files in a portable directory tree while selecting that tree at runtime. -Absolute paths are normalized and used directly. +Absolute paths are normalized and used directly. ## Resolvers @@ -89,9 +160,10 @@ The following built-in resolvers are available: | --- | --- | --- | | `env` | Environment variable | The value of the named environment variable. An error is raised if it is not set. | | `path` | A file or directory path | The absolute, normalized path, resolved relative to pyAML's configuration root. The target is not loaded as part of loading the configuration. | -| `file` | A YAML, YML, or JSON file path | The path to a file which should be loaded and expanded into the configuration as part of loading it. Relative paths use the configuration root. +| `file` | A YAML, YML, or JSON file path | The path to a file which should be loaded and expanded into the configuration as part of loading it. Relative paths use the configuration root. | Configuration files can also be included without an explicit `file` expression. A string ending in `.yaml`, `.yml`, or `.json` is loaded automatically for convenience. This makes it easy to split the configuration into several files if one wishes. ```{warning} If you include a `.yaml`, `.yml`, or `.json` in the configuration file which you do not want to be loaded and expanded into the configuration (for example a lattice in JSON format), remember to put `${path:filename}` or you will get an error when loading the configuration. +``` diff --git a/docs/source/explanation/control-modes.md b/docs/source/explanation/control-modes.md new file mode 100644 index 0000000..31b025f --- /dev/null +++ b/docs/source/explanation/control-modes.md @@ -0,0 +1,93 @@ +# Control Modes + +A control mode is one way of accessing the accelerator. pyAML is built so that the core interactions with the machine work in exactly the same way in every control mode. The following requirements guide the design: + +- Core interactions work identically in all control modes. +- All configured control modes are available at all times, and can be used at the same time in one script. +- All control modes are defined in the configuration. +- Standard measurements and high-level applications behave in the same way in every control mode. + +## Available Control Modes + +Two kinds of control modes are implemented today. + +**Live** (`ControlSystem`) +: Access to the accelerator through its control system. The values are read from and written to the control system through control-system bindings such as `tango-pyaml` or `pyaml-cs-oa`. The control system can be the real machine or a *virtual accelerator*, a simulation exposed through the same control-system interface as the real machine. By convention this mode is named `live`. + +**Design** (`Simulator`) +: Access to a simulation of the accelerator. Values are read from and written to a [pyAT](https://atcollab.github.io/at/p/index.html) lattice. Diagnostics such as BPMs and tune monitors return the values computed by the simulator. No control system is needed. By convention this mode is named `design`. + +The modes are declared in the configuration: control systems in `controls` and simulators in `simulators`. Each of them has a `name`, which is also the name of the attribute used to access it: + +```yaml +class: pyaml.accelerator.Accelerator +facility: My facility +machine: sr +energy: 1.0e9 +simulators: + - class: pyaml.lattice.simulator.Simulator + name: design + lattice: ${path:sr_lattice.json} +controls: + - class: pyaml_cs_oa.controlsystem.OphydAsyncControlSystem + name: live + catalog: catalog.yaml +``` + +```python +accelerator.design # the Simulator +accelerator.live # the ControlSystem +``` + +Several simulators or control systems can be defined, as long as they have different names. For example a second simulator loaded with a lattice including errors could be named `errors` and reached with `accelerator.errors`. + +## Using Control Modes + +Because every mode exposes the same elements, arrays and tools, a script can be written once and run in any mode: + +```python +def correct_tune(sr): + sr.tune.set([0.2, 0.3]) + +correct_tune(accelerator.design) # try it on the simulator first +correct_tune(accelerator.live) # then run it on the machine +``` + +A common pattern is to select the mode once at the top of a script: + +```python +SR = accelerator.design # switch to accelerator.live to act on the machine +``` + +Different modes can also be used side by side, for example to compare the measured orbit with the simulated one: + +```python +delta = ( + accelerator.live.bpms.get("BPM").positions.get() + - accelerator.design.bpms.get("BPM").positions.get() +) +``` + +or to copy the corrector settings of the machine into the model: + +```python +hcorr = accelerator.live.magnets.get("HCorr").strengths.get() +accelerator.design.magnets.get("HCorr").strengths.set(hcorr) +``` + +## Planned Control Modes + +```{admonition} Planned, not implemented yet +:class: note + +The following control modes are described in the pyAML specification. They are part of the long-term plan of the collaboration and are **not available yet**. +``` + +**Errors / commissioning simulations** +: Simulators with lattices containing errors, and arrays of randomly generated error seeds, used to run simulated commissioning of a machine. + +**Shadow (digital shadow)** +: A simulator that follows the real machine: settings read from the control system are applied to the model, which then computes the expected optics and diagnostics. Writing is forbidden in this mode. + +**Archive** +: A simulator loaded with the machine settings found in the archiving system at a given time, to reproduce and investigate a past situation. diff --git a/docs/source/explanation/index.md b/docs/source/explanation/index.md index ed5850c..a59fd96 100644 --- a/docs/source/explanation/index.md +++ b/docs/source/explanation/index.md @@ -5,7 +5,9 @@ Here you can find explanations of the concepts, design decisions, and underlying ```{toctree} :maxdepth: 1 - +about +architecture +control-modes configuration schema_and_validation catalog diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index 671290b..dc78076 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -1,31 +1,287 @@ # Create and Load Configuration -The structure and syntax of the configuration are explained in detail in [Configuration Structure and Syntax](../../explanation/configuration). This guide focuses on the different ways to create it. +This guide shows how to write a pyAML configuration as a text file and load it into an `Accelerator`. The structure and syntax of the configuration are explained in detail in [Configuration Structure and Syntax](../../explanation/configuration). -There are many ways to create a configuration. It is recommended to test the different -options and see which one you prefer: +## The Rule to Remember -- Use a [JSON Schema in VS Code](./use-vscode-json-schema.md) +```{important} +Each item of the configuration names a Python class in its `class` field. **Every other field is an argument of the constructor of that class**, with the same name. When an argument is an object, its value is a nested item with its own `class` field. +``` -- Use a JSON Schema in the [MetaConfigurator](./use-meta-configurator.md) +Writing a configuration is therefore the same as writing the Python code that creates the objects, in YAML (or JSON) instead of Python. -- [Use ConfigurationSchema](./use-configuration-schema.ipynb) objects and export as a dictionary or text file +## Find the Fields of a Class -Another option is to use AI coding assistance tools. You can then for example supply a lattice file, information describing the naming conventions for your control system and a JSON Schema for the pyAML configuration and get help to write it. +Before writing an item, look up the constructor arguments of its class. Any of these works: -For information about what a JSON Schema is and how to generate it, see [Configuration Schemas and Validation](../../explanation/schema_and_validation.md) and [Generate JSON Schemas](./generate-json-schema.ipynb). +- `help()` in Python, which shows the constructor signature and describes each argument: + + ```python + from pyaml.bpm.bpm import BPM + + help(BPM) + # class BPM(...) + # | BPM(name: str, lattice_names: str | None = None, description: str | None = None, + # | x_pos: str | None = None, y_pos: str | None = None, ...) + # | + # | Parameters + # | ---------- + # | name : str + # | Name of the BPM. + # | x_pos : str | None, optional + # | Device catalog key for the horizontal beam position. + # | ... + ``` + +- the [API documentation](https://pyaml.readthedocs.io/en/stable/) of the class, +- the `describe()` method of the schema in the [schema registry](./use-schema-registry.ipynb). + +Arguments without a default value (here `name`) are required fields; the others can be left out. + +## Write the Configuration File + +Create a file, for example `accelerator.yaml`, with any text editor. The steps below build a small but complete configuration, using the names of the [test lattice](../../tutorials/functionality/01_create_accelerator). + +### 1. The Accelerator + +The root item is the `Accelerator`. Its required arguments are `facility`, `machine` and `energy`: + +```yaml +class: pyaml.accelerator.Accelerator +facility: My facility +machine: sr +energy: 1.0e9 +``` + +### 2. The Control Modes + +Add the control modes as lists in `simulators` and `controls`. Their `name` is also the name used to access them (`accelerator.design`, `accelerator.live`). + +A simulator needs the path to a lattice file. Use `${path:...}` so that the path is resolved relative to the [configuration root](../../explanation/configuration.md#configuration-root) and the lattice is not loaded as a configuration file: + +```yaml +simulators: + - class: pyaml.lattice.simulator.Simulator + name: design + lattice: ${path:lattice.json} +``` + +A control system is given by the class of the bindings you use. Its arguments depend on the bindings, for example for `pyaml-cs-oa`: + +```yaml +controls: + - class: pyaml_cs_oa.controlsystem.OphydAsyncControlSystem + name: live + backend: tango + catalog: catalog.yaml +``` + +The catalog describes how the keys used by the devices map to control-system signals. It follows exactly the same rule. A static catalog for `pyaml-cs-oa` looks like this (one entry per key): + +```yaml +class: pyaml_cs_oa.static_catalog.StaticCatalog +entries: + - class: pyaml_cs_oa.static_catalog_entry.StaticCatalogEntry + key: AN01-AR/EM-QP/QF.01/magnetic_strength + device: + class: pyaml_cs_oa.tangoAtt.TangoAtt + attribute: AN01-AR/EM-QP/QF.01/magnetic_strength + unit: 1/m + # ... one entry for each key used in the configuration +``` + +See [Control System Catalogs](../../explanation/catalog.md) for the different types of catalogs. If you only want to use the simulator, you can leave out `controls` entirely. + +### 3. The Devices + +Add the elements of the machine in `devices`. For a magnet, the `model` argument is an object (the magnet model, which also handles the unit conversion), so it is written as a nested item: + +```yaml +devices: + - class: pyaml.magnet.quadrupole.Quadrupole + name: QF_001 + model: + class: pyaml.magnet.identity_model.IdentityMagnetModel + unit: 1/m + physics: AN01-AR/EM-QP/QF.01/magnetic_strength + - class: pyaml.magnet.quadrupole.Quadrupole + name: QD_001 + model: + class: pyaml.magnet.identity_model.IdentityMagnetModel + unit: 1/m + physics: AN01-AR/EM-QP/QD.01/magnetic_strength + - class: pyaml.bpm.bpm.BPM + name: BPM_001 + x_pos: AN01-AR/DG-EPOS/BPM.01/x + y_pos: AN01-AR/DG-EPOS/BPM.01/y +``` + +By default, the `name` of an element is also the name of the element in the lattice of the simulator. Use `lattice_names` if they differ. The strings given to `physics`, `x_pos` and `y_pos` are keys looked up in the catalog of the control system. + +### 4. The Arrays + +Group elements in named arrays in `arrays`. Element names can contain wildcards: + +```yaml +arrays: + - class: pyaml.arrays.magnet.Magnet + name: Quadrupoles + elements: + - QF_001 + - QD_001 + - class: pyaml.arrays.bpm.BPM + name: BPMs + elements: + - BPM_* +``` + +### Complete File + +Putting it all together: + +```yaml +class: pyaml.accelerator.Accelerator +facility: My facility +machine: sr +energy: 1.0e9 +simulators: + - class: pyaml.lattice.simulator.Simulator + name: design + lattice: ${path:lattice.json} +controls: + - class: pyaml_cs_oa.controlsystem.OphydAsyncControlSystem + name: live + backend: tango + catalog: catalog.yaml +devices: + - class: pyaml.magnet.quadrupole.Quadrupole + name: QF_001 + model: + class: pyaml.magnet.identity_model.IdentityMagnetModel + unit: 1/m + physics: AN01-AR/EM-QP/QF.01/magnetic_strength + - class: pyaml.magnet.quadrupole.Quadrupole + name: QD_001 + model: + class: pyaml.magnet.identity_model.IdentityMagnetModel + unit: 1/m + physics: AN01-AR/EM-QP/QD.01/magnetic_strength + - class: pyaml.bpm.bpm.BPM + name: BPM_001 + x_pos: AN01-AR/DG-EPOS/BPM.01/x + y_pos: AN01-AR/DG-EPOS/BPM.01/y +arrays: + - class: pyaml.arrays.magnet.Magnet + name: Quadrupoles + elements: + - QF_001 + - QD_001 + - class: pyaml.arrays.bpm.BPM + name: BPMs + elements: + - BPM_* +``` + +## Split the Configuration into Several Files + +For a real machine the configuration becomes long. Any string ending with `.yaml`, `.yml` or `.json` in a list is replaced by the content of that file. If the file contains a list, its items are added to the parent list. For example, move the quadrupoles to `devices/quadrupoles.yaml`: + +```yaml +# devices/quadrupoles.yaml +- class: pyaml.magnet.quadrupole.Quadrupole + name: QF_001 + model: + class: pyaml.magnet.identity_model.IdentityMagnetModel + unit: 1/m + physics: AN01-AR/EM-QP/QF.01/magnetic_strength +- class: pyaml.magnet.quadrupole.Quadrupole + name: QD_001 + # ... +``` + +and refer to it from the main file: + +```yaml +devices: + - devices/quadrupoles.yaml + - class: pyaml.bpm.bpm.BPM + name: BPM_001 + x_pos: AN01-AR/DG-EPOS/BPM.01/x + y_pos: AN01-AR/DG-EPOS/BPM.01/y +``` + +Values can also come from environment variables with `${env:NAME}`. See [Resolvers](../../explanation/configuration.md#resolvers) for all options. + +## Use Your Own Classes + +The rule is not limited to pyAML classes. Any class that can be imported can be used in the configuration, for example a magnet model specific to your facility: + +```python +# my_facility/models.py +from pyaml.magnet.model import MagnetModel + +class MyMagnetModel(MagnetModel): + def __init__(self, power_supply: str, calibration: float): + ... +``` + +```yaml +model: + class: my_facility.models.MyMagnetModel + power_supply: PS/QF/01 + calibration: 1.02 +``` ## Load the Configuration -The configuration can be loaded into the `Accelerator` in two ways: +Set the configuration root, which is the directory used to resolve relative paths, then load the file with `Accelerator.load()`: + +```python +from pyaml.configuration import ROOT +from pyaml.accelerator import Accelerator + +ROOT.set("/path/to/configuration") +accelerator = Accelerator.load("accelerator.yaml") + +accelerator.design.magnets.get("Quadrupoles").strengths.get() +``` + +If the control-system bindings are not installed or you only want to use the simulator, add `ignore_external=True`. The `controls` section is then skipped. + +The configuration can also be given as a nested dictionary with `Accelerator.from_dict()`, following the same rule: + +```python +import yaml + +with open("accelerator.yaml") as file: + config = yaml.safe_load(file) + +accelerator = Accelerator.from_dict(config) +``` + +See the API documentation for the [Accelerator](https://pyaml.readthedocs.io/en/stable/api/pyaml.accelerator.html#module-pyaml.accelerator) for all options. + +## Validate the Configuration + +Each item is checked when it is created: a missing required field or an unknown field raises a `PyAMLConfigException` naming the class and the field. The whole configuration can also be validated before anything is created, which gives all errors at once: + +```python +from pyaml.validation import SchemaRegistry + +SchemaRegistry().discover() +accelerator = Accelerator.load("accelerator.yaml", validate=True) +``` + +The configuration can also be validated without loading it, which is useful if you maintain it separately from pyAML. See [Validate Configuration](./validate-configuration). + +## Tools That Help Writing the Configuration -| Type| Command | Description | -| --- | --- | --- | -| File | `Accelerator.load()` | A text file in JSON or YAML format. -| Dictionary | `Accelerator.from_dict()` | A nested dictionary. +Writing the file by hand is often the simplest way to start, but tools based on a [JSON Schema](../../explanation/schema_and_validation.md) can suggest the available fields and check their types while you write: -See the API documentation for the [Accelerator](https://pyaml.readthedocs.io/en/stable/api/pyaml.accelerator.html#module-pyaml.accelerator) for more details. +- [Use a JSON Schema in VS Code](./use-vscode-json-schema.md) +- [Use the MetaConfigurator](./use-meta-configurator.md), a form-based editor in the browser +- [Use ConfigurationSchema](./use-configuration-schema.ipynb) objects to create the configuration in Python and export it as a dictionary or text file -## Validation +AI coding assistants can also help: supply for example a lattice file, a description of the naming conventions of your control system, and the JSON Schema of the pyAML configuration. -The configuration is validated when loading it into the `Accelerator` but it can also be validated without having to load it. This is useful if you want to be able to maintain it separately from pyAML. See [Validate Configuration](./validate-configuration) for details. \ No newline at end of file +For information about JSON Schemas and how to generate them, see [Configuration Schemas and Validation](../../explanation/schema_and_validation.md) and [Generate JSON Schemas](./generate-json-schema.ipynb). diff --git a/docs/source/index.md b/docs/source/index.md index 6c870f9..13cff3a 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -6,6 +6,8 @@ html_theme.sidebar_secondary.remove: true Python Accelerator Middle Layer (pyAML) is a joint technology platform for design, commissioning, and operation of particle accelerators. +It is developed by a collaboration of accelerator facilities to give physicists a common, physics-oriented way to access their machines, whether real or simulated, and to share measurement and correction tools between laboratories. Read [What is pyAML?](explanation/about.md) to learn more about the motivation and goals of the project. + The features include, among others: - A control system-agnostic interface to interact with the accelerator. diff --git a/docs/tutorials/GALLERY_HEADER.rst b/docs/tutorials/GALLERY_HEADER.rst index eef415e..ec4fd84 100644 --- a/docs/tutorials/GALLERY_HEADER.rst +++ b/docs/tutorials/GALLERY_HEADER.rst @@ -3,6 +3,10 @@ Tutorials Here you can find learning-oriented tutorials that guide you through the functionality of pyAML step by step. +If you are new to pyAML, start with +:doc:`Introduction to pyAML `, +which presents pyAML and its main concepts. + The notebooks can be run using `Binder `_ or be downloaded and run on your own computer. If you want to run them on your own computer, see :doc:`User Installation <../how-to/installation/user-installation>` for diff --git a/docs/tutorials/config.yaml b/docs/tutorials/config.yaml index 00b68ad..73ba6bf 100644 --- a/docs/tutorials/config.yaml +++ b/docs/tutorials/config.yaml @@ -1,15 +1,14 @@ -type: pyaml.accelerator +class: pyaml.accelerator.Accelerator facility: pyAML test facility machine: pyaml test machine -data_folder: null -energy: 1000000000.0 +energy: 1.0e9 simulators: -- type: pyaml.lattice.simulator - lattice: ${env:PYAML_TEST_LATTICE} - name: design + - class: pyaml.lattice.simulator.Simulator + name: design + lattice: ${env:PYAML_TEST_LATTICE} devices: -- type: pyaml.magnet.quadrupole - name: QF_001 - model: - type: pyaml.magnet.identity_model - physics: '' + - class: pyaml.magnet.quadrupole.Quadrupole + name: QF_001 + model: + class: pyaml.magnet.identity_model.IdentityMagnetModel + physics: '' diff --git a/docs/tutorials/functionality/00_introduction.py b/docs/tutorials/functionality/00_introduction.py new file mode 100644 index 0000000..6851ce9 --- /dev/null +++ b/docs/tutorials/functionality/00_introduction.py @@ -0,0 +1,155 @@ +# --- +# jupyter: +# jupytext: +# cell_metadata_filter: -all +# custom_cell_magics: kql +# text_representation: +# extension: .py +# format_name: percent +# format_version: '1.3' +# jupytext_version: 1.11.2 +# kernelspec: +# display_name: pyaml-documentation +# language: python +# name: python3 +# --- + +# %% +""" +Introduction to pyAML +========================================================== + +This tutorial is the starting point of the pyAML tutorials. It introduces what pyAML is, +the main concepts you will meet in the other tutorials, and how the tutorials are organized. +It does not contain any code: the hands-on part starts in the next tutorial. +""" + +# %% +# What is pyAML? +# -------------- +# +# The Python Accelerator Middle Layer (pyAML) is a Python library, developed by a +# collaboration of accelerator facilities, that gives a common, physics-oriented way to +# access a particle accelerator. With pyAML you read "the orbit at all BPMs" or set +# "the strength of quadrupole QF_001" without having to know which control system variable +# holds the value, or how a power-supply current is converted into a magnetic strength. +# +# Its main goals are to: +# +# - provide the same interface for the real machine, a virtual accelerator and a simulation +# model, independently of the control system (TANGO, EPICS, ...), +# - allow measurement and correction tools (orbit, tune, chromaticity, ...) to be written +# once and shared between facilities, +# - allow those tools to be developed and tested on a simulation, without using beam time. +# +# You can read more about the motivation and goals in +# :doc:`What is pyAML? <../../explanation/about>`. + +# %% +# Main Concepts +# ------------- +# +# Working with pyAML means working with a small hierarchy of objects: +# +# .. figure:: /_static/pyaml-hierarchy.svg +# :alt: Hierarchy of pyAML objects +# :width: 100% +# +# - **Accelerator**: the top-level object describing one machine. It is created from a +# configuration. +# - **Control mode**: one way of accessing the accelerator. ``accelerator.live`` talks to the +# control system (the real machine or a virtual accelerator), ``accelerator.design`` talks +# to a simulation of the machine made with `pyAT `_. +# Both provide exactly the same interface. +# - **Element**: one object of the machine, such as a magnet, a BPM or the RF plant. +# - **Array**: a named group of elements which can be read or set in one call, such as all +# the BPMs. +# - **Attribute**: a value of an element that can be read with ``get()`` and written +# with ``set()``, such as the ``strength`` of a magnet. +# - **Configuration**: a YAML or JSON file describing all of the above. Each item of the +# configuration names a Python class with its ``class`` field, and **each other field is an +# argument of that class's constructor**. This rule is explored in the first tutorial. +# +# See :doc:`pyAML Structure <../../explanation/architecture>` and +# :doc:`Control Modes <../../explanation/control-modes>` for more details. + +# %% +# How the Tutorials Work +# ---------------------- +# +# **Running the tutorials.** Each tutorial can be run in the cloud with +# `Binder `_ using the launcher on its page, with nothing to install, +# or downloaded and run on your own computer, as a Jupyter notebook or a Python script +# (see below). +# +# **The test machine.** The tutorials use a small test storage ring, with its lattice and +# ready-made pyAML configurations, provided by the ``pyaml-test-lattice`` package. It is +# presented in the tutorials where it is first used. +# +# **The control mode.** The tutorials use the ``design`` control mode, a simulation of the +# machine, so no control system is needed. To run them on the ``live`` control mode, you need a +# control system, for example a virtual accelerator of the test machine +# (see :doc:`Installing Apptainer <../../how-to/virtual-accelerator/apptainer>`). +# Since all control modes have the same interface, switching is a single line: +# +# .. code-block:: python +# +# SR = accelerator.design # replace by accelerator.live to use the control system + +# %% +# Run the Tutorials Locally +# ------------------------- +# +# You need Python 3.11 or newer. Always install in a virtual environment, to avoid breaking +# your Python installation (see :doc:`New to Python <../../how-to/getting-started/python-basics>` +# if you are not familiar with virtual environments). Then install: +# +# .. code-block:: bash +# +# pip install "accelerator-middle-layer[cs-oa-tango]" pyaml-test-lattice jupyterlab +# +# This installs: +# +# - ``accelerator-middle-layer``: the ``pyaml`` core package. It also installs pyAT, used by +# the ``design`` control mode, together with numpy and matplotlib. +# - ``[cs-oa-tango]``: the ``pyaml-cs-oa`` control-system bindings for TANGO. The ready-made +# configurations of the test machine declare a ``live`` control mode using these bindings, +# so they are needed to load them, even if you only use the ``design`` control mode. +# - ``pyaml-test-lattice``: the test machine, with its lattice and pyAML configurations. +# - ``jupyterlab``: to open the tutorials as notebooks. It is not needed to run them as +# Python scripts. +# +# The other control-system bindings (``cs-oa-epics``, ``tango-pyaml``) are only needed to +# connect to your own control system, see +# :doc:`User Installation <../../how-to/installation/user-installation>`. To get exactly the +# same environment as on Binder, install the +# `Binder requirements `_ +# instead: ``pip install -r binder/requirements.txt`` from a clone of the documentation +# repository. +# +# Finally, download a tutorial as a notebook or a Python script with the download links in +# the right sidebar of its page, and run it with ``jupyter lab`` or ``python``. + +# %% +# Where to Go Next +# ---------------- +# +# The tutorials are designed to be followed in this order: +# +# 1. :doc:`Create an Accelerator <01_create_accelerator>`: create an accelerator +# interactively and from a configuration file, and learn how the configuration maps to +# Python classes. +# 2. :doc:`Inspect an Accelerator <02_inspect_accelerator>`: explore what a complete +# accelerator configuration contains. +# 3. The **use cases**, which apply pyAML to real tasks: +# :doc:`tune correction <../use_cases/tune-correction>`, +# :doc:`orbit correction <../use_cases/orbit_correction>` and +# :doc:`chromaticity measurement <../use_cases/chromaticity-measurement>`. +# +# For more background, read the explanations: +# :doc:`What is pyAML? <../../explanation/about>`, +# :doc:`pyAML Structure <../../explanation/architecture>`, +# :doc:`Control Modes <../../explanation/control-modes>` and +# :doc:`Configuration Structure and Syntax <../../explanation/configuration>`. + +# sphinx_gallery_thumbnail_path = '_static/pyaml-hierarchy.svg' diff --git a/docs/tutorials/functionality/01_create_accelerator.py b/docs/tutorials/functionality/01_create_accelerator.py index 52a02c2..a7da77a 100644 --- a/docs/tutorials/functionality/01_create_accelerator.py +++ b/docs/tutorials/functionality/01_create_accelerator.py @@ -21,11 +21,15 @@ This tutorial shows the different ways to create a pyAML accelerator. -The example only uses the simulator mode. The other modes are explored in other tutorials. +The example only uses the ``design`` control mode, which is a simulator of the machine based +on pyAT. See :doc:`Introduction to pyAML <00_introduction>` for an +overview of the control modes and the other concepts used here. The first approach constructs the objects interactively whereas the second one creates them by loading a configuration file. Both produce the same final interface, but each is suited -to different use cases which will be explained in the tutorial. +to different use cases which will be explained in the tutorial. You will also see that the +two approaches are closely related: **each field of the configuration file is an argument of +the constructor of a Python class**. """ # %% @@ -36,6 +40,26 @@ # `pyAT `_. # # The example uses the lattice provided by the ``pyaml-test-lattice`` package. +# +# The Test Lattice +# ~~~~~~~~~~~~~~~~ +# +# The test lattice, ``fodo_1gev_6d``, is a small 1 GeV electron storage ring made of +# **16 identical FODO cells** of 4.8 m, for a circumference of 76.8 m. Each cell contains a +# focusing quadrupole ``QF`` and sextupole ``SF``, a beam position monitor ``BPM``, a corrector +# ``COR`` acting in both planes, a dipole ``B``, a defocusing quadrupole ``QD`` and sextupole +# ``SD``, and a second dipole ``B``: +# +# .. figure:: /_static/fodo-cell.svg +# :alt: Layout of one FODO cell of the test lattice +# :width: 100% +# +# One cell of the test lattice with the control-system name of each element +# (``cc`` is the cell number, from 01 to 16). +# +# Each element is named after its family and a three-digit index equal to the cell number: +# ``QF_001`` is the focusing quadrupole of the first cell. This is the magnet used in +# this tutorial. # Get the path to the lattice file # sphinx_gallery_thumbnail_path = '_static/create_accelerator.png' @@ -120,51 +144,80 @@ # ------------------------------------- # Devices can also be created by loading a configuration file. # -# Configuration files are loaded through the interface of the accelerator and +# Configuration files are loaded through the interface of the accelerator and # are intended to be used for use cases with many devices, several control modes etc. # # Configuration files can be written in YAML or JSON. This example shows a YAML file. +# %% +# The Configuration Rule +# ~~~~~~~~~~~~~~~~~~~~~~ +# A configuration file describes the same objects as the ones created in approach 1, +# following one simple rule: +# +# - the ``class`` field gives the full path of the Python class to create, +# - **every other field is an argument of the constructor of that class**, with the same name, +# - when an argument is itself an object, its value is a nested item with its own ``class``. +# +# For example, in approach 1 the quadrupole was created with: +# +# .. code-block:: python +# +# Quadrupole(name="QF_001", model=IdentityMagnetModel(physics="")) +# +# which becomes in the configuration file: +# +# .. code-block:: yaml +# +# class: pyaml.magnet.quadrupole.Quadrupole +# name: QF_001 +# model: +# class: pyaml.magnet.identity_model.IdentityMagnetModel +# physics: '' +# +# The accepted fields of any class are therefore given by the arguments of its constructor, +# which you can see with ``help()``. The first lines show the constructor signature, and the +# ``Parameters`` section describes each argument: + +help(Quadrupole) # %% -# Create a YAML file -# ~~~~~~~~~~~~~~~~~~ -# The YAML file can be created using different tools. More details can be found in the the how-to guides. -# In this example we create it directly here. - -import yaml - -data = { - "type": "pyaml.accelerator", - "facility": "pyAML test facility", - "machine": "pyaml test machine", - "data_folder": None, - "energy": 1e9, - "simulators": [ - { - "type": "pyaml.lattice.simulator", - "lattice": "${env:PYAML_TEST_LATTICE}", - "name": "design", - } - ], - "devices": [ - { - "type": "pyaml.magnet.quadrupole", - "name": "QF_001", - "model": { - "type": "pyaml.magnet.identity_model", - "physics": "", - }, - } - ], -} +# Write the Configuration File +# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +# A configuration file is a plain text file. You can write it with any text editor. Other +# tools which can help you are described in the how-to guide +# :doc:`Create and Load Configuration <../../how-to/configuration/create-configuration>`. +# +# The file below describes the same accelerator as in approach 1. Compare each item with +# the Python code above: ``Accelerator(facility=..., machine=..., energy=..., simulators=[...], +# devices=[...])``, ``Simulator(name=..., lattice=...)`` and ``Quadrupole(name=..., model=...)``. +# +# The lattice path is given by an environment variable, using the ``${env:NAME}`` syntax. +# It could also be written directly as an absolute path, or relative to a root directory. + +configuration = """\ +class: pyaml.accelerator.Accelerator +facility: pyAML test facility +machine: pyaml test machine +energy: 1.0e9 +simulators: + - class: pyaml.lattice.simulator.Simulator + name: design + lattice: ${env:PYAML_TEST_LATTICE} +devices: + - class: pyaml.magnet.quadrupole.Quadrupole + name: QF_001 + model: + class: pyaml.magnet.identity_model.IdentityMagnetModel + physics: '' +""" with open("config.yaml", "w", encoding="utf-8") as file: - yaml.safe_dump(data, file, sort_keys=False) + file.write(configuration) # %% -# Specify the Path to the Configuration File -# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +# Specify the Paths +# ~~~~~~~~~~~~~~~~~ # The path to the configuration file can be specified as absolute or relative to a root directory. # Set the root directory @@ -174,19 +227,12 @@ current_dir = Path.cwd() ROOT.set(current_dir) -# Display the loaded content +# Display the content of the file config_path = Path('config.yaml') print(config_path.read_text()) - # %% -# Specify the Path to the Lattice File -# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -# The path to the lattice file can be specified in the configuration file -# as absolute or relative to the root directory. -# -# It is also possible to specify it using an environment variable. -# The syntax for that is shown in this example. +# Set the environment variable used in the configuration file for the lattice path. import os os.environ["PYAML_TEST_LATTICE"] = lattice_file @@ -205,3 +251,22 @@ quad.strength.get() # %% +# Mistakes Are Detected +# ~~~~~~~~~~~~~~~~~~~~~ +# Since each field must be a constructor argument, a misspelled or unknown field is +# rejected when the configuration is loaded. Here ``phyiscs`` is written instead of ``physics``: + +wrong_configuration = configuration.replace("physics:", "phyiscs:") + +with open("config.yaml", "w", encoding="utf-8") as file: + file.write(wrong_configuration) + +try: + Accelerator.load("config.yaml") +except Exception as error: + print(type(error).__name__) + print(error) + +# Restore the correct configuration file +with open("config.yaml", "w", encoding="utf-8") as file: + file.write(configuration) diff --git a/docs/tutorials/functionality/02_inspect_accelerator.py b/docs/tutorials/functionality/02_inspect_accelerator.py index 88e789f..b201ca6 100644 --- a/docs/tutorials/functionality/02_inspect_accelerator.py +++ b/docs/tutorials/functionality/02_inspect_accelerator.py @@ -20,6 +20,11 @@ ========================================================== This tutorial shows how to inspect and access the content of the accelerator. + +As a reminder, an accelerator contains one or several control modes (for example ``live`` +and ``design``). Each control mode contains the same elements (magnets, BPMs, ...), arrays +(named groups of elements) and tuning tools. See +:doc:`Introduction to pyAML <00_introduction>` for an overview. """ # %% @@ -44,6 +49,45 @@ from pyaml.accelerator import Accelerator accelerator = Accelerator.load(configurations["pyaml/tango/pyaml-cs-oa/fodo_1gev_6d_pyaml-oa.yaml"]) +# %% +# About This Configuration +# ------------------------ +# +# The configuration describes the test lattice introduced in +# :doc:`Create an Accelerator <01_create_accelerator>`: 16 FODO cells, each containing the +# magnets ``QF``, ``SF``, ``COR``, ``QD``, ``SD`` and a ``BPM``. Elements are named after their +# family and cell number, for example ``QF_001``. +# +# The configuration links each element to its name in the control system, which follows a +# TANGO naming convention of the form ``ANcc-AR//.01/``, +# where ``cc`` is the cell number: +# +# ============ ==================================================== +# Element name Control-system name +# ============ ==================================================== +# ``QF_001`` ``AN01-AR/EM-QP/QF.01/magnetic_strength`` +# ``QD_001`` ``AN01-AR/EM-QP/QD.01/magnetic_strength`` +# ``SF_001`` ``AN01-AR/EM-SX/SF.01/magnetic_strength`` +# ``SD_001`` ``AN01-AR/EM-SX/SD.01/magnetic_strength`` +# ``COR_001`` ``AN01-AR/EM-COR/CH.01/magnetic_strength`` (horizontal) +# ``AN01-AR/EM-COR/CV.01/magnetic_strength`` (vertical) +# ``BPM_001`` ``AN01-AR/DG-EPOS/BPM.01/x`` and ``.../BPM.01/y`` +# ============ ==================================================== +# +# It also defines the following arrays and tuning tools, used in the use-case tutorials: +# +# ======================== ========================================================== +# Name Content +# ======================== ========================================================== +# ``Cell1`` ... ``Cell16`` All the elements of one cell +# ``BPM`` The 16 BPMs +# ``HCorr``, ``VCorr`` The horizontal and vertical correctors +# ``QForTune`` The 32 quadrupoles used for tune correction +# ``BETATRON_TUNE`` The betatron tune monitor +# ``DEFAULT_TUNE_...`` Tune correction and tune response matrix +# ``DEFAULT_ORBIT_...`` Orbit correction and orbit response matrix +# ======================== ========================================================== + # %% # Inspect the Accelerator Contents # ------------------------------------ @@ -53,6 +97,16 @@ accelerator.yellow_pages +# %% +# Access the Control Modes +# ------------------------ +# +# Each control mode is an attribute of the accelerator, named after the mode in the +# configuration. ``modes()`` lists them all. + +for name, mode in accelerator.modes().items(): + print(f"accelerator.{name}: {type(mode).__name__}") + # %% # Show the Configuration of an Array # ------------------------------------ @@ -61,9 +115,20 @@ print(quads) # %% -# Show the Configuration of an Magnet in Array -# -------------------------------------------- +# Show the Configuration of a Magnet in an Array +# ----------------------------------------------- print(quads[0]) # %% +# Find the Accepted Configuration Fields +# -------------------------------------- +# +# The configuration fields of the magnet, such as ``name`` and ``model``, are the arguments +# of the constructor of its class. You can check +# which arguments a class accepts, and therefore which fields you can write in a +# configuration file, with ``help()``: + +help(type(quads[0])) + +# %% diff --git a/docs/tutorials/functionality/config.yaml b/docs/tutorials/functionality/config.yaml index 00b68ad..73ba6bf 100644 --- a/docs/tutorials/functionality/config.yaml +++ b/docs/tutorials/functionality/config.yaml @@ -1,15 +1,14 @@ -type: pyaml.accelerator +class: pyaml.accelerator.Accelerator facility: pyAML test facility machine: pyaml test machine -data_folder: null -energy: 1000000000.0 +energy: 1.0e9 simulators: -- type: pyaml.lattice.simulator - lattice: ${env:PYAML_TEST_LATTICE} - name: design + - class: pyaml.lattice.simulator.Simulator + name: design + lattice: ${env:PYAML_TEST_LATTICE} devices: -- type: pyaml.magnet.quadrupole - name: QF_001 - model: - type: pyaml.magnet.identity_model - physics: '' + - class: pyaml.magnet.quadrupole.Quadrupole + name: QF_001 + model: + class: pyaml.magnet.identity_model.IdentityMagnetModel + physics: '' From 5e458d3b864a6c3b10dc27b154afd7ff89da6a12 Mon Sep 17 00:00:00 2001 From: Alexis Gamelin Date: Wed, 23 Sep 2026 17:36:02 +0200 Subject: [PATCH 02/57] Edit --- docs/source/explanation/about.md | 6 +-- docs/source/explanation/architecture.md | 8 ++-- docs/source/explanation/catalog.md | 45 ++++++++++++------- docs/source/explanation/configuration.md | 4 -- docs/source/explanation/control-modes.md | 2 + .../configuration/create-configuration.md | 2 +- 6 files changed, 39 insertions(+), 28 deletions(-) diff --git a/docs/source/explanation/about.md b/docs/source/explanation/about.md index e5e0e37..d387273 100644 --- a/docs/source/explanation/about.md +++ b/docs/source/explanation/about.md @@ -14,11 +14,11 @@ pyAML carries that idea forward with a few important changes: ## Goals -The collaboration has identified the following key features for pyAML: +The collaboration has identified the following key features for pyAML. They are the goals of the project: some of them are already available, others are still being developed. - An agnostic interface between an accelerator control system (TANGO, EPICS, ...), a virtual accelerator and a digital model. - A base for developing and sharing beam measurement tools, such as orbit, trajectory, linear and non-linear optics corrections. -- A virtual accelerator / digital twin which allows testing tuning tools in real-life conditions without the need for beam time. +- The possibility to use a virtual accelerator or a digital twin, to test tuning tools in real-life conditions without the need for beam time. - Handling of both physics and hardware units, with a flexible unit-conversion interface. - The possibility to configure different types of accelerators: transfer lines, linear and circular accelerators, and ramped accelerators. - Configuration and measurement data managed in a standardized manner. @@ -31,7 +31,7 @@ The collaboration has identified the following key features for pyAML: The software is organized in layers: **Core** -: The features needed to configure a machine and communicate with the different backends: abstraction of devices (magnets, BPMs, tune monitors, ...), grouping of devices in arrays, abstraction of the control system, connection to simulators, and conversion between hardware and physics units. This is the `pyaml` package. +: The features needed to configure a machine and communicate with the different backends: abstraction of devices (magnets, BPMs, tune monitors, ...), grouping of devices in arrays, the simulator backend based on pyAT, conversion between hardware and physics units, and the *abstract interface* to control systems. This is the `pyaml` package. The actual communication with a given control system is implemented in separate packages, the control-system bindings (`tango-pyaml`, `pyaml-cs-oa`), so that a facility only installs the ones it needs. **Common high-level applications** : Tools shared between facilities, built on top of the core: tune and chromaticity correction, response-matrix measurements, orbit correction, dispersion measurement, beam-based alignment, LOCO, etc. diff --git a/docs/source/explanation/architecture.md b/docs/source/explanation/architecture.md index 8f29ee3..a80a4cd 100644 --- a/docs/source/explanation/architecture.md +++ b/docs/source/explanation/architecture.md @@ -8,13 +8,13 @@ pyAML is not a single package but a small ecosystem. You only install what your | Package | Role | | --- | --- | -| `pyaml` (PyPI: `accelerator-middle-layer`) | The core: accelerator, control modes, elements, arrays, unit conversion, configuration loading and validation, tuning tools. It includes the simulator backend based on [pyAT](https://atcollab.github.io/at/p/index.html). | +| `pyaml` (PyPI: `accelerator-middle-layer`) | The core: accelerator, elements, arrays, unit conversion, configuration loading and validation, tuning tools, the simulator backend based on [pyAT](https://atcollab.github.io/at/p/index.html), and the abstract interface (`ControlSystem`, `DeviceAccess`) that control-system bindings implement. It does not communicate with any control system by itself. | | `tango-pyaml` | Control-system bindings for TANGO. | | `pyaml-cs-oa` | Control-system bindings based on [ophyd-async](https://blueskyproject.io/ophyd-async/), supporting EPICS (Channel Access and PV Access) and TANGO. | | Facility packages | Optional packages containing classes specific to one facility (special magnet models, devices, applications, ...). | | `pyaml-test-lattice` | A test lattice with ready-made configurations, used in the tutorials. | -The core never imports a control system directly. A control system is selected in the configuration by naming the class of its bindings. Only the bindings you use have to be installed. See [User Installation](../how-to/installation/user-installation.md) and the [API Reference](../reference/index.md). +The core never imports a control system library directly. A control system is selected in the configuration by naming the class of its bindings, which implement the abstract interface of the core. Only the bindings you use have to be installed. See [User Installation](../how-to/installation/user-installation.md) and the [API Reference](../reference/index.md). ## Object Hierarchy @@ -77,7 +77,7 @@ Using explicit `get()` and `set()` methods instead of plain assignment is a deli ### Arrays -Arrays are named groups of elements, declared in the `arrays` section of the configuration. They allow reading or setting all elements of a group in a single, synchronized call: +Arrays are named groups of elements, declared in the `arrays` section of the configuration. They allow reading or setting all elements of a group in a single call. When the backend supports it, the individual requests are grouped (for example `pyaml-cs-oa` sends them concurrently, and the simulator computes the closed orbit once for all the BPMs of an array): ```python quads = accelerator.design.magnets.get("QForTune") @@ -102,7 +102,7 @@ At the bottom of the hierarchy, attributes talk to a backend: ## Discovering What Is Available -The accelerator provides a `yellow_pages` object listing everything that is configured (arrays, tools, diagnostics) and in which control modes it is available: +The accelerator provides a `yellow_pages` object listing everything that is configured: control modes, arrays, tools and diagnostics. Its `availability()` method tells in which control modes a given entry is available: ```python print(accelerator.yellow_pages) diff --git a/docs/source/explanation/catalog.md b/docs/source/explanation/catalog.md index e5c7ca9..1055360 100644 --- a/docs/source/explanation/catalog.md +++ b/docs/source/explanation/catalog.md @@ -26,13 +26,24 @@ Example of configuration for dynamic catalog: ```yaml controls: - - type: pyaml_cs_oa.controlsystem + - class: pyaml_cs_oa.controlsystem.OphydAsyncControlSystem name: live - catalog: - - type: pyaml_cs_oa.dynamic_catalog - backend: tango + catalog: + class: pyaml_cs_oa.dynamic_catalog.DynamicCatalog + backend: tango ``` +With `pyaml-cs-oa`, a dynamic catalog is also used when no catalog is given, based on the `backend` field of the control system: + +```yaml +controls: + - class: pyaml_cs_oa.controlsystem.OphydAsyncControlSystem + name: live + backend: tango +``` + +Dynamic catalogs are currently provided by `pyaml-cs-oa`. Check the documentation of your control-system bindings for the catalogs they support. + ## Static Catalog The static catalog is mainly intended for testing purposes. It consists of a file of entries where each entry corresponds to the configuration for a specific key. It can be seen as a simple, static database of control system signal configurations. @@ -43,22 +54,24 @@ Example of configuration for static catalog: ```yaml controls: -- type: pyaml_cs_oa.controlsystem - name: live - catalog: fodo_1gev_6d_pyaml_catalogs-oa.yaml -``` + - class: pyaml_cs_oa.controlsystem.OphydAsyncControlSystem + name: live + catalog: fodo_1gev_6d_pyaml_catalogs-oa.yaml +``` -Example of an entry in the static catalog: +Example of a static catalog file for `pyaml-cs-oa`, with one entry: ```yaml -class: tango.pyaml.static_catalog.StaticCatalog +class: pyaml_cs_oa.static_catalog.StaticCatalog entries: -- class: tango.pyaml.static_catalog_entry.StaticCatalogEntry - key: AN01-AR/EM-QP/QF.01/magnetic_strength - device: - class: tango.pyaml.attribute.Attribute - attribute: AN01-AR/EM-QP/QF.01/magnetic_strength - unit: 1/m + - class: pyaml_cs_oa.static_catalog_entry.StaticCatalogEntry + key: AN01-AR/EM-QP/QF.01/magnetic_strength + device: + class: pyaml_cs_oa.tangoAtt.TangoAtt + attribute: AN01-AR/EM-QP/QF.01/magnetic_strength + unit: 1/m ``` +The same catalog for `tango-pyaml` uses the classes of that package (`tango.pyaml.static_catalog.StaticCatalog`, `tango.pyaml.static_catalog_entry.StaticCatalogEntry` and `tango.pyaml.attribute.Attribute`). + This format follows the same syntax as for the rest of the pyAML configuration since during the loading process the file is read and the content added to the rest of the pyAML configuration. \ No newline at end of file diff --git a/docs/source/explanation/configuration.md b/docs/source/explanation/configuration.md index ad324b4..1260190 100644 --- a/docs/source/explanation/configuration.md +++ b/docs/source/explanation/configuration.md @@ -108,10 +108,6 @@ model: physics: AN01-AR/EM-QP/QF.01/magnetic_strength ``` -```{note} -Older configurations, including the ones of the `pyaml-test-lattice` package, use the legacy `type` field instead of `class`. It contains the path of the **module** instead of the class, for example `type: pyaml.magnet.quadrupole`. The class is then found from the module. This form is still supported, but `class` with the full class path is recommended for new configurations. The two forms cannot be mixed in the same item. -``` - ## Separation between Configuration and Source Code The configuration describes what should be constructed; it does not contain executable Python code. This keeps configuration readable, reviewable, and usable by tools such as JSON Schema editors. diff --git a/docs/source/explanation/control-modes.md b/docs/source/explanation/control-modes.md index 31b025f..b246082 100644 --- a/docs/source/explanation/control-modes.md +++ b/docs/source/explanation/control-modes.md @@ -32,6 +32,8 @@ controls: - class: pyaml_cs_oa.controlsystem.OphydAsyncControlSystem name: live catalog: catalog.yaml +devices: + # ... magnets, BPMs, tuning tools ``` ```python diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index dc78076..8c3fb74 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -184,7 +184,7 @@ arrays: ## Split the Configuration into Several Files -For a real machine the configuration becomes long. Any string ending with `.yaml`, `.yml` or `.json` in a list is replaced by the content of that file. If the file contains a list, its items are added to the parent list. For example, move the quadrupoles to `devices/quadrupoles.yaml`: +For a real machine the configuration becomes long. When the configuration is loaded from a file, any string value ending with `.yaml`, `.yml` or `.json` is replaced by the content of that file (this is how `catalog: catalog.yaml` above is loaded). Inside a list, if the file contains a list, its items are added to the parent list. For example, move the quadrupoles to `devices/quadrupoles.yaml`: ```yaml # devices/quadrupoles.yaml From 518565ed7f43ddb6c45d58692804dea5910458e3 Mon Sep 17 00:00:00 2001 From: Alexis Gamelin Date: Wed, 23 Sep 2026 17:57:24 +0200 Subject: [PATCH 03/57] Edit 2 --- docs/source/_static/fodo-cell.svg | 14 +- docs/source/index.md | 4 +- .../functionality/02_inspect_accelerator.py | 129 +++++++++--------- 3 files changed, 78 insertions(+), 69 deletions(-) diff --git a/docs/source/_static/fodo-cell.svg b/docs/source/_static/fodo-cell.svg index 3dedfab..2b7a2fe 100644 --- a/docs/source/_static/fodo-cell.svg +++ b/docs/source/_static/fodo-cell.svg @@ -8,7 +8,9 @@ .qd { fill: #e69138; } .sx { fill: #93c47d; } .cor { fill: #b4a7d6; } - .bpm { fill: #f1c232; } + .bpm { fill: #f1c232; stroke: #3b4150; stroke-width: 1.2; } + .bpmlink { stroke: #d4a017; stroke-width: 1.5; stroke-dasharray: 3 2; } + .bpmx { stroke: #3b4150; stroke-width: 1.2; } .name { fill: #8a8f98; font-weight: bold; } .dev { fill: #8a8f98; font-family: Menlo, Consolas, monospace; font-size: 11px; } .ax { stroke: #8a8f98; stroke-width: 1; } @@ -22,10 +24,14 @@ - - + + + + + + @@ -38,7 +44,7 @@ QF SF - BPM + BPM COR B QD diff --git a/docs/source/index.md b/docs/source/index.md index 13cff3a..3ffd40c 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -6,7 +6,7 @@ html_theme.sidebar_secondary.remove: true Python Accelerator Middle Layer (pyAML) is a joint technology platform for design, commissioning, and operation of particle accelerators. -It is developed by a collaboration of accelerator facilities to give physicists a common, physics-oriented way to access their machines, whether real or simulated, and to share measurement and correction tools between laboratories. Read [What is pyAML?](explanation/about.md) to learn more about the motivation and goals of the project. +It is developed by a collaboration of accelerator facilities to give physicists a common, physics-oriented way to access their machines, whether real or simulated, and to share measurement and correction tools between laboratories. The features include, among others: @@ -17,6 +17,8 @@ The features include, among others: - Automatic generation of metadata and a standardized format for measurement data. - A set of standard applications and a framework for developing new applications. +Read [What is pyAML?](explanation/about.md) to learn more about the motivation and goals of the project. + ```{toctree} :hidden: :maxdepth: 1 diff --git a/docs/tutorials/functionality/02_inspect_accelerator.py b/docs/tutorials/functionality/02_inspect_accelerator.py index b201ca6..2dc3831 100644 --- a/docs/tutorials/functionality/02_inspect_accelerator.py +++ b/docs/tutorials/functionality/02_inspect_accelerator.py @@ -19,7 +19,8 @@ Inspect an Accelerator ========================================================== -This tutorial shows how to inspect and access the content of the accelerator. +This tutorial shows how to find out what an accelerator contains using the *yellow pages*, +and how to access what you found. As a reminder, an accelerator contains one or several control modes (for example ``live`` and ``design``). Each control mode contains the same elements (magnets, BPMs, ...), arrays @@ -50,85 +51,85 @@ accelerator = Accelerator.load(configurations["pyaml/tango/pyaml-cs-oa/fodo_1gev_6d_pyaml-oa.yaml"]) # %% -# About This Configuration -# ------------------------ +# The Yellow Pages +# ---------------- # -# The configuration describes the test lattice introduced in -# :doc:`Create an Accelerator <01_create_accelerator>`: 16 FODO cells, each containing the -# magnets ``QF``, ``SF``, ``COR``, ``QD``, ``SD`` and a ``BPM``. Elements are named after their -# family and cell number, for example ``QF_001``. +# The yellow pages are a directory of everything the accelerator provides: the control modes +# and, sorted by category, the arrays, the tuning tools and the diagnostics. They are built +# automatically by scanning every control mode, so they always reflect what is configured. # -# The configuration links each element to its name in the control system, which follows a -# TANGO naming convention of the form ``ANcc-AR//.01/``, -# where ``cc`` is the cell number: -# -# ============ ==================================================== -# Element name Control-system name -# ============ ==================================================== -# ``QF_001`` ``AN01-AR/EM-QP/QF.01/magnetic_strength`` -# ``QD_001`` ``AN01-AR/EM-QP/QD.01/magnetic_strength`` -# ``SF_001`` ``AN01-AR/EM-SX/SF.01/magnetic_strength`` -# ``SD_001`` ``AN01-AR/EM-SX/SD.01/magnetic_strength`` -# ``COR_001`` ``AN01-AR/EM-COR/CH.01/magnetic_strength`` (horizontal) -# ``AN01-AR/EM-COR/CV.01/magnetic_strength`` (vertical) -# ``BPM_001`` ``AN01-AR/DG-EPOS/BPM.01/x`` and ``.../BPM.01/y`` -# ============ ==================================================== -# -# It also defines the following arrays and tuning tools, used in the use-case tutorials: -# -# ======================== ========================================================== -# Name Content -# ======================== ========================================================== -# ``Cell1`` ... ``Cell16`` All the elements of one cell -# ``BPM`` The 16 BPMs -# ``HCorr``, ``VCorr`` The horizontal and vertical correctors -# ``QForTune`` The 32 quadrupoles used for tune correction -# ``BETATRON_TUNE`` The betatron tune monitor -# ``DEFAULT_TUNE_...`` Tune correction and tune response matrix -# ``DEFAULT_ORBIT_...`` Orbit correction and orbit response matrix -# ======================== ========================================================== +# Printing them gives an overview: + +yp = accelerator.yellow_pages +print(yp) # %% -# Inspect the Accelerator Contents -# ------------------------------------ -# -# The yellow pages provide an overview of the accelerator. -# This shows what is configured and available for use. +# Categories and Entries +# ~~~~~~~~~~~~~~~~~~~~~~ +# The entries are sorted in categories. ``keys()`` lists the entries of one category, or of +# all categories if none is given. -accelerator.yellow_pages +print(yp.categories()) +print("Arrays: ", yp.keys("Arrays")) +print("Tools: ", yp.keys("Tools")) +print("Diagnostics:", yp.keys("Diagnostics")) # %% -# Access the Control Modes -# ------------------------ -# -# Each control mode is an attribute of the accelerator, named after the mode in the -# configuration. ``modes()`` lists them all. +# Check an Entry +# ~~~~~~~~~~~~~~ +# ``has()`` tells whether an entry exists, and ``availability()`` in which control modes it +# can be used. -for name, mode in accelerator.modes().items(): - print(f"accelerator.{name}: {type(mode).__name__}") +print(yp.has("QForTune"), yp.has("QF")) +print(yp.availability("DEFAULT_TUNE_CORRECTION")) # %% -# Show the Configuration of an Array -# ------------------------------------ +# Get an Object +# ~~~~~~~~~~~~~ +# An entry can be accessed as an attribute of the yellow pages. The result gives the object +# in each control mode where it is available. -quads = accelerator.design.magnets.get("QForTune") -print(quads) +qfortune = yp.QForTune +print(qfortune.keys()) # %% -# Show the Configuration of a Magnet in an Array -# ----------------------------------------------- +# The object of a given mode is the same as the one reached through the control mode itself: -print(quads[0]) +print(qfortune["design"] is accelerator.design.magnets.get("QForTune")) # %% -# Find the Accepted Configuration Fields -# -------------------------------------- -# -# The configuration fields of the magnet, such as ``name`` and ``model``, are the arguments -# of the constructor of its class. You can check -# which arguments a class accepts, and therefore which fields you can write in a -# configuration file, with ``help()``: +# Search Element Names +# ~~~~~~~~~~~~~~~~~~~~ +# Indexing the yellow pages searches the names of the elements. With the name of an array, +# it returns the names of the elements of the array: + +print(yp["Cell1"]) + +# %% +# Wildcards can be used to match several names: + +print(yp["QF_00*"]) + +# %% +# For more complex searches, use a regular expression by starting the query with ``re:``: + +print(yp["re:^(QF|QD)_01[0-2]$"]) + +# %% +# ``get()`` does the same search and can be restricted to one control mode. Here the +# correctors ``COR_00x`` are combined-function magnets: the search returns both the magnets and +# their horizontal (``.hcorrector``) and vertical (``.vcorrector``) parts, which are the +# elements of the ``HCorr`` and ``VCorr`` arrays. + +print(yp.get("COR_00*", mode="design")) + +# %% +# Use the Result +# ~~~~~~~~~~~~~~ +# The names returned by a search can be used to access the elements in any control mode: -help(type(quads[0])) +for name in yp["re:^QF_00[1-3]$"]: + magnet = accelerator.design.magnet.get(name) + print(name, magnet.strength.get()) # %% From a27c9941986689cdf655595e8be56c81ed70ddea Mon Sep 17 00:00:00 2001 From: Alexis Gamelin Date: Thu, 24 Sep 2026 09:48:27 +0200 Subject: [PATCH 04/57] Separate glossary --- docs/source/explanation/about.md | 29 +----------------- docs/source/explanation/glossary.md | 30 +++++++++++++++++++ docs/source/explanation/index.md | 1 + .../functionality/00_introduction.py | 3 +- 4 files changed, 34 insertions(+), 29 deletions(-) create mode 100644 docs/source/explanation/glossary.md diff --git a/docs/source/explanation/about.md b/docs/source/explanation/about.md index d387273..915ffb1 100644 --- a/docs/source/explanation/about.md +++ b/docs/source/explanation/about.md @@ -52,31 +52,4 @@ A facility adopts pyAML by writing a *configuration* describing its machine. The There is one simple rule behind the configuration: each item names a Python class, and each of its other fields is an argument of that class's constructor. See [Configuration Structure and Syntax](configuration.md). -## Glossary - -Accelerator -: The top-level pyAML object describing one machine (a storage ring, a booster, a transfer line, ...). It holds all the control modes, arrays and devices. - -Control mode -: One way of accessing the accelerator, for example `live` (the real machine) or `design` (a simulation). All control modes offer the same interface. See [Control Modes](control-modes.md). - -Element -: A single object of the accelerator that can be read or set: a magnet, a BPM, an RF plant, a tune monitor, ... - -Array -: A named group of elements, for example all BPMs or all the quadrupoles used for tune correction, that can be read or set in one call. - -Tuning tool -: A high-level measurement or correction tool (tune correction, orbit correction, response matrix measurement, ...) configured like any other device. - -Backend -: The system that a control mode talks to: a control system (through control-system bindings such as `tango-pyaml` or `pyaml-cs-oa`) or a simulation code (pyAT). - -Catalog -: The backend-specific description of how a key used in the configuration maps to a signal of the control system. See [Control System Catalogs](catalog.md). - -Virtual accelerator -: A simulated machine exposed through a real control system (for example TANGO devices backed by a simulation), so that it can be used exactly like the real machine. - -Digital shadow / digital twin -: A simulation that follows the real machine. In a shadow, changes on the real machine are reflected in the simulation, but not the other way around. In a twin, changes are reflected both ways. +The terms used throughout the documentation are defined in the [Glossary](glossary.md). diff --git a/docs/source/explanation/glossary.md b/docs/source/explanation/glossary.md new file mode 100644 index 0000000..b6da766 --- /dev/null +++ b/docs/source/explanation/glossary.md @@ -0,0 +1,30 @@ +# Glossary + +Terms used throughout the pyAML documentation. + +Accelerator +: The top-level pyAML object describing one machine (a storage ring, a booster, a transfer line, ...). It holds all the control modes, arrays and devices. + +Control mode +: One way of accessing the accelerator, for example `live` (the real machine) or `design` (a simulation). All control modes offer the same interface. See [Control Modes](control-modes.md). + +Element +: A single object of the accelerator that can be read or set: a magnet, a BPM, an RF plant, a tune monitor, ... + +Array +: A named group of elements, for example all BPMs or all the quadrupoles used for tune correction, that can be read or set in one call. + +Tuning tool +: A high-level measurement or correction tool (tune correction, orbit correction, response matrix measurement, ...) configured like any other device. + +Backend +: The system that a control mode talks to: a control system (through control-system bindings such as `tango-pyaml` or `pyaml-cs-oa`) or a simulation code (pyAT). + +Catalog +: The backend-specific description of how a key used in the configuration maps to a signal of the control system. See [Control System Catalogs](catalog.md). + +Virtual accelerator +: A simulated machine exposed through a real control system (for example TANGO devices backed by a simulation), so that it can be used exactly like the real machine. + +Digital shadow / digital twin +: A simulation that follows the real machine. In a shadow, changes on the real machine are reflected in the simulation, but not the other way around. In a twin, changes are reflected both ways. diff --git a/docs/source/explanation/index.md b/docs/source/explanation/index.md index a59fd96..03ce3bd 100644 --- a/docs/source/explanation/index.md +++ b/docs/source/explanation/index.md @@ -11,4 +11,5 @@ control-modes configuration schema_and_validation catalog +glossary ``` diff --git a/docs/tutorials/functionality/00_introduction.py b/docs/tutorials/functionality/00_introduction.py index 6851ce9..69e659c 100644 --- a/docs/tutorials/functionality/00_introduction.py +++ b/docs/tutorials/functionality/00_introduction.py @@ -71,7 +71,8 @@ # argument of that class's constructor**. This rule is explored in the first tutorial. # # See :doc:`pyAML Structure <../../explanation/architecture>` and -# :doc:`Control Modes <../../explanation/control-modes>` for more details. +# :doc:`Control Modes <../../explanation/control-modes>` for more details, and the +# :doc:`Glossary <../../explanation/glossary>` for the definitions of the terms used in pyAML. # %% # How the Tutorials Work From 1522d34c9355d241a3665c427e6f9a34c33b0dec Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 11:52:01 +0200 Subject: [PATCH 05/57] Made the link to what is pyAML more visible by adding a tip. --- docs/source/index.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/source/index.md b/docs/source/index.md index 3ffd40c..9a16fa4 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -17,7 +17,9 @@ The features include, among others: - Automatic generation of metadata and a standardized format for measurement data. - A set of standard applications and a framework for developing new applications. +```{tip} Read [What is pyAML?](explanation/about.md) to learn more about the motivation and goals of the project. +``` ```{toctree} :hidden: From a81249a4878456314acf36f1799b642558580644 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 12:02:47 +0200 Subject: [PATCH 06/57] Updated the description of pyAML to include that it is more than a single library. --- docs/source/explanation/about.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/explanation/about.md b/docs/source/explanation/about.md index 915ffb1..d74d74e 100644 --- a/docs/source/explanation/about.md +++ b/docs/source/explanation/about.md @@ -1,6 +1,6 @@ # What is pyAML? -The Python Accelerator Middle Layer (pyAML) is a Python library that sits between the people who operate or study a particle accelerator and the many systems that make up that accelerator: control systems, simulation codes, archivers, databases, and so on. It is developed by a collaboration of accelerator facilities as a common platform for the design, commissioning, and operation of particle accelerators. +The Python Accelerator Middle Layer (pyAML) is an ecosystem of Python packages that provides a common layer between people who operate or study particle accelerators and the different tools they need to work with, such as control systems and simulation codes. It is developed by a collaboration of accelerator facilities as a common framework for the design, commissioning, and operation of particle accelerators. ## Why a Middle Layer? From a2d92ffb800db91601f1b22fb8e15ac033760249 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 12:28:23 +0200 Subject: [PATCH 07/57] Updated why a middle layer part to better explain what a middle layer is an focus the list on features that are new in pyAML. --- docs/source/explanation/about.md | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/docs/source/explanation/about.md b/docs/source/explanation/about.md index d74d74e..622f4a3 100644 --- a/docs/source/explanation/about.md +++ b/docs/source/explanation/about.md @@ -4,13 +4,22 @@ The Python Accelerator Middle Layer (pyAML) is an ecosystem of Python packages t ## Why a Middle Layer? -Many accelerator facilities have for years relied on a *Middle Layer*, the MATLAB Middle Layer (MML) being the best-known example. A middle layer gives physicists a uniform, physics-oriented way to access a machine: they read "the orbit at all BPMs" or set "the strength of quadrupole QF1" without having to know which control system variable holds that value, which units it is in, or how a current is converted into a magnetic strength. +Many accelerator facilities rely on a *Middle Layer*, with the [MATLAB Middle Layer (MML)](https://github.com/atcollab/MML) being a well-known example at synchrotron light sources. -pyAML carries that idea forward with a few important changes: +A middle layer provides a physics-oriented interface to a particle accelerator. It represents the machine in terms of familiar accelerator concepts, such as magnets, BPMs, and RF cavities, while hiding details such as control-system variables, hardware interfaces, units, and conversions from users. + +The same interface can be used to interact with either the real accelerator, where values are read from and written to the control system, or a simulated accelerator, where they are obtained from and passed to a simulation code. + +PyAML builds on this idea, but is implemented as a modern, Python-based, and extensible framework. Its key characteristics are: + + - **Python-native**: pyAML integrates naturally with the Python scientific ecosystem and the growing number of accelerator-physics tools available in Python. + +- **Modular and extensible**: Control systems, simulation codes, and physics applications are connected through well-defined interfaces, allowing new implementations to be added without changing the applications that use them. + +- **Configuration-driven**: Accelerator-specific information is kept separate from application code, allowing the same software to be configured for different machines and facilities. + +- **Real and virtual machines**: The same interface can be used with the real accelerator, an external digital twin, or an internal simulator, allowing physics applications to work independently of how the accelerator is represented. -- **Python instead of MATLAB.** Python is open, free and widely used in the scientific community, and it gives access to a large ecosystem of scientific and machine-learning tools. -- **Shared between facilities.** Measurement and correction tools written for pyAML (orbit correction, tune correction, response matrices, etc.) should run at any facility that has configured pyAML, instead of being rewritten at each laboratory. -- **Simulation as a first-class citizen.** The same script can act on the real machine or on a simulated one. Tools can then be developed and tested without using expensive and limited beam time. ## Goals From 1feac741a0988d6cd423b4af9dd1e0e1b918646b Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 12:33:08 +0200 Subject: [PATCH 08/57] Remove space. --- docs/source/explanation/about.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/source/explanation/about.md b/docs/source/explanation/about.md index 622f4a3..1525a2d 100644 --- a/docs/source/explanation/about.md +++ b/docs/source/explanation/about.md @@ -20,7 +20,6 @@ PyAML builds on this idea, but is implemented as a modern, Python-based, and ext - **Real and virtual machines**: The same interface can be used with the real accelerator, an external digital twin, or an internal simulator, allowing physics applications to work independently of how the accelerator is represented. - ## Goals The collaboration has identified the following key features for pyAML. They are the goals of the project: some of them are already available, others are still being developed. From 069e6f29a1dfb6ac35186010440cf781b7fe994a Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 12:43:33 +0200 Subject: [PATCH 09/57] Add that the test lattice is also used for integration tests. --- docs/source/explanation/architecture.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/explanation/architecture.md b/docs/source/explanation/architecture.md index a80a4cd..73b053f 100644 --- a/docs/source/explanation/architecture.md +++ b/docs/source/explanation/architecture.md @@ -12,7 +12,7 @@ pyAML is not a single package but a small ecosystem. You only install what your | `tango-pyaml` | Control-system bindings for TANGO. | | `pyaml-cs-oa` | Control-system bindings based on [ophyd-async](https://blueskyproject.io/ophyd-async/), supporting EPICS (Channel Access and PV Access) and TANGO. | | Facility packages | Optional packages containing classes specific to one facility (special magnet models, devices, applications, ...). | -| `pyaml-test-lattice` | A test lattice with ready-made configurations, used in the tutorials. | +| `pyaml-test-lattice` | A test lattice with ready-made configurations, used in the tutorials and for integration tests. | The core never imports a control system library directly. A control system is selected in the configuration by naming the class of its bindings, which implement the abstract interface of the core. Only the bindings you use have to be installed. See [User Installation](../how-to/installation/user-installation.md) and the [API Reference](../reference/index.md). From 69042bea5bf9c1f333fc059480d3634897aa6123 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 12:50:53 +0200 Subject: [PATCH 10/57] Change that all measurements behave in the same way to similar way. --- docs/source/explanation/control-modes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/explanation/control-modes.md b/docs/source/explanation/control-modes.md index b246082..823b862 100644 --- a/docs/source/explanation/control-modes.md +++ b/docs/source/explanation/control-modes.md @@ -5,7 +5,7 @@ A control mode is one way of accessing the accelerator. pyAML is built so that t - Core interactions work identically in all control modes. - All configured control modes are available at all times, and can be used at the same time in one script. - All control modes are defined in the configuration. -- Standard measurements and high-level applications behave in the same way in every control mode. +- Standard measurements and high-level applications behave in a similar way in every control mode. ## Available Control Modes From 1db6e259ad1811a406cff81db553f72e191924c5 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 16:11:28 +0200 Subject: [PATCH 11/57] Mark pyAT as a name as for other names. --- docs/source/explanation/glossary.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/explanation/glossary.md b/docs/source/explanation/glossary.md index b6da766..4edc826 100644 --- a/docs/source/explanation/glossary.md +++ b/docs/source/explanation/glossary.md @@ -18,7 +18,7 @@ Tuning tool : A high-level measurement or correction tool (tune correction, orbit correction, response matrix measurement, ...) configured like any other device. Backend -: The system that a control mode talks to: a control system (through control-system bindings such as `tango-pyaml` or `pyaml-cs-oa`) or a simulation code (pyAT). +: The system that a control mode talks to: a control system (through control-system bindings such as `tango-pyaml` or `pyaml-cs-oa`) or a simulation code (`pyAT`). Catalog : The backend-specific description of how a key used in the configuration maps to a signal of the control system. See [Control System Catalogs](catalog.md). From 79efc5daa265551410e33f5a89bf94ae3f538236 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 16:18:48 +0200 Subject: [PATCH 12/57] Added a tip to highlight that using the dynamic catalog is the recommended option. --- docs/source/explanation/catalog.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/source/explanation/catalog.md b/docs/source/explanation/catalog.md index 1055360..754e697 100644 --- a/docs/source/explanation/catalog.md +++ b/docs/source/explanation/catalog.md @@ -16,7 +16,9 @@ The catalog makes it possible to use simple keys in the pyAML configuration and ## Dynamic Catalog +```{tip} The dynamic catalog does not require a configuration file and it is therefore the recommended option for most use cases. +``` In this version, the configuration is extracted from a dynamic source. This can be directly from the control system or some other source, for example a database, depending on the chosen backend and its catalog implementation. From 8a31ec6cfc51e7bcd475ba181cb7e8ba96ef2ba9 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 16:30:40 +0200 Subject: [PATCH 13/57] List classes for static catalog in tango-pyaml as a list. --- docs/source/explanation/catalog.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/source/explanation/catalog.md b/docs/source/explanation/catalog.md index 754e697..a8c8d9c 100644 --- a/docs/source/explanation/catalog.md +++ b/docs/source/explanation/catalog.md @@ -61,7 +61,7 @@ controls: catalog: fodo_1gev_6d_pyaml_catalogs-oa.yaml ``` -Example of a static catalog file for `pyaml-cs-oa`, with one entry: +Example of a static catalog file for `pyaml-cs-oa` with a single entry: ```yaml class: pyaml_cs_oa.static_catalog.StaticCatalog @@ -74,6 +74,9 @@ entries: unit: 1/m ``` -The same catalog for `tango-pyaml` uses the classes of that package (`tango.pyaml.static_catalog.StaticCatalog`, `tango.pyaml.static_catalog_entry.StaticCatalogEntry` and `tango.pyaml.attribute.Attribute`). +The static catalog for `tango-pyaml` uses the corresponding classes for that package: +- `tango.pyaml.static_catalog.StaticCatalog` +- `tango.pyaml.static_catalog_entry.StaticCatalogEntry` +- `tango.pyaml.attribute.Attribute` This format follows the same syntax as for the rest of the pyAML configuration since during the loading process the file is read and the content added to the rest of the pyAML configuration. \ No newline at end of file From 8ac91ca021584d4235f7d5f9269125a4f7104d44 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 16:41:35 +0200 Subject: [PATCH 14/57] Specify that each field is the name of the field in the constructor and the value to be passed. --- docs/source/explanation/configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/explanation/configuration.md b/docs/source/explanation/configuration.md index 1260190..aaf2461 100644 --- a/docs/source/explanation/configuration.md +++ b/docs/source/explanation/configuration.md @@ -25,7 +25,7 @@ The syntax has been chosen to allow configuration and construction of objects fo The whole configuration follows a single rule: ```{important} -Each configuration item names a Python class in its `class` field. **Every other field of the item is an argument of that class's constructor**, with the same name. +Each configuration item names a Python class in its `class` field. **Every other field of the item is an argument to that class's constructor**, with the same name as the field and the value to be passed to the constructor. ``` When pyAML reads an item, it imports the class given by `class` and calls it with the remaining fields as keyword arguments. The configuration below and the Python code next to it build exactly the same object: From d230d26ea77c4330d56c417fc74c2d9854aa11a6 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 16:45:50 +0200 Subject: [PATCH 15/57] Remove the grid since it required scrolling to be able to see both sets of codes at the same time. --- docs/source/explanation/configuration.md | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/docs/source/explanation/configuration.md b/docs/source/explanation/configuration.md index aaf2461..0c8efa7 100644 --- a/docs/source/explanation/configuration.md +++ b/docs/source/explanation/configuration.md @@ -28,12 +28,8 @@ The whole configuration follows a single rule: Each configuration item names a Python class in its `class` field. **Every other field of the item is an argument to that class's constructor**, with the same name as the field and the value to be passed to the constructor. ``` -When pyAML reads an item, it imports the class given by `class` and calls it with the remaining fields as keyword arguments. The configuration below and the Python code next to it build exactly the same object: +When pyAML reads an item, it imports the class given by `class` and calls it with the remaining fields as keyword arguments. The YAML configuration and Python code below build equivalent objects: -`````{grid} 2 -:gutter: 2 - -````{grid-item} **Configuration** ```yaml @@ -44,9 +40,7 @@ model: unit: 1/m physics: AN01-AR/EM-QP/QF.01/magnetic_strength ``` -```` -````{grid-item} **Python** ```python @@ -61,8 +55,6 @@ Quadrupole( ), ) ``` -```` -````` Consequences of this rule: From f80e3b62be52ef5b57633f53985ea3cd73c831a7 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 16:47:56 +0200 Subject: [PATCH 16/57] Removed unknown fields are rejected since this is only true if validation is turned on and not a general requirement for the configuration. --- docs/source/explanation/configuration.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/source/explanation/configuration.md b/docs/source/explanation/configuration.md index 0c8efa7..49c9efe 100644 --- a/docs/source/explanation/configuration.md +++ b/docs/source/explanation/configuration.md @@ -60,7 +60,6 @@ Consequences of this rule: - **Nested objects are nested items.** If an argument expects an object (here `model` expects a magnet model), the field contains another item with its own `class` field. Lists of objects (such as `devices` or `simulators` of the `Accelerator`) are lists of items. - **Optional arguments are optional fields.** Arguments with a default value can be left out. -- **Unknown fields are rejected.** A field which is not an argument of the constructor, for example a misspelled one, raises an error when the configuration is loaded. - **Any class can be used.** Nothing is specific to pyAML classes: a facility-specific class from your own package can be used in the same way, as long as it can be imported. ### Finding the Accepted Fields From fda240a2a10c4ddf94079da4c6630293c0bf6e62 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 16:50:00 +0200 Subject: [PATCH 17/57] Changed link for API documentation to the page with links for all packages and not only pyaml. --- docs/source/explanation/configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/explanation/configuration.md b/docs/source/explanation/configuration.md index 49c9efe..074ffa7 100644 --- a/docs/source/explanation/configuration.md +++ b/docs/source/explanation/configuration.md @@ -66,7 +66,7 @@ Consequences of this rule: Since fields are constructor arguments, the documentation of a class tells you what to write in the configuration. You can: -- read the [API documentation](https://pyaml.readthedocs.io/en/stable/) of the class, +- read the [API documentation](./../reference/index.md) of the class, - use `help()` in Python, which shows the signature of the constructor: ```python From 60194c268e1be287858f639b33ab0f0a9fc3bd5a Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 16:55:15 +0200 Subject: [PATCH 18/57] Add JSON schema and link to tools to list of option to find fields. --- docs/source/explanation/configuration.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/source/explanation/configuration.md b/docs/source/explanation/configuration.md index 074ffa7..81b417c 100644 --- a/docs/source/explanation/configuration.md +++ b/docs/source/explanation/configuration.md @@ -66,8 +66,8 @@ Consequences of this rule: Since fields are constructor arguments, the documentation of a class tells you what to write in the configuration. You can: -- read the [API documentation](./../reference/index.md) of the class, -- use `help()` in Python, which shows the signature of the constructor: +- Read the [API documentation](./../reference/index.md) of the class, +- Use `help()` in Python, which shows the signature of the constructor: ```python from pyaml.magnet.quadrupole import Quadrupole @@ -76,7 +76,8 @@ Since fields are constructor arguments, the documentation of a class tells you w # lattice_names: str | None = None, description: str | None = None) ``` -- use the schema registry, whose `describe()` method lists the fields of a registered class with their types. See [Use the Schema Registry](../how-to/configuration/use-schema-registry.ipynb). +- Use the schema registry, whose `describe()` method lists the fields of a registered class with their types. See [Use the Schema Registry](../how-to/configuration/use-schema-registry.ipynb). +- Use a JSON Schema in an external tool. See [JSON Schema Tools](../how-to/configuration/create-configuration.md#tools-that-help-writing-the-configuration). ## Configuration Items From 9d1b0bb94da61a3b5e3333ba88c0a2475003d48e Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 18:36:47 +0200 Subject: [PATCH 19/57] Updated info and config examples for the dynamic catalog. --- docs/source/explanation/catalog.md | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/docs/source/explanation/catalog.md b/docs/source/explanation/catalog.md index a8c8d9c..f449245 100644 --- a/docs/source/explanation/catalog.md +++ b/docs/source/explanation/catalog.md @@ -24,27 +24,26 @@ In this version, the configuration is extracted from a dynamic source. This can This requires access to the source, for example by being on the same network, but no configuration file for the control system configuration has to be loaded by pyAML. -Example of configuration for dynamic catalog: +Example of configuration for dynamic catalog for the `pyaml-cs-oa` bindings : ```yaml controls: - class: pyaml_cs_oa.controlsystem.OphydAsyncControlSystem name: live - catalog: - class: pyaml_cs_oa.dynamic_catalog.DynamicCatalog - backend: tango + backend: tango ``` -With `pyaml-cs-oa`, a dynamic catalog is also used when no catalog is given, based on the `backend` field of the control system: +The backend (currently TANGO or EPICS) is specified using the `backend` field. + +For the `tango-pyaml` bindings no backend needs to be specified: ```yaml controls: - - class: pyaml_cs_oa.controlsystem.OphydAsyncControlSystem + - class: tango.pyaml.tango_catalog.TangoCatalog name: live - backend: tango ``` -Dynamic catalogs are currently provided by `pyaml-cs-oa`. Check the documentation of your control-system bindings for the catalogs they support. +Check the [API documentation](../reference/index.md) for all the options for the bindings you want to use. ## Static Catalog From d118c89332e026dc4864427e6623101364842cd0 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 18:42:34 +0200 Subject: [PATCH 20/57] Move glossary to main menu. --- docs/source/{explanation => }/glossary.md | 0 docs/source/index.md | 1 + 2 files changed, 1 insertion(+) rename docs/source/{explanation => }/glossary.md (100%) diff --git a/docs/source/explanation/glossary.md b/docs/source/glossary.md similarity index 100% rename from docs/source/explanation/glossary.md rename to docs/source/glossary.md diff --git a/docs/source/index.md b/docs/source/index.md index 9a16fa4..7fe7329 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -29,6 +29,7 @@ Tutorials how-to/index explanation/index reference/index +glossary ```
From 375f908539920fddd02724d6072a91fe53ac822d Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 18:51:16 +0200 Subject: [PATCH 21/57] Remove section about finding the fields to move it to the how-to guide instead. --- docs/source/explanation/configuration.md | 17 ----------------- 1 file changed, 17 deletions(-) diff --git a/docs/source/explanation/configuration.md b/docs/source/explanation/configuration.md index 81b417c..02a7f95 100644 --- a/docs/source/explanation/configuration.md +++ b/docs/source/explanation/configuration.md @@ -62,23 +62,6 @@ Consequences of this rule: - **Optional arguments are optional fields.** Arguments with a default value can be left out. - **Any class can be used.** Nothing is specific to pyAML classes: a facility-specific class from your own package can be used in the same way, as long as it can be imported. -### Finding the Accepted Fields - -Since fields are constructor arguments, the documentation of a class tells you what to write in the configuration. You can: - -- Read the [API documentation](./../reference/index.md) of the class, -- Use `help()` in Python, which shows the signature of the constructor: - - ```python - from pyaml.magnet.quadrupole import Quadrupole - help(Quadrupole) - # Quadrupole(name: str, model: MagnetModel | None = None, - # lattice_names: str | None = None, description: str | None = None) - ``` - -- Use the schema registry, whose `describe()` method lists the fields of a registered class with their types. See [Use the Schema Registry](../how-to/configuration/use-schema-registry.ipynb). -- Use a JSON Schema in an external tool. See [JSON Schema Tools](../how-to/configuration/create-configuration.md#tools-that-help-writing-the-configuration). - ## Configuration Items Each configurable item is represented by a mapping which describes the attributes and values needed to construct one Python object. The field `class` (or its alias `class_path`) identifies the type to construct. It should be written as a fully qualified Python class path, consisting of the module and class name. For example: From a1cc72394be5bfcd519d4cad8af51fd124bf5ef4 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 18:59:16 +0200 Subject: [PATCH 22/57] Merge info in configuration item into previous section. --- docs/source/explanation/configuration.md | 27 ++++++------------------ 1 file changed, 6 insertions(+), 21 deletions(-) diff --git a/docs/source/explanation/configuration.md b/docs/source/explanation/configuration.md index 02a7f95..5cc854f 100644 --- a/docs/source/explanation/configuration.md +++ b/docs/source/explanation/configuration.md @@ -28,6 +28,12 @@ The whole configuration follows a single rule: Each configuration item names a Python class in its `class` field. **Every other field of the item is an argument to that class's constructor**, with the same name as the field and the value to be passed to the constructor. ``` +The `class` should be written as a fully qualified Python class path, consisting of the module and class name in the format `package.module.Class`. + +```{note} +Since `class` is a reserved name in Python, the attribute is called `class_path` in the source code. That is also an accepted alias to use in the configuration. +``` + When pyAML reads an item, it imports the class given by `class` and calls it with the remaining fields as keyword arguments. The YAML configuration and Python code below build equivalent objects: **Configuration** @@ -62,27 +68,6 @@ Consequences of this rule: - **Optional arguments are optional fields.** Arguments with a default value can be left out. - **Any class can be used.** Nothing is specific to pyAML classes: a facility-specific class from your own package can be used in the same way, as long as it can be imported. -## Configuration Items - -Each configurable item is represented by a mapping which describes the attributes and values needed to construct one Python object. The field `class` (or its alias `class_path`) identifies the type to construct. It should be written as a fully qualified Python class path, consisting of the module and class name. For example: - -```yaml -class: pyaml.magnet.quadrupole.Quadrupole -``` - -When pyAML reads the configuration, it uses this path to select the class and passes the remaining configuration fields to the constructor of the class. - -An object can contain other configurable objects. The nested objects follow the same principle: each has its own `class` field and the values needed to construct it. For example: - -```yaml -class: pyaml.magnet.quadrupole.Quadrupole -name: QF_001 -model: - class: pyaml.magnet.identity_model.IdentityMagnetModel - unit: 1/m - physics: AN01-AR/EM-QP/QF.01/magnetic_strength -``` - ## Separation between Configuration and Source Code The configuration describes what should be constructed; it does not contain executable Python code. This keeps configuration readable, reviewable, and usable by tools such as JSON Schema editors. From 8387d09cc83b2ba8986a993fee6bdff74d4e1f19 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 19:25:07 +0200 Subject: [PATCH 23/57] Remove glossary from explanations index. --- docs/source/explanation/index.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/source/explanation/index.md b/docs/source/explanation/index.md index 03ce3bd..a59fd96 100644 --- a/docs/source/explanation/index.md +++ b/docs/source/explanation/index.md @@ -11,5 +11,4 @@ control-modes configuration schema_and_validation catalog -glossary ``` From 61a03e834181f14511534dd813cb808dd7828cfc Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 19:51:54 +0200 Subject: [PATCH 24/57] Add tip to read the explanation page before starting. --- docs/source/how-to/configuration/create-configuration.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index 8c3fb74..e4574fc 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -1,6 +1,10 @@ # Create and Load Configuration -This guide shows how to write a pyAML configuration as a text file and load it into an `Accelerator`. The structure and syntax of the configuration are explained in detail in [Configuration Structure and Syntax](../../explanation/configuration). +This guide shows how to write a pyAML configuration as a text file and load it into an `Accelerator`. + +```{tip} +Read [Configuration Structure and Syntax](../../explanation/configuration) which explains the concepts and ideas behind the configuration before you start. +``` ## The Rule to Remember From 210ba64305548028e1788f87d998eeca75cae154 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 19:56:22 +0200 Subject: [PATCH 25/57] Add some more details to the introduction. --- docs/source/how-to/configuration/create-configuration.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index e4574fc..2307f9d 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -1,6 +1,6 @@ # Create and Load Configuration -This guide shows how to write a pyAML configuration as a text file and load it into an `Accelerator`. +This guide shows how to write a pyAML configuration as a text file and load it into an `Accelerator`. It gives recommendations for how to write it and list tools that are available to help. ```{tip} Read [Configuration Structure and Syntax](../../explanation/configuration) which explains the concepts and ideas behind the configuration before you start. @@ -9,7 +9,7 @@ Read [Configuration Structure and Syntax](../../explanation/configuration) which ## The Rule to Remember ```{important} -Each item of the configuration names a Python class in its `class` field. **Every other field is an argument of the constructor of that class**, with the same name. When an argument is an object, its value is a nested item with its own `class` field. +Each item of the configuration names a Python class in its `class` field. **Every other field is an argument of the constructor of that class**, with the same name as the field. When an argument is an object, its value is a nested item with its own `class` field. ``` Writing a configuration is therefore the same as writing the Python code that creates the objects, in YAML (or JSON) instead of Python. From 4064839f8c04fa069e78bcfc1ff1ec141f3e368c Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 20:23:08 +0200 Subject: [PATCH 26/57] Remove in the introduction that the guide is about text file since it also includes dictionary. --- docs/source/how-to/configuration/create-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index 2307f9d..ed35e5e 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -1,6 +1,6 @@ # Create and Load Configuration -This guide shows how to write a pyAML configuration as a text file and load it into an `Accelerator`. It gives recommendations for how to write it and list tools that are available to help. +This guide shows how to write a pyAML configuration and load it into an `Accelerator`. It gives recommendations for how to write it and list tools that are available to help. ```{tip} Read [Configuration Structure and Syntax](../../explanation/configuration) which explains the concepts and ideas behind the configuration before you start. From f4197be4d1a5a6eed07e205575c8a08041231f2a Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 20:33:16 +0200 Subject: [PATCH 27/57] Update the rule to remember section to make it clearer how using a configuration differs from building objects yourself. --- docs/source/how-to/configuration/create-configuration.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index ed35e5e..9642dcc 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -8,11 +8,13 @@ Read [Configuration Structure and Syntax](../../explanation/configuration) which ## The Rule to Remember +The configuration consists of a set of items which tells pyAML which objects to build when loading the configuration. + ```{important} -Each item of the configuration names a Python class in its `class` field. **Every other field is an argument of the constructor of that class**, with the same name as the field. When an argument is an object, its value is a nested item with its own `class` field. +Each item of the configuration names a Python class in its `class` field. **Every other field is an argument of the constructor of that class** with the same name as the field and the value to pass to the constructor. When a constructor argument is an object, its value in the configuration is a nested item with its own `class` field. ``` -Writing a configuration is therefore the same as writing the Python code that creates the objects, in YAML (or JSON) instead of Python. +Writing and loading the configuration is the equivalent of writing the Python code that creates the objects yourself. The configuration just allows pyAML to create the objects for you. ## Find the Fields of a Class From 39485afe13768e967114f0bef0c6e9d90ec8d8f8 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 20:42:06 +0200 Subject: [PATCH 28/57] Update finding the fields section with content from moved from the configuration explanation. --- .../how-to/configuration/create-configuration.md | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index 9642dcc..a3c8984 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -16,11 +16,14 @@ Each item of the configuration names a Python class in its `class` field. **Ever Writing and loading the configuration is the equivalent of writing the Python code that creates the objects yourself. The configuration just allows pyAML to create the objects for you. -## Find the Fields of a Class +## Finding the Accepted Fields -Before writing an item, look up the constructor arguments of its class. Any of these works: +Before writing an item, look up the constructor parameters of the class you want pyAML to build an object of. This can be done in several ways: -- `help()` in Python, which shows the constructor signature and describes each argument: +- Read the [API documentation](./../reference/index.md) of the class +- Use `help()` in Python since this shows the signature of the constructor and description of each parameter + +For example: ```python from pyaml.bpm.bpm import BPM @@ -39,10 +42,10 @@ Before writing an item, look up the constructor arguments of its class. Any of t # | ... ``` -- the [API documentation](https://pyaml.readthedocs.io/en/stable/) of the class, -- the `describe()` method of the schema in the [schema registry](./use-schema-registry.ipynb). +- Use the schema registry. The `describe()` method lists the fields of a registered class with their types. See [Use the Schema Registry](../how-to/configuration/use-schema-registry.ipynb). +- Use a JSON Schema in an external tool. See [JSON Schema Tools](../how-to/configuration/create-configuration.md#tools-that-help-writing-the-configuration). -Arguments without a default value (here `name`) are required fields; the others can be left out. +Parameters with a default value is optional and can be left out of the configuration if you wish. ## Write the Configuration File From 9616c0be9a9c7e94672a7c6b1f43a9dd59116739 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 21:49:40 +0200 Subject: [PATCH 29/57] Add that the configuration can be written as yaml, json or dict. --- docs/source/how-to/configuration/create-configuration.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index a3c8984..5b7f819 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -5,6 +5,7 @@ This guide shows how to write a pyAML configuration and load it into an `Acceler ```{tip} Read [Configuration Structure and Syntax](../../explanation/configuration) which explains the concepts and ideas behind the configuration before you start. ``` +The configuration can be written as a text file in YAML or JSON or as a dictionary. ## The Rule to Remember From 33db49d569680c1e73626d3907341c4da1496f87 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 21:59:15 +0200 Subject: [PATCH 30/57] Change heading to make clear that the example is for using a text file since the dictionary option also exist. --- docs/source/how-to/configuration/create-configuration.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index 5b7f819..92a6456 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -48,9 +48,11 @@ For example: Parameters with a default value is optional and can be left out of the configuration if you wish. -## Write the Configuration File +## Write Configuration as a Text File -Create a file, for example `accelerator.yaml`, with any text editor. The steps below build a small but complete configuration, using the names of the [test lattice](../../tutorials/functionality/01_create_accelerator). +Here an example is shown for how to create the configuration in a YAML file. The steps are similar if using JSON. The steps below build a small but complete configuration, using the names of the [test lattice](../../tutorials/functionality/01_create_accelerator). + +Create a file, for example `accelerator.yaml`, with any text editor. If you want the editor to suggest the fields, you can use VS Code together with a JSON Schema. See [Use JSON Schema in VS Code](../configuration/use-vscode-json-schema.md) for instructions. ### 1. The Accelerator From bfd9280c65ea0290945d1627f7e362307f6669f4 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Thu, 24 Sep 2026 22:05:00 +0200 Subject: [PATCH 31/57] Update the facility and machine name. --- docs/source/how-to/configuration/create-configuration.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index 92a6456..128dc1b 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -60,8 +60,8 @@ The root item is the `Accelerator`. Its required arguments are `facility`, `mach ```yaml class: pyaml.accelerator.Accelerator -facility: My facility -machine: sr +facility: my_facility +machine: storage_ring energy: 1.0e9 ``` From 0d5ad1dc2707a6bde62815c1cfcd819e8af3ddd5 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 25 Sep 2026 08:21:25 +0200 Subject: [PATCH 32/57] Update the section about control modes since some information wrong. --- .../configuration/create-configuration.md | 25 +++++-------------- 1 file changed, 6 insertions(+), 19 deletions(-) diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index 128dc1b..e380443 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -67,9 +67,9 @@ energy: 1.0e9 ### 2. The Control Modes -Add the control modes as lists in `simulators` and `controls`. Their `name` is also the name used to access them (`accelerator.design`, `accelerator.live`). +Add the control modes as lists in `simulators` and `controls`. Their `name` is also the name used to access them, for example `accelerator.design`, `accelerator.live` etc. -A simulator needs the path to a lattice file. Use `${path:...}` so that the path is resolved relative to the [configuration root](../../explanation/configuration.md#configuration-root) and the lattice is not loaded as a configuration file: +A simulator needs the path to a lattice file. All [formats that can be loaded by pyAT](https://atcollab.github.io/at/p/api/at.load.html#module-at.load) works. If you use the JSON format, you need to use the `${path:...}` [resolver](../../explanation/configuration.md#resolvers) to avoid the lattice being loaded as if it was a configuration file. ```yaml simulators: @@ -78,31 +78,18 @@ simulators: lattice: ${path:lattice.json} ``` -A control system is given by the class of the bindings you use. Its arguments depend on the bindings, for example for `pyaml-cs-oa`: +A control system is given by the class of the bindings you use. The arguments depend on the bindings and catalog type you decide to use. The catalog describes how the keys used by the devices map to control-system signals. See [Control System Catalogs](../../explanation/catalog.md) for the different types of catalogs. + +For example for `pyaml-cs-oa` using a dynamic catalog for TANGO: ```yaml controls: - class: pyaml_cs_oa.controlsystem.OphydAsyncControlSystem name: live backend: tango - catalog: catalog.yaml -``` - -The catalog describes how the keys used by the devices map to control-system signals. It follows exactly the same rule. A static catalog for `pyaml-cs-oa` looks like this (one entry per key): - -```yaml -class: pyaml_cs_oa.static_catalog.StaticCatalog -entries: - - class: pyaml_cs_oa.static_catalog_entry.StaticCatalogEntry - key: AN01-AR/EM-QP/QF.01/magnetic_strength - device: - class: pyaml_cs_oa.tangoAtt.TangoAtt - attribute: AN01-AR/EM-QP/QF.01/magnetic_strength - unit: 1/m - # ... one entry for each key used in the configuration ``` -See [Control System Catalogs](../../explanation/catalog.md) for the different types of catalogs. If you only want to use the simulator, you can leave out `controls` entirely. +If you only want to use the simulator, you can leave out `controls` entirely. ### 3. The Devices From b6ee30aa2ef8a9360db7b80bbb9b59c846e934de Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 25 Sep 2026 08:26:03 +0200 Subject: [PATCH 33/57] Update full configuration to match other changes. --- .../how-to/configuration/create-configuration.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index e380443..98f5345 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -93,7 +93,9 @@ If you only want to use the simulator, you can leave out `controls` entirely. ### 3. The Devices -Add the elements of the machine in `devices`. For a magnet, the `model` argument is an object (the magnet model, which also handles the unit conversion), so it is written as a nested item: +Add the elements of the machine in `devices`. For a magnet, the `model` argument is an object (the magnet model, which also handles the unit conversion), so it is written as a nested item. + +The strings given to `physics`, `x_pos` and `y_pos` are keys looked up in the catalog of the control system. ```yaml devices: @@ -115,7 +117,7 @@ devices: y_pos: AN01-AR/DG-EPOS/BPM.01/y ``` -By default, the `name` of an element is also the name of the element in the lattice of the simulator. Use `lattice_names` if they differ. The strings given to `physics`, `x_pos` and `y_pos` are keys looked up in the catalog of the control system. +By default, the `name` of an element is also the name of the element in the lattice of the simulator. If you want to use a different name in pyAML, use `lattice_names` to map between pyAML and the lattice. ### 4. The Arrays @@ -140,8 +142,8 @@ Putting it all together: ```yaml class: pyaml.accelerator.Accelerator -facility: My facility -machine: sr +facility: my_facility +machine: storage_ring energy: 1.0e9 simulators: - class: pyaml.lattice.simulator.Simulator From 7c9436f25284dc65a370056f4f31aa2ac78294c5 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 25 Sep 2026 08:27:29 +0200 Subject: [PATCH 34/57] Update catalog in example to match previous changes. --- docs/source/how-to/configuration/create-configuration.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index 98f5345..19b5def 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -153,7 +153,6 @@ controls: - class: pyaml_cs_oa.controlsystem.OphydAsyncControlSystem name: live backend: tango - catalog: catalog.yaml devices: - class: pyaml.magnet.quadrupole.Quadrupole name: QF_001 @@ -185,7 +184,7 @@ arrays: ## Split the Configuration into Several Files -For a real machine the configuration becomes long. When the configuration is loaded from a file, any string value ending with `.yaml`, `.yml` or `.json` is replaced by the content of that file (this is how `catalog: catalog.yaml` above is loaded). Inside a list, if the file contains a list, its items are added to the parent list. For example, move the quadrupoles to `devices/quadrupoles.yaml`: +For a real machine the configuration becomes long. When the configuration is loaded from a file, any string value ending with `.yaml`, `.yml` or `.json` is replaced by the content of that file. Inside a list, if the file contains a list, its items are added to the parent list. For example, move the quadrupoles to `devices/quadrupoles.yaml`: ```yaml # devices/quadrupoles.yaml From 567e3414c4264888fc057f9a3f1d16b530ad37a8 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 25 Sep 2026 08:29:43 +0200 Subject: [PATCH 35/57] Update text about using own classes. --- docs/source/how-to/configuration/create-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index 19b5def..95c2297 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -214,7 +214,7 @@ Values can also come from environment variables with `${env:NAME}`. See [Resolve ## Use Your Own Classes -The rule is not limited to pyAML classes. Any class that can be imported can be used in the configuration, for example a magnet model specific to your facility: +The configuration is not limited to pyAML classes. Any class that can be imported can be used in the configuration, for example a magnet model specific to your facility: ```python # my_facility/models.py From 0e1f53f0b8a9bc909fe79a8841a9750615168b75 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 25 Sep 2026 08:34:25 +0200 Subject: [PATCH 36/57] Update section about loading the config. --- docs/source/how-to/configuration/create-configuration.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index 95c2297..4932121 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -234,7 +234,7 @@ model: ## Load the Configuration -Set the configuration root, which is the directory used to resolve relative paths, then load the file with `Accelerator.load()`: +Set the [configuration root](../../explanation/configuration.md#configuration-root), which is the directory used to resolve relative paths, then load the file with `Accelerator.load()`. ```python from pyaml.configuration import ROOT @@ -246,9 +246,9 @@ accelerator = Accelerator.load("accelerator.yaml") accelerator.design.magnets.get("Quadrupoles").strengths.get() ``` -If the control-system bindings are not installed or you only want to use the simulator, add `ignore_external=True`. The `controls` section is then skipped. +If the control-system bindings are not installed or you only want to use the simulator, add `ignore_external=True` and the `controls` section is skipped without having to remove it from the configuration. -The configuration can also be given as a nested dictionary with `Accelerator.from_dict()`, following the same rule: +The configuration can also be loaded as a nested dictionary with `Accelerator.from_dict()`. This also allows to write the configuration directly as a dictionary instead of a text file if you prefer. ```python import yaml From e87fa30b3fa5922f5ab5a8022f4b9428cf733837 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 25 Sep 2026 08:42:20 +0200 Subject: [PATCH 37/57] Update validation section since information in there was wrong. --- docs/source/how-to/configuration/create-configuration.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index 4932121..e6bd8c0 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -263,7 +263,9 @@ See the API documentation for the [Accelerator](https://pyaml.readthedocs.io/en/ ## Validate the Configuration -Each item is checked when it is created: a missing required field or an unknown field raises a `PyAMLConfigException` naming the class and the field. The whole configuration can also be validated before anything is created, which gives all errors at once: +If the classes you use have enabled validation during object creation (this is the default for all common pyAML classes), the configuration will be validated as part of creating the objects: a missing required field or an unknown field raises a `PyAMLConfigException` naming the class and the field. + +The whole configuration can also be validated before anything is created. ```python from pyaml.validation import SchemaRegistry From a49e578cb70538406765268e231512693f64ba63 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 25 Sep 2026 09:05:47 +0200 Subject: [PATCH 38/57] Update section about tools. --- .../configuration/create-configuration.md | 22 ++++++++++++++----- 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index e6bd8c0..7427139 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -7,6 +7,10 @@ Read [Configuration Structure and Syntax](../../explanation/configuration) which ``` The configuration can be written as a text file in YAML or JSON or as a dictionary. +```{note} +Tools are available to help writing the configuration. See [Tools That Help Writing the Configuration](../configuration/create-configuration.md#tools-that-help-writing-the-configuration) for the options. +``` + ## The Rule to Remember The configuration consists of a set of items which tells pyAML which objects to build when loading the configuration. @@ -44,7 +48,7 @@ For example: ``` - Use the schema registry. The `describe()` method lists the fields of a registered class with their types. See [Use the Schema Registry](../how-to/configuration/use-schema-registry.ipynb). -- Use a JSON Schema in an external tool. See [JSON Schema Tools](../how-to/configuration/create-configuration.md#tools-that-help-writing-the-configuration). +- Use a JSON Schema in an external tool. See [Tools That Help Writing the Configuration](../configuration/create-configuration.md#tools-that-help-writing-the-configuration) for the options. Parameters with a default value is optional and can be left out of the configuration if you wish. @@ -278,12 +282,18 @@ The configuration can also be validated without loading it, which is useful if y ## Tools That Help Writing the Configuration -Writing the file by hand is often the simplest way to start, but tools based on a [JSON Schema](../../explanation/schema_and_validation.md) can suggest the available fields and check their types while you write: +There are tools available to help to write and modify the configuration. They can suggest the fields and check their types while you write. The tools are under development and testing so new or other tools might be available in the future based on user feedback. + +Some of the tools are based on a [JSON Schema](https://json-schema.org). For information about JSON Schemas and how to generate them, see [Configuration Schemas and Validation](../../explanation/schema_and_validation.md) and [Generate JSON Schemas](./generate-json-schema.ipynb). + +Currently these tools are available: + +- [Use ConfigurationSchema](./use-configuration-schema.ipynb) objects to create the configuration in Python and export it as a dictionary or text file. This allows to program the configuration. - [Use a JSON Schema in VS Code](./use-vscode-json-schema.md) -- [Use the MetaConfigurator](./use-meta-configurator.md), a form-based editor in the browser -- [Use ConfigurationSchema](./use-configuration-schema.ipynb) objects to create the configuration in Python and export it as a dictionary or text file -AI coding assistants can also help: supply for example a lattice file, a description of the naming conventions of your control system, and the JSON Schema of the pyAML configuration. +- [Use the MetaConfigurator](./use-meta-configurator.md), a form-based editor in the browser -For information about JSON Schemas and how to generate them, see [Configuration Schemas and Validation](../../explanation/schema_and_validation.md) and [Generate JSON Schemas](./generate-json-schema.ipynb). +```{tip} +AI coding assistants can also help: supply for example a lattice file, a description of the naming conventions of your control system, and the JSON Schema of the pyAML configuration and ask it to write the configuration for you. +``` \ No newline at end of file From 87ea33233742c388331dab83bde92dd0b2f925f0 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 2 Oct 2026 13:29:32 +0200 Subject: [PATCH 39/57] Make more generic so it doesn't sound like live and design are all possible control modes. --- docs/tutorials/functionality/00_introduction.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/tutorials/functionality/00_introduction.py b/docs/tutorials/functionality/00_introduction.py index 69e659c..23331bb 100644 --- a/docs/tutorials/functionality/00_introduction.py +++ b/docs/tutorials/functionality/00_introduction.py @@ -57,8 +57,8 @@ # # - **Accelerator**: the top-level object describing one machine. It is created from a # configuration. -# - **Control mode**: one way of accessing the accelerator. ``accelerator.live`` talks to the -# control system (the real machine or a virtual accelerator), ``accelerator.design`` talks +# - **Control mode**: one way of accessing the accelerator. In the figure above ``accelerator.live`` talks to the +# a control system and ``accelerator.design`` talks # to a simulation of the machine made with `pyAT `_. # Both provide exactly the same interface. # - **Element**: one object of the machine, such as a magnet, a BPM or the RF plant. From 0d96f5ab1d1a1e6c4c8084adff89b39c550315ce Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 2 Oct 2026 13:30:17 +0200 Subject: [PATCH 40/57] Change RF plant to RF cavity since is better understood by the users. --- docs/tutorials/functionality/00_introduction.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/functionality/00_introduction.py b/docs/tutorials/functionality/00_introduction.py index 23331bb..00d2296 100644 --- a/docs/tutorials/functionality/00_introduction.py +++ b/docs/tutorials/functionality/00_introduction.py @@ -61,7 +61,7 @@ # a control system and ``accelerator.design`` talks # to a simulation of the machine made with `pyAT `_. # Both provide exactly the same interface. -# - **Element**: one object of the machine, such as a magnet, a BPM or the RF plant. +# - **Element**: one object of the machine, such as a magnet, a BPM or a RF cavity. # - **Array**: a named group of elements which can be read or set in one call, such as all # the BPMs. # - **Attribute**: a value of an element that can be read with ``get()`` and written From 027365d58caa171934bcd0ddd531f8a48929badf Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 2 Oct 2026 13:32:20 +0200 Subject: [PATCH 41/57] Make the explanation of the configuration more general since doesn't have to be a text file. --- docs/tutorials/functionality/00_introduction.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/functionality/00_introduction.py b/docs/tutorials/functionality/00_introduction.py index 00d2296..cb45aa9 100644 --- a/docs/tutorials/functionality/00_introduction.py +++ b/docs/tutorials/functionality/00_introduction.py @@ -66,7 +66,7 @@ # the BPMs. # - **Attribute**: a value of an element that can be read with ``get()`` and written # with ``set()``, such as the ``strength`` of a magnet. -# - **Configuration**: a YAML or JSON file describing all of the above. Each item of the +# - **Configuration**: a structure (most commonly a YAML or JSON file) describing all of the above. Each item of the # configuration names a Python class with its ``class`` field, and **each other field is an # argument of that class's constructor**. This rule is explored in the first tutorial. # From 25e9e832a9b62a1adadaa5536695a65264fafdff Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 2 Oct 2026 13:34:46 +0200 Subject: [PATCH 42/57] Fix the link to the glossary since has been moved. --- docs/tutorials/functionality/00_introduction.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/functionality/00_introduction.py b/docs/tutorials/functionality/00_introduction.py index cb45aa9..daf2071 100644 --- a/docs/tutorials/functionality/00_introduction.py +++ b/docs/tutorials/functionality/00_introduction.py @@ -72,7 +72,7 @@ # # See :doc:`pyAML Structure <../../explanation/architecture>` and # :doc:`Control Modes <../../explanation/control-modes>` for more details, and the -# :doc:`Glossary <../../explanation/glossary>` for the definitions of the terms used in pyAML. +# :doc:`Glossary <../../glossary>` for the definitions of the terms used in pyAML. # %% # How the Tutorials Work From 2c688f817b47a76ddc8cda1faec1d90c57afce83 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 2 Oct 2026 13:40:28 +0200 Subject: [PATCH 43/57] Modify how the tutorials work for better reading flow. --- docs/tutorials/functionality/00_introduction.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/tutorials/functionality/00_introduction.py b/docs/tutorials/functionality/00_introduction.py index daf2071..e5f00a4 100644 --- a/docs/tutorials/functionality/00_introduction.py +++ b/docs/tutorials/functionality/00_introduction.py @@ -78,10 +78,10 @@ # How the Tutorials Work # ---------------------- # -# **Running the tutorials.** Each tutorial can be run in the cloud with -# `Binder `_ using the launcher on its page, with nothing to install, -# or downloaded and run on your own computer, as a Jupyter notebook or a Python script -# (see below). +# **Running the tutorials.** Each tutorial can be run in the cloud without requiring +# to install or download anything by using the `Binder `_ launcher +# on the tutorial page or as a Jupyter notebook or a Python script on your own computer +# (see below for instructions). # # **The test machine.** The tutorials use a small test storage ring, with its lattice and # ready-made pyAML configurations, provided by the ``pyaml-test-lattice`` package. It is From 37c7e0f269dcb8733669ed55ea12069533314391 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 2 Oct 2026 14:01:29 +0200 Subject: [PATCH 44/57] Update the instructions for how run locally so not need to be updated every time a new requirement is added. --- .../functionality/00_introduction.py | 32 +++++++------------ 1 file changed, 11 insertions(+), 21 deletions(-) diff --git a/docs/tutorials/functionality/00_introduction.py b/docs/tutorials/functionality/00_introduction.py index e5f00a4..c23dea3 100644 --- a/docs/tutorials/functionality/00_introduction.py +++ b/docs/tutorials/functionality/00_introduction.py @@ -103,33 +103,23 @@ # # You need Python 3.11 or newer. Always install in a virtual environment, to avoid breaking # your Python installation (see :doc:`New to Python <../../how-to/getting-started/python-basics>` -# if you are not familiar with virtual environments). Then install: +# if you are not familiar with virtual environments). Then do: +# +# 1. Install the requirements for the tutorials # -# .. code-block:: bash +# .. code-block:: bash # -# pip install "accelerator-middle-layer[cs-oa-tango]" pyaml-test-lattice jupyterlab +# pip install -r https://raw.githubusercontent.com/python-accelerator-middle-layer/documentation/main/binder/requirements.txt # -# This installs: +# 2. [Optional] If you want to run as notebooks, install JupyterLab # -# - ``accelerator-middle-layer``: the ``pyaml`` core package. It also installs pyAT, used by -# the ``design`` control mode, together with numpy and matplotlib. -# - ``[cs-oa-tango]``: the ``pyaml-cs-oa`` control-system bindings for TANGO. The ready-made -# configurations of the test machine declare a ``live`` control mode using these bindings, -# so they are needed to load them, even if you only use the ``design`` control mode. -# - ``pyaml-test-lattice``: the test machine, with its lattice and pyAML configurations. -# - ``jupyterlab``: to open the tutorials as notebooks. It is not needed to run them as -# Python scripts. +# .. code-block:: bash # -# The other control-system bindings (``cs-oa-epics``, ``tango-pyaml``) are only needed to -# connect to your own control system, see -# :doc:`User Installation <../../how-to/installation/user-installation>`. To get exactly the -# same environment as on Binder, install the -# `Binder requirements `_ -# instead: ``pip install -r binder/requirements.txt`` from a clone of the documentation -# repository. +# pip install jupyterlab # -# Finally, download a tutorial as a notebook or a Python script with the download links in -# the right sidebar of its page, and run it with ``jupyter lab`` or ``python``. +# Finally, download a tutorial as a notebook or a Python script using the download link in +# the right sidebar of the tutorial's page, and run it with ``jupyter lab`` or ``python``. +# You can also download all tutorials in one go on the :doc:`Tutorials <../index>` main page. # %% # Where to Go Next From 73f2b6650d489110ba623ba173d4abb799cfb82f Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 2 Oct 2026 14:09:41 +0200 Subject: [PATCH 45/57] Change where to go next to not have to modify every time a new tutorial is added. --- .../functionality/00_introduction.py | 32 +++++++++---------- 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/docs/tutorials/functionality/00_introduction.py b/docs/tutorials/functionality/00_introduction.py index c23dea3..8615b93 100644 --- a/docs/tutorials/functionality/00_introduction.py +++ b/docs/tutorials/functionality/00_introduction.py @@ -124,23 +124,21 @@ # %% # Where to Go Next # ---------------- +# +# There are two type of tutorials: +# +# 1. **Functionality**: These cover the functionality of pyAML. +# +# 2. **Use cases**: These apply pyAML to real tasks. +# +# The tutorials are designed to first follow the functionality and then the use cases +# but they can also be run independently depending on your interests. +# +# For more background, read the explanations: # -# The tutorials are designed to be followed in this order: -# -# 1. :doc:`Create an Accelerator <01_create_accelerator>`: create an accelerator -# interactively and from a configuration file, and learn how the configuration maps to -# Python classes. -# 2. :doc:`Inspect an Accelerator <02_inspect_accelerator>`: explore what a complete -# accelerator configuration contains. -# 3. The **use cases**, which apply pyAML to real tasks: -# :doc:`tune correction <../use_cases/tune-correction>`, -# :doc:`orbit correction <../use_cases/orbit_correction>` and -# :doc:`chromaticity measurement <../use_cases/chromaticity-measurement>`. -# -# For more background, read the explanations: -# :doc:`What is pyAML? <../../explanation/about>`, -# :doc:`pyAML Structure <../../explanation/architecture>`, -# :doc:`Control Modes <../../explanation/control-modes>` and -# :doc:`Configuration Structure and Syntax <../../explanation/configuration>`. +# - :doc:`What is pyAML? <../../explanation/about>` +# - :doc:`pyAML Structure <../../explanation/architecture>` +# - :doc:`Control Modes <../../explanation/control-modes>` +# - :doc:`Configuration Structure and Syntax <../../explanation/configuration>` # sphinx_gallery_thumbnail_path = '_static/pyaml-hierarchy.svg' From 095c635750557f7b036cf38639e49a8e01d0e55b Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 2 Oct 2026 19:10:47 +0200 Subject: [PATCH 46/57] Moved test lattice explanation to separate page and reformatted content for the configuration explanation. --- docs/source/explanation/index.md | 1 + docs/source/explanation/test_lattice.md | 15 +++++ .../functionality/01_create_accelerator.py | 57 ++++++------------- 3 files changed, 34 insertions(+), 39 deletions(-) create mode 100644 docs/source/explanation/test_lattice.md diff --git a/docs/source/explanation/index.md b/docs/source/explanation/index.md index a59fd96..d640dec 100644 --- a/docs/source/explanation/index.md +++ b/docs/source/explanation/index.md @@ -11,4 +11,5 @@ control-modes configuration schema_and_validation catalog +test_lattice ``` diff --git a/docs/source/explanation/test_lattice.md b/docs/source/explanation/test_lattice.md new file mode 100644 index 0000000..0e408bc --- /dev/null +++ b/docs/source/explanation/test_lattice.md @@ -0,0 +1,15 @@ +# Test Lattice + +The test lattice, `fodo_1gev_6d`, is a small 1 GeV electron storage ring made of +**16 identical FODO cells** of 4.8 m, for a circumference of 76.8 m. Each cell contains a +focusing quadrupole `QF` and sextupole `SF`, a beam position monitor `BPM`, a corrector +`COR` acting in both planes, a dipole `B`, a defocusing quadrupole `QD` and sextupole +`SD`, and a second dipole `B`: + +```{figure} /_static/fodo-cell.svg +:alt: Layout of one FODO cell of the test lattice +:width: 100% + +Each element is named after its family and a three-digit index equal to the cell number: +`QF_001` is the focusing quadrupole of the first cell. This is the magnet used in +this tutorial. \ No newline at end of file diff --git a/docs/tutorials/functionality/01_create_accelerator.py b/docs/tutorials/functionality/01_create_accelerator.py index a7da77a..b2ab6a2 100644 --- a/docs/tutorials/functionality/01_create_accelerator.py +++ b/docs/tutorials/functionality/01_create_accelerator.py @@ -40,26 +40,7 @@ # `pyAT `_. # # The example uses the lattice provided by the ``pyaml-test-lattice`` package. -# -# The Test Lattice -# ~~~~~~~~~~~~~~~~ -# -# The test lattice, ``fodo_1gev_6d``, is a small 1 GeV electron storage ring made of -# **16 identical FODO cells** of 4.8 m, for a circumference of 76.8 m. Each cell contains a -# focusing quadrupole ``QF`` and sextupole ``SF``, a beam position monitor ``BPM``, a corrector -# ``COR`` acting in both planes, a dipole ``B``, a defocusing quadrupole ``QD`` and sextupole -# ``SD``, and a second dipole ``B``: -# -# .. figure:: /_static/fodo-cell.svg -# :alt: Layout of one FODO cell of the test lattice -# :width: 100% -# -# One cell of the test lattice with the control-system name of each element -# (``cc`` is the cell number, from 01 to 16). -# -# Each element is named after its family and a three-digit index equal to the cell number: -# ``QF_001`` is the focusing quadrupole of the first cell. This is the magnet used in -# this tutorial. +# Go to `Test lattice <../../explanation/test_lattice.html>`_ for details about the lattice. # Get the path to the lattice file # sphinx_gallery_thumbnail_path = '_static/create_accelerator.png' @@ -150,22 +131,23 @@ # Configuration files can be written in YAML or JSON. This example shows a YAML file. # %% -# The Configuration Rule -# ~~~~~~~~~~~~~~~~~~~~~~ -# A configuration file describes the same objects as the ones created in approach 1, -# following one simple rule: +# .. admonition:: The Configuration Rule +# +# A configuration file describes the same objects as the ones created in approach 1, +# following one simple rule: # -# - the ``class`` field gives the full path of the Python class to create, -# - **every other field is an argument of the constructor of that class**, with the same name, -# - when an argument is itself an object, its value is a nested item with its own ``class``. +# - The ``class`` field gives the full path of the Python class to create, +# - **Every other field is an argument of the constructor of that class** with the same +# name as the field and the value to pass to the constructor, +# - When an argument is itself an object, its value is a nested item with its own ``class`` field. # -# For example, in approach 1 the quadrupole was created with: +# In approach 1 the quadrupole was created with: # # .. code-block:: python # # Quadrupole(name="QF_001", model=IdentityMagnetModel(physics="")) # -# which becomes in the configuration file: +# When writing a configuration file instead it becomes: # # .. code-block:: yaml # @@ -175,8 +157,8 @@ # class: pyaml.magnet.identity_model.IdentityMagnetModel # physics: '' # -# The accepted fields of any class are therefore given by the arguments of its constructor, -# which you can see with ``help()``. The first lines show the constructor signature, and the +# Since the accepted fields of a class are given by the arguments of its constructor, you +# can see them using ``help()``. The first lines show the constructor signature, and the # ``Parameters`` section describes each argument: help(Quadrupole) @@ -184,15 +166,12 @@ # %% # Write the Configuration File # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -# A configuration file is a plain text file. You can write it with any text editor. Other -# tools which can help you are described in the how-to guide +# You can write a configuration file with any text editor. +# More details, advice and tools that can help are described in the how-to guide # :doc:`Create and Load Configuration <../../how-to/configuration/create-configuration>`. # -# The file below describes the same accelerator as in approach 1. Compare each item with -# the Python code above: ``Accelerator(facility=..., machine=..., energy=..., simulators=[...], -# devices=[...])``, ``Simulator(name=..., lattice=...)`` and ``Quadrupole(name=..., model=...)``. -# -# The lattice path is given by an environment variable, using the ``${env:NAME}`` syntax. +# The file below describes the same accelerator as in approach 1. The main difference is +# that the lattice path is given by an environment variable, using the ``${env:NAME}`` syntax. # It could also be written directly as an absolute path, or relative to a root directory. configuration = """\ @@ -216,7 +195,7 @@ file.write(configuration) # %% -# Specify the Paths +# Specify the Path # ~~~~~~~~~~~~~~~~~ # The path to the configuration file can be specified as absolute or relative to a root directory. From 7d392da55a3c358cd73d313e35f4d5e8b7ba6d80 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 2 Oct 2026 19:17:32 +0200 Subject: [PATCH 47/57] Make yellow pages bold. --- docs/tutorials/functionality/02_inspect_accelerator.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/functionality/02_inspect_accelerator.py b/docs/tutorials/functionality/02_inspect_accelerator.py index 2dc3831..ed4f06e 100644 --- a/docs/tutorials/functionality/02_inspect_accelerator.py +++ b/docs/tutorials/functionality/02_inspect_accelerator.py @@ -19,7 +19,7 @@ Inspect an Accelerator ========================================================== -This tutorial shows how to find out what an accelerator contains using the *yellow pages*, +This tutorial shows how to find out what an accelerator contains using the **yellow pages**, and how to access what you found. As a reminder, an accelerator contains one or several control modes (for example ``live`` From e962ebfcdcbf765ccceb5d296f7f84a9506d32c5 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 2 Oct 2026 19:25:21 +0200 Subject: [PATCH 48/57] Add how to try for an EPICS configuration. --- docs/tutorials/functionality/02_inspect_accelerator.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/tutorials/functionality/02_inspect_accelerator.py b/docs/tutorials/functionality/02_inspect_accelerator.py index ed4f06e..348fde0 100644 --- a/docs/tutorials/functionality/02_inspect_accelerator.py +++ b/docs/tutorials/functionality/02_inspect_accelerator.py @@ -43,6 +43,10 @@ # List available files and their descriptions configurations +# %% +# This tutorials uses a pyAML configuration for TANGO but you can switch to the configuration +# for EPICS if you want to try with that instead. + # %% # Load the Accelerator # -------------------- From 9109943955617b7a22a431e93d573b0a1dabe3a9 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 2 Oct 2026 20:14:36 +0200 Subject: [PATCH 49/57] Change to the interface is exactly the same. --- docs/source/explanation/control-modes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/explanation/control-modes.md b/docs/source/explanation/control-modes.md index 823b862..b46b417 100644 --- a/docs/source/explanation/control-modes.md +++ b/docs/source/explanation/control-modes.md @@ -1,6 +1,6 @@ # Control Modes -A control mode is one way of accessing the accelerator. pyAML is built so that the core interactions with the machine work in exactly the same way in every control mode. The following requirements guide the design: +A control mode is one way of accessing the accelerator. PyAML is built so the interface is exactly the same in every control mode. The following requirements guide the design: - Core interactions work identically in all control modes. - All configured control modes are available at all times, and can be used at the same time in one script. From 254777919fe9aaa188963a654e529a3b6ecac9a1 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 2 Oct 2026 20:29:26 +0200 Subject: [PATCH 50/57] Make it clearer that the names of the modes can be freely chosen and improve readability. --- docs/source/explanation/control-modes.md | 48 ++++++++++++++---------- 1 file changed, 28 insertions(+), 20 deletions(-) diff --git a/docs/source/explanation/control-modes.md b/docs/source/explanation/control-modes.md index b46b417..1e1bc77 100644 --- a/docs/source/explanation/control-modes.md +++ b/docs/source/explanation/control-modes.md @@ -9,20 +9,26 @@ A control mode is one way of accessing the accelerator. PyAML is built so the in ## Available Control Modes -Two kinds of control modes are implemented today. +Two kinds of control modes are implemented today. Each mode is given a `name` which is used to access the mode. The name can be freely chosen but some standardized names are defined which should not be used for other modes. -**Live** (`ControlSystem`) -: Access to the accelerator through its control system. The values are read from and written to the control system through control-system bindings such as `tango-pyaml` or `pyaml-cs-oa`. The control system can be the real machine or a *virtual accelerator*, a simulation exposed through the same control-system interface as the real machine. By convention this mode is named `live`. +**ControlSystem** +: Access to the accelerator through a control system. The values are read from and written to the control system through control-system bindings such as `tango-pyaml` or `pyaml-cs-oa`. The control system can be the real machine or a *virtual accelerator*, i.e. a simulation exposed through the same control-system interface as the real machine. -**Design** (`Simulator`) -: Access to a simulation of the accelerator. Values are read from and written to a [pyAT](https://atcollab.github.io/at/p/index.html) lattice. Diagnostics such as BPMs and tune monitors return the values computed by the simulator. No control system is needed. By convention this mode is named `design`. + Current standardized names: + - `live` -The modes are declared in the configuration: control systems in `controls` and simulators in `simulators`. Each of them has a `name`, which is also the name of the attribute used to access it: +**Simulator** +: Access to a simulation of the accelerator. Values are read from and written to a [pyAT](https://atcollab.github.io/at/p/index.html) lattice. Diagnostics such as BPMs and tune monitors return the values computed by the simulator. No control system is needed. + + Current standardized names: + - `design` + +The modes are declared in the configuration: control systems in `controls` and simulators in `simulators`. For example: ```yaml class: pyaml.accelerator.Accelerator -facility: My facility -machine: sr +facility: my_facility +machine: accelerator energy: 1.0e9 simulators: - class: pyaml.lattice.simulator.Simulator @@ -36,29 +42,31 @@ devices: # ... magnets, BPMs, tuning tools ``` +The modes are accessed using: + ```python -accelerator.design # the Simulator -accelerator.live # the ControlSystem +accelerator.design # The simulator +accelerator.live # The controlSystem ``` -Several simulators or control systems can be defined, as long as they have different names. For example a second simulator loaded with a lattice including errors could be named `errors` and reached with `accelerator.errors`. +Several simulators or control systems can be defined, as long as they have different names. For example a second simulator loaded with a lattice including errors could be named `errors` and accessed with `accelerator.errors`. ## Using Control Modes -Because every mode exposes the same elements, arrays and tools, a script can be written once and run in any mode: +Since every mode exposes the same elements, arrays and tools, a script can be written once and run in any mode: ```python def correct_tune(sr): - sr.tune.set([0.2, 0.3]) + accelerator.tune.set([0.2, 0.3]) -correct_tune(accelerator.design) # try it on the simulator first -correct_tune(accelerator.live) # then run it on the machine +correct_tune(accelerator.design) # Try it on the simulator first +correct_tune(accelerator.live) # Then run it on the machine ``` A common pattern is to select the mode once at the top of a script: ```python -SR = accelerator.design # switch to accelerator.live to act on the machine +SR = accelerator.design # Change to accelerator.live to act on the machine ``` Different modes can also be used side by side, for example to compare the measured orbit with the simulated one: @@ -82,14 +90,14 @@ accelerator.design.magnets.get("HCorr").strengths.set(hcorr) ```{admonition} Planned, not implemented yet :class: note -The following control modes are described in the pyAML specification. They are part of the long-term plan of the collaboration and are **not available yet**. +The following control modes are described in the pyAML specification. They are part of the long-term plan and are **not available yet**. ``` **Errors / commissioning simulations** -: Simulators with lattices containing errors, and arrays of randomly generated error seeds, used to run simulated commissioning of a machine. +: Simulators with lattices containing errors used to run simulated commissioning of a machine. **Shadow (digital shadow)** -: A simulator that follows the real machine: settings read from the control system are applied to the model, which then computes the expected optics and diagnostics. Writing is forbidden in this mode. +: A simulator that follows the real machine: settings read from the control system are applied to the model which then computes the expected optics and diagnostics. Writing is forbidden in this mode. **Archive** -: A simulator loaded with the machine settings found in the archiving system at a given time, to reproduce and investigate a past situation. +: A simulator loaded with machine settings from an archiving system at a given timestamp to reproduce and investigate a past situation. From 206e68791ff0d4ca7ea66fea515cd9b6170a6fa8 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 2 Oct 2026 20:32:26 +0200 Subject: [PATCH 51/57] Change name in configuration example since was wrongly changed. --- docs/source/explanation/control-modes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/explanation/control-modes.md b/docs/source/explanation/control-modes.md index 1e1bc77..7ec0149 100644 --- a/docs/source/explanation/control-modes.md +++ b/docs/source/explanation/control-modes.md @@ -28,7 +28,7 @@ The modes are declared in the configuration: control systems in `controls` and s ```yaml class: pyaml.accelerator.Accelerator facility: my_facility -machine: accelerator +machine: storage_ring energy: 1.0e9 simulators: - class: pyaml.lattice.simulator.Simulator From 7c52937c02b1129e0246dfcfc502972fbe4d595a Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Fri, 2 Oct 2026 20:33:50 +0200 Subject: [PATCH 52/57] Add a sentence to explain that accelerator is the name of the accelerator object. --- docs/source/explanation/control-modes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/explanation/control-modes.md b/docs/source/explanation/control-modes.md index 7ec0149..942be2a 100644 --- a/docs/source/explanation/control-modes.md +++ b/docs/source/explanation/control-modes.md @@ -42,7 +42,7 @@ devices: # ... magnets, BPMs, tuning tools ``` -The modes are accessed using: +After loading the accelerator into an object called `accelerator`. The modes are accessed using: ```python accelerator.design # The simulator From 00ad2ebb04293b66c052e2c98740b9b0f800df24 Mon Sep 17 00:00:00 2001 From: Alexis Gamelin Date: Tue, 6 Oct 2026 17:43:47 +0200 Subject: [PATCH 53/57] Add back the test lattice text in the tutorial --- docs/source/explanation/index.md | 1 - docs/source/explanation/test_lattice.md | 15 ------------ .../functionality/01_create_accelerator.py | 23 +++++++++++++++++-- 3 files changed, 21 insertions(+), 18 deletions(-) delete mode 100644 docs/source/explanation/test_lattice.md diff --git a/docs/source/explanation/index.md b/docs/source/explanation/index.md index d640dec..a59fd96 100644 --- a/docs/source/explanation/index.md +++ b/docs/source/explanation/index.md @@ -11,5 +11,4 @@ control-modes configuration schema_and_validation catalog -test_lattice ``` diff --git a/docs/source/explanation/test_lattice.md b/docs/source/explanation/test_lattice.md deleted file mode 100644 index 0e408bc..0000000 --- a/docs/source/explanation/test_lattice.md +++ /dev/null @@ -1,15 +0,0 @@ -# Test Lattice - -The test lattice, `fodo_1gev_6d`, is a small 1 GeV electron storage ring made of -**16 identical FODO cells** of 4.8 m, for a circumference of 76.8 m. Each cell contains a -focusing quadrupole `QF` and sextupole `SF`, a beam position monitor `BPM`, a corrector -`COR` acting in both planes, a dipole `B`, a defocusing quadrupole `QD` and sextupole -`SD`, and a second dipole `B`: - -```{figure} /_static/fodo-cell.svg -:alt: Layout of one FODO cell of the test lattice -:width: 100% - -Each element is named after its family and a three-digit index equal to the cell number: -`QF_001` is the focusing quadrupole of the first cell. This is the magnet used in -this tutorial. \ No newline at end of file diff --git a/docs/tutorials/functionality/01_create_accelerator.py b/docs/tutorials/functionality/01_create_accelerator.py index b2ab6a2..d9858f3 100644 --- a/docs/tutorials/functionality/01_create_accelerator.py +++ b/docs/tutorials/functionality/01_create_accelerator.py @@ -40,7 +40,26 @@ # `pyAT `_. # # The example uses the lattice provided by the ``pyaml-test-lattice`` package. -# Go to `Test lattice <../../explanation/test_lattice.html>`_ for details about the lattice. +# +# The Test Lattice +# ~~~~~~~~~~~~~~~~ +# +# The test lattice, ``fodo_1gev_6d``, is a small 1 GeV electron storage ring made of +# **16 identical FODO cells** of 4.8 m, for a circumference of 76.8 m. Each cell contains a +# focusing quadrupole ``QF`` and sextupole ``SF``, a beam position monitor ``BPM``, a corrector +# ``COR`` acting in both planes, a dipole ``B``, a defocusing quadrupole ``QD`` and sextupole +# ``SD``, and a second dipole ``B``: +# +# .. figure:: /_static/fodo-cell.svg +# :alt: Layout of one FODO cell of the test lattice +# :width: 100% +# +# One cell of the test lattice with the control-system name of each element +# (``cc`` is the cell number, from 01 to 16). +# +# Each element is named after its family and a three-digit index equal to the cell number: +# ``QF_001`` is the focusing quadrupole of the first cell. This is the magnet used in +# this tutorial. # Get the path to the lattice file # sphinx_gallery_thumbnail_path = '_static/create_accelerator.png' @@ -196,7 +215,7 @@ # %% # Specify the Path -# ~~~~~~~~~~~~~~~~~ +# ~~~~~~~~~~~~~~~~ # The path to the configuration file can be specified as absolute or relative to a root directory. # Set the root directory From 15b65b7ba529b29e933b960d14dedbed6b558f0e Mon Sep 17 00:00:00 2001 From: Alexis Gamelin Date: Tue, 6 Oct 2026 17:38:51 +0200 Subject: [PATCH 54/57] Add a new tool pages --- .../configuration/create-configuration.md | 61 +++---------------- .../how-to/configuration/tools/index.md | 26 ++++++++ .../use-configuration-schema.ipynb | 2 +- .../{ => tools}/use-meta-configurator.md | 2 +- .../{ => tools}/use-vscode-json-schema.md | 0 .../validate-configuration.ipynb | 16 ++++- docs/source/how-to/index.md | 5 +- 7 files changed, 53 insertions(+), 59 deletions(-) create mode 100644 docs/source/how-to/configuration/tools/index.md rename docs/source/how-to/configuration/{ => tools}/use-configuration-schema.ipynb (99%) rename docs/source/how-to/configuration/{ => tools}/use-meta-configurator.md (98%) rename docs/source/how-to/configuration/{ => tools}/use-vscode-json-schema.md (100%) diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index 7427139..a287b0c 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -1,6 +1,6 @@ # Create and Load Configuration -This guide shows how to write a pyAML configuration and load it into an `Accelerator`. It gives recommendations for how to write it and list tools that are available to help. +This guide shows how to write a pyAML configuration and load it into an `Accelerator` using a minimal example. ```{tip} Read [Configuration Structure and Syntax](../../explanation/configuration) which explains the concepts and ideas behind the configuration before you start. @@ -8,7 +8,7 @@ Read [Configuration Structure and Syntax](../../explanation/configuration) which The configuration can be written as a text file in YAML or JSON or as a dictionary. ```{note} -Tools are available to help writing the configuration. See [Tools That Help Writing the Configuration](../configuration/create-configuration.md#tools-that-help-writing-the-configuration) for the options. +Tools are available to help writing the configuration. See [Tools That Help Writing the Configuration](./tools/index.md) for the options. ``` ## The Rule to Remember @@ -25,7 +25,7 @@ Writing and loading the configuration is the equivalent of writing the Python co Before writing an item, look up the constructor parameters of the class you want pyAML to build an object of. This can be done in several ways: -- Read the [API documentation](./../reference/index.md) of the class +- Read the [API documentation](../../reference/index.md) of the class - Use `help()` in Python since this shows the signature of the constructor and description of each parameter For example: @@ -47,16 +47,16 @@ For example: # | ... ``` -- Use the schema registry. The `describe()` method lists the fields of a registered class with their types. See [Use the Schema Registry](../how-to/configuration/use-schema-registry.ipynb). -- Use a JSON Schema in an external tool. See [Tools That Help Writing the Configuration](../configuration/create-configuration.md#tools-that-help-writing-the-configuration) for the options. +- Use the schema registry. The `describe()` method lists the fields of a registered class with their types. See [Use the Schema Registry](./use-schema-registry.ipynb). +- Use a JSON Schema in an external tool. See [Tools That Help Writing the Configuration](./tools/index.md) for the options. Parameters with a default value is optional and can be left out of the configuration if you wish. ## Write Configuration as a Text File -Here an example is shown for how to create the configuration in a YAML file. The steps are similar if using JSON. The steps below build a small but complete configuration, using the names of the [test lattice](../../tutorials/functionality/01_create_accelerator). +Here an example is shown for how to create the configuration in a YAML file. The steps are similar if using JSON. The steps below build a small but complete configuration. -Create a file, for example `accelerator.yaml`, with any text editor. If you want the editor to suggest the fields, you can use VS Code together with a JSON Schema. See [Use JSON Schema in VS Code](../configuration/use-vscode-json-schema.md) for instructions. +Create a file, for example `accelerator.yaml`, with any text editor. If you want the editor to suggest the fields, you can use VS Code together with a JSON Schema. See [Use JSON Schema in VS Code](./tools/use-vscode-json-schema.md) for instructions. ### 1. The Accelerator @@ -208,10 +208,7 @@ and refer to it from the main file: ```yaml devices: - devices/quadrupoles.yaml - - class: pyaml.bpm.bpm.BPM - name: BPM_001 - x_pos: AN01-AR/DG-EPOS/BPM.01/x - y_pos: AN01-AR/DG-EPOS/BPM.01/y + # ... ``` Values can also come from environment variables with `${env:NAME}`. See [Resolvers](../../explanation/configuration.md#resolvers) for all options. @@ -254,46 +251,6 @@ If the control-system bindings are not installed or you only want to use the sim The configuration can also be loaded as a nested dictionary with `Accelerator.from_dict()`. This also allows to write the configuration directly as a dictionary instead of a text file if you prefer. -```python -import yaml - -with open("accelerator.yaml") as file: - config = yaml.safe_load(file) - -accelerator = Accelerator.from_dict(config) -``` +To validate the whole configuration before any object is created, add `validate=True`. See [Validate Configuration](./validate-configuration) for details. See the API documentation for the [Accelerator](https://pyaml.readthedocs.io/en/stable/api/pyaml.accelerator.html#module-pyaml.accelerator) for all options. - -## Validate the Configuration - -If the classes you use have enabled validation during object creation (this is the default for all common pyAML classes), the configuration will be validated as part of creating the objects: a missing required field or an unknown field raises a `PyAMLConfigException` naming the class and the field. - -The whole configuration can also be validated before anything is created. - -```python -from pyaml.validation import SchemaRegistry - -SchemaRegistry().discover() -accelerator = Accelerator.load("accelerator.yaml", validate=True) -``` - -The configuration can also be validated without loading it, which is useful if you maintain it separately from pyAML. See [Validate Configuration](./validate-configuration). - -## Tools That Help Writing the Configuration - -There are tools available to help to write and modify the configuration. They can suggest the fields and check their types while you write. The tools are under development and testing so new or other tools might be available in the future based on user feedback. - -Some of the tools are based on a [JSON Schema](https://json-schema.org). For information about JSON Schemas and how to generate them, see [Configuration Schemas and Validation](../../explanation/schema_and_validation.md) and [Generate JSON Schemas](./generate-json-schema.ipynb). - -Currently these tools are available: - -- [Use ConfigurationSchema](./use-configuration-schema.ipynb) objects to create the configuration in Python and export it as a dictionary or text file. This allows to program the configuration. - -- [Use a JSON Schema in VS Code](./use-vscode-json-schema.md) - -- [Use the MetaConfigurator](./use-meta-configurator.md), a form-based editor in the browser - -```{tip} -AI coding assistants can also help: supply for example a lattice file, a description of the naming conventions of your control system, and the JSON Schema of the pyAML configuration and ask it to write the configuration for you. -``` \ No newline at end of file diff --git a/docs/source/how-to/configuration/tools/index.md b/docs/source/how-to/configuration/tools/index.md new file mode 100644 index 0000000..c01d850 --- /dev/null +++ b/docs/source/how-to/configuration/tools/index.md @@ -0,0 +1,26 @@ +# Tools That Help Writing the Configuration + +There are tools available to help to write and modify the configuration. They can suggest the fields and check their types while you write. The tools are under development and testing so new or other tools might be available in the future based on user feedback. + +Some of the tools are based on a [JSON Schema](https://json-schema.org). For information about JSON Schemas and how to generate them, see [Configuration Schemas and Validation](../../../explanation/schema_and_validation.md) and [Generate JSON Schemas](../generate-json-schema.ipynb). + +Currently these tools are available: + +- [Use ConfigurationSchema](./use-configuration-schema.ipynb) objects to create the configuration in Python and export it as a dictionary or text file. This allows to program the configuration. + +- [Use a JSON Schema in VS Code](./use-vscode-json-schema.md) + +- [Use the MetaConfigurator](./use-meta-configurator.md), a form-based editor in the browser + +```{tip} +AI coding assistants can also help: supply for example a lattice file, a description of the naming conventions of your control system, and the JSON Schema of the pyAML configuration and ask it to write the configuration for you. +``` + +```{toctree} +:maxdepth: 1 +:hidden: + +use-configuration-schema +use-vscode-json-schema +use-meta-configurator +``` diff --git a/docs/source/how-to/configuration/use-configuration-schema.ipynb b/docs/source/how-to/configuration/tools/use-configuration-schema.ipynb similarity index 99% rename from docs/source/how-to/configuration/use-configuration-schema.ipynb rename to docs/source/how-to/configuration/tools/use-configuration-schema.ipynb index a46f34c..695faf5 100644 --- a/docs/source/how-to/configuration/use-configuration-schema.ipynb +++ b/docs/source/how-to/configuration/tools/use-configuration-schema.ipynb @@ -7,7 +7,7 @@ "source": [ "# Generate Configuration Using Configuration Schemas\n", "\n", - "This guide shows how to use `ConfigurationSchema` to create a configuration. This allows to program the configuration if you prefer instead of using any of the [options based on a JSON Schema](.//create-configuration.md)." + "This guide shows how to use `ConfigurationSchema` to create a configuration. This allows to program the configuration if you prefer instead of using any of the [options based on a JSON Schema](./index.md)." ] }, { diff --git a/docs/source/how-to/configuration/use-meta-configurator.md b/docs/source/how-to/configuration/tools/use-meta-configurator.md similarity index 98% rename from docs/source/how-to/configuration/use-meta-configurator.md rename to docs/source/how-to/configuration/tools/use-meta-configurator.md index 49e8126..6db9eaa 100644 --- a/docs/source/how-to/configuration/use-meta-configurator.md +++ b/docs/source/how-to/configuration/tools/use-meta-configurator.md @@ -6,7 +6,7 @@ Pre-generated schemas are available in [pyaml-schemas](https://github.com/python-accelerator-middle-layer/pyaml-schemas). These include the classes that are part of the pyAML ecosystem. -If you have facility specific classes that you want to include in your configuration, you can use the `SchemaRegistry` to generate a JSON Schema including them. See [Generate JSON Schemas](../configuration/generate-json-schema) for details. +If you have facility specific classes that you want to include in your configuration, you can use the `SchemaRegistry` to generate a JSON Schema including them. See [Generate JSON Schemas](../generate-json-schema) for details. The schema must describe the document you want to create. For example, a schema for a quadrupole is suitable for editing a quadrupole object, but not for editing a complete accelerator containing controls, simulators, and devices. diff --git a/docs/source/how-to/configuration/use-vscode-json-schema.md b/docs/source/how-to/configuration/tools/use-vscode-json-schema.md similarity index 100% rename from docs/source/how-to/configuration/use-vscode-json-schema.md rename to docs/source/how-to/configuration/tools/use-vscode-json-schema.md diff --git a/docs/source/how-to/configuration/validate-configuration.ipynb b/docs/source/how-to/configuration/validate-configuration.ipynb index 06cacfd..7b3b944 100644 --- a/docs/source/how-to/configuration/validate-configuration.ipynb +++ b/docs/source/how-to/configuration/validate-configuration.ipynb @@ -7,7 +7,21 @@ "source": [ "# Validate Configuration\n", "\n", - "To make sure that the input data provided to pyAML has the correct format, validation is important. The validation is by default automatically performed when loading a configuration into the `Accelerator`, but it can also be done separately if one wishes. This guide shows how to do that.\n", + "To make sure that the input data provided to pyAML has the correct format, validation is important.\n", + "\n", + "If the classes you use have enabled validation during object creation (this is the default for all common pyAML classes), the configuration is validated as part of creating the objects: a missing required field or an unknown field raises a `PyAMLConfigException` naming the class and the field.\n", + "\n", + "The whole configuration can also be validated before anything is created when loading it:\n", + "\n", + "```python\n", + "from pyaml.accelerator import Accelerator\n", + "from pyaml.validation import SchemaRegistry\n", + "\n", + "SchemaRegistry().discover()\n", + "accelerator = Accelerator.load(\"accelerator.yaml\", validate=True)\n", + "```\n", + "\n", + "The configuration can also be validated without loading it, which is useful if you maintain it separately from pyAML. This guide shows how to do that.\n", "\n", "For more details about schemas and the different types of validation, see [Configuration Schemas and Validation](./../../explanation/schema_and_validation.md).\n", "\n", diff --git a/docs/source/how-to/index.md b/docs/source/how-to/index.md index 7a075ab..150b94b 100644 --- a/docs/source/how-to/index.md +++ b/docs/source/how-to/index.md @@ -25,10 +25,7 @@ configuration/create-configuration configuration/use-schema-registry configuration/validate-configuration configuration/generate-json-schema -configuration/use-configuration-schema -configuration/use-meta-configurator -configuration/use-vscode-json-schema - +configuration/tools/index ``` ```{toctree} From fcbc9d292d8fd6e2a6282265207bad2d139e43d8 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Wed, 7 Oct 2026 09:32:09 +0200 Subject: [PATCH 55/57] Fix broken link to glossary. --- docs/source/explanation/about.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/explanation/about.md b/docs/source/explanation/about.md index 1525a2d..ce16b21 100644 --- a/docs/source/explanation/about.md +++ b/docs/source/explanation/about.md @@ -60,4 +60,4 @@ A facility adopts pyAML by writing a *configuration* describing its machine. The There is one simple rule behind the configuration: each item names a Python class, and each of its other fields is an argument of that class's constructor. See [Configuration Structure and Syntax](configuration.md). -The terms used throughout the documentation are defined in the [Glossary](glossary.md). +The terms used throughout the documentation are defined in the [Glossary](../glossary.md). From 95c80ce07278718748eb7e57a86a0d285c630f97 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Wed, 7 Oct 2026 09:34:34 +0200 Subject: [PATCH 56/57] Fix broken link to control modes. --- docs/source/glossary.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/glossary.md b/docs/source/glossary.md index 4edc826..5f681bc 100644 --- a/docs/source/glossary.md +++ b/docs/source/glossary.md @@ -6,7 +6,7 @@ Accelerator : The top-level pyAML object describing one machine (a storage ring, a booster, a transfer line, ...). It holds all the control modes, arrays and devices. Control mode -: One way of accessing the accelerator, for example `live` (the real machine) or `design` (a simulation). All control modes offer the same interface. See [Control Modes](control-modes.md). +: One way of accessing the accelerator, for example `live` (the real machine) or `design` (a simulation). All control modes offer the same interface. See [Control Modes](./explanation/control-modes.md). Element : A single object of the accelerator that can be read or set: a magnet, a BPM, an RF plant, a tune monitor, ... From fcf8d3dd311bfb0b6b0b7f5e117a0425e4b579f0 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Wed, 7 Oct 2026 09:35:58 +0200 Subject: [PATCH 57/57] Fix broken link to catalogs. --- docs/source/glossary.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/glossary.md b/docs/source/glossary.md index 5f681bc..01a4d5b 100644 --- a/docs/source/glossary.md +++ b/docs/source/glossary.md @@ -21,7 +21,7 @@ Backend : The system that a control mode talks to: a control system (through control-system bindings such as `tango-pyaml` or `pyaml-cs-oa`) or a simulation code (`pyAT`). Catalog -: The backend-specific description of how a key used in the configuration maps to a signal of the control system. See [Control System Catalogs](catalog.md). +: The backend-specific description of how a key used in the configuration maps to a signal of the control system. See [Control System Catalogs](./explanation/catalog.md). Virtual accelerator : A simulated machine exposed through a real control system (for example TANGO devices backed by a simulation), so that it can be used exactly like the real machine.