diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index 8c3fb74..a287b0c 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -1,20 +1,34 @@ # Create and Load Configuration -This guide shows how to write a pyAML configuration as a text file and load it into an `Accelerator`. The structure and syntax of the configuration are explained in detail in [Configuration Structure and Syntax](../../explanation/configuration). +This guide shows how to write a pyAML configuration and load it into an `Accelerator` using a minimal example. + +```{tip} +Read [Configuration Structure and Syntax](../../explanation/configuration) which explains the concepts and ideas behind the configuration before you start. +``` +The configuration can be written as a text file in YAML or JSON or as a dictionary. + +```{note} +Tools are available to help writing the configuration. See [Tools That Help Writing the Configuration](./tools/index.md) for the options. +``` ## The Rule to Remember +The configuration consists of a set of items which tells pyAML which objects to build when loading the configuration. + ```{important} -Each item of the configuration names a Python class in its `class` field. **Every other field is an argument of the constructor of that class**, with the same name. When an argument is an object, its value is a nested item with its own `class` field. +Each item of the configuration names a Python class in its `class` field. **Every other field is an argument of the constructor of that class** with the same name as the field and the value to pass to the constructor. When a constructor argument is an object, its value in the configuration is a nested item with its own `class` field. ``` -Writing a configuration is therefore the same as writing the Python code that creates the objects, in YAML (or JSON) instead of Python. +Writing and loading the configuration is the equivalent of writing the Python code that creates the objects yourself. The configuration just allows pyAML to create the objects for you. -## Find the Fields of a Class +## Finding the Accepted Fields -Before writing an item, look up the constructor arguments of its class. Any of these works: +Before writing an item, look up the constructor parameters of the class you want pyAML to build an object of. This can be done in several ways: -- `help()` in Python, which shows the constructor signature and describes each argument: +- Read the [API documentation](../../reference/index.md) of the class +- Use `help()` in Python since this shows the signature of the constructor and description of each parameter + +For example: ```python from pyaml.bpm.bpm import BPM @@ -33,14 +47,16 @@ Before writing an item, look up the constructor arguments of its class. Any of t # | ... ``` -- the [API documentation](https://pyaml.readthedocs.io/en/stable/) of the class, -- the `describe()` method of the schema in the [schema registry](./use-schema-registry.ipynb). +- Use the schema registry. The `describe()` method lists the fields of a registered class with their types. See [Use the Schema Registry](./use-schema-registry.ipynb). +- Use a JSON Schema in an external tool. See [Tools That Help Writing the Configuration](./tools/index.md) for the options. + +Parameters with a default value is optional and can be left out of the configuration if you wish. -Arguments without a default value (here `name`) are required fields; the others can be left out. +## Write Configuration as a Text File -## Write the Configuration File +Here an example is shown for how to create the configuration in a YAML file. The steps are similar if using JSON. The steps below build a small but complete configuration. -Create a file, for example `accelerator.yaml`, with any text editor. The steps below build a small but complete configuration, using the names of the [test lattice](../../tutorials/functionality/01_create_accelerator). +Create a file, for example `accelerator.yaml`, with any text editor. If you want the editor to suggest the fields, you can use VS Code together with a JSON Schema. See [Use JSON Schema in VS Code](./tools/use-vscode-json-schema.md) for instructions. ### 1. The Accelerator @@ -48,16 +64,16 @@ The root item is the `Accelerator`. Its required arguments are `facility`, `mach ```yaml class: pyaml.accelerator.Accelerator -facility: My facility -machine: sr +facility: my_facility +machine: storage_ring energy: 1.0e9 ``` ### 2. The Control Modes -Add the control modes as lists in `simulators` and `controls`. Their `name` is also the name used to access them (`accelerator.design`, `accelerator.live`). +Add the control modes as lists in `simulators` and `controls`. Their `name` is also the name used to access them, for example `accelerator.design`, `accelerator.live` etc. -A simulator needs the path to a lattice file. Use `${path:...}` so that the path is resolved relative to the [configuration root](../../explanation/configuration.md#configuration-root) and the lattice is not loaded as a configuration file: +A simulator needs the path to a lattice file. All [formats that can be loaded by pyAT](https://atcollab.github.io/at/p/api/at.load.html#module-at.load) works. If you use the JSON format, you need to use the `${path:...}` [resolver](../../explanation/configuration.md#resolvers) to avoid the lattice being loaded as if it was a configuration file. ```yaml simulators: @@ -66,35 +82,24 @@ simulators: lattice: ${path:lattice.json} ``` -A control system is given by the class of the bindings you use. Its arguments depend on the bindings, for example for `pyaml-cs-oa`: +A control system is given by the class of the bindings you use. The arguments depend on the bindings and catalog type you decide to use. The catalog describes how the keys used by the devices map to control-system signals. See [Control System Catalogs](../../explanation/catalog.md) for the different types of catalogs. + +For example for `pyaml-cs-oa` using a dynamic catalog for TANGO: ```yaml controls: - class: pyaml_cs_oa.controlsystem.OphydAsyncControlSystem name: live backend: tango - catalog: catalog.yaml -``` - -The catalog describes how the keys used by the devices map to control-system signals. It follows exactly the same rule. A static catalog for `pyaml-cs-oa` looks like this (one entry per key): - -```yaml -class: pyaml_cs_oa.static_catalog.StaticCatalog -entries: - - class: pyaml_cs_oa.static_catalog_entry.StaticCatalogEntry - key: AN01-AR/EM-QP/QF.01/magnetic_strength - device: - class: pyaml_cs_oa.tangoAtt.TangoAtt - attribute: AN01-AR/EM-QP/QF.01/magnetic_strength - unit: 1/m - # ... one entry for each key used in the configuration ``` -See [Control System Catalogs](../../explanation/catalog.md) for the different types of catalogs. If you only want to use the simulator, you can leave out `controls` entirely. +If you only want to use the simulator, you can leave out `controls` entirely. ### 3. The Devices -Add the elements of the machine in `devices`. For a magnet, the `model` argument is an object (the magnet model, which also handles the unit conversion), so it is written as a nested item: +Add the elements of the machine in `devices`. For a magnet, the `model` argument is an object (the magnet model, which also handles the unit conversion), so it is written as a nested item. + +The strings given to `physics`, `x_pos` and `y_pos` are keys looked up in the catalog of the control system. ```yaml devices: @@ -116,7 +121,7 @@ devices: y_pos: AN01-AR/DG-EPOS/BPM.01/y ``` -By default, the `name` of an element is also the name of the element in the lattice of the simulator. Use `lattice_names` if they differ. The strings given to `physics`, `x_pos` and `y_pos` are keys looked up in the catalog of the control system. +By default, the `name` of an element is also the name of the element in the lattice of the simulator. If you want to use a different name in pyAML, use `lattice_names` to map between pyAML and the lattice. ### 4. The Arrays @@ -141,8 +146,8 @@ Putting it all together: ```yaml class: pyaml.accelerator.Accelerator -facility: My facility -machine: sr +facility: my_facility +machine: storage_ring energy: 1.0e9 simulators: - class: pyaml.lattice.simulator.Simulator @@ -152,7 +157,6 @@ controls: - class: pyaml_cs_oa.controlsystem.OphydAsyncControlSystem name: live backend: tango - catalog: catalog.yaml devices: - class: pyaml.magnet.quadrupole.Quadrupole name: QF_001 @@ -184,7 +188,7 @@ arrays: ## Split the Configuration into Several Files -For a real machine the configuration becomes long. When the configuration is loaded from a file, any string value ending with `.yaml`, `.yml` or `.json` is replaced by the content of that file (this is how `catalog: catalog.yaml` above is loaded). Inside a list, if the file contains a list, its items are added to the parent list. For example, move the quadrupoles to `devices/quadrupoles.yaml`: +For a real machine the configuration becomes long. When the configuration is loaded from a file, any string value ending with `.yaml`, `.yml` or `.json` is replaced by the content of that file. Inside a list, if the file contains a list, its items are added to the parent list. For example, move the quadrupoles to `devices/quadrupoles.yaml`: ```yaml # devices/quadrupoles.yaml @@ -204,17 +208,14 @@ and refer to it from the main file: ```yaml devices: - devices/quadrupoles.yaml - - class: pyaml.bpm.bpm.BPM - name: BPM_001 - x_pos: AN01-AR/DG-EPOS/BPM.01/x - y_pos: AN01-AR/DG-EPOS/BPM.01/y + # ... ``` Values can also come from environment variables with `${env:NAME}`. See [Resolvers](../../explanation/configuration.md#resolvers) for all options. ## Use Your Own Classes -The rule is not limited to pyAML classes. Any class that can be imported can be used in the configuration, for example a magnet model specific to your facility: +The configuration is not limited to pyAML classes. Any class that can be imported can be used in the configuration, for example a magnet model specific to your facility: ```python # my_facility/models.py @@ -234,7 +235,7 @@ model: ## Load the Configuration -Set the configuration root, which is the directory used to resolve relative paths, then load the file with `Accelerator.load()`: +Set the [configuration root](../../explanation/configuration.md#configuration-root), which is the directory used to resolve relative paths, then load the file with `Accelerator.load()`. ```python from pyaml.configuration import ROOT @@ -246,42 +247,10 @@ accelerator = Accelerator.load("accelerator.yaml") accelerator.design.magnets.get("Quadrupoles").strengths.get() ``` -If the control-system bindings are not installed or you only want to use the simulator, add `ignore_external=True`. The `controls` section is then skipped. - -The configuration can also be given as a nested dictionary with `Accelerator.from_dict()`, following the same rule: +If the control-system bindings are not installed or you only want to use the simulator, add `ignore_external=True` and the `controls` section is skipped without having to remove it from the configuration. -```python -import yaml +The configuration can also be loaded as a nested dictionary with `Accelerator.from_dict()`. This also allows to write the configuration directly as a dictionary instead of a text file if you prefer. -with open("accelerator.yaml") as file: - config = yaml.safe_load(file) - -accelerator = Accelerator.from_dict(config) -``` +To validate the whole configuration before any object is created, add `validate=True`. See [Validate Configuration](./validate-configuration) for details. See the API documentation for the [Accelerator](https://pyaml.readthedocs.io/en/stable/api/pyaml.accelerator.html#module-pyaml.accelerator) for all options. - -## Validate the Configuration - -Each item is checked when it is created: a missing required field or an unknown field raises a `PyAMLConfigException` naming the class and the field. The whole configuration can also be validated before anything is created, which gives all errors at once: - -```python -from pyaml.validation import SchemaRegistry - -SchemaRegistry().discover() -accelerator = Accelerator.load("accelerator.yaml", validate=True) -``` - -The configuration can also be validated without loading it, which is useful if you maintain it separately from pyAML. See [Validate Configuration](./validate-configuration). - -## Tools That Help Writing the Configuration - -Writing the file by hand is often the simplest way to start, but tools based on a [JSON Schema](../../explanation/schema_and_validation.md) can suggest the available fields and check their types while you write: - -- [Use a JSON Schema in VS Code](./use-vscode-json-schema.md) -- [Use the MetaConfigurator](./use-meta-configurator.md), a form-based editor in the browser -- [Use ConfigurationSchema](./use-configuration-schema.ipynb) objects to create the configuration in Python and export it as a dictionary or text file - -AI coding assistants can also help: supply for example a lattice file, a description of the naming conventions of your control system, and the JSON Schema of the pyAML configuration. - -For information about JSON Schemas and how to generate them, see [Configuration Schemas and Validation](../../explanation/schema_and_validation.md) and [Generate JSON Schemas](./generate-json-schema.ipynb). diff --git a/docs/source/how-to/configuration/tools/index.md b/docs/source/how-to/configuration/tools/index.md new file mode 100644 index 0000000..c01d850 --- /dev/null +++ b/docs/source/how-to/configuration/tools/index.md @@ -0,0 +1,26 @@ +# Tools That Help Writing the Configuration + +There are tools available to help to write and modify the configuration. They can suggest the fields and check their types while you write. The tools are under development and testing so new or other tools might be available in the future based on user feedback. + +Some of the tools are based on a [JSON Schema](https://json-schema.org). For information about JSON Schemas and how to generate them, see [Configuration Schemas and Validation](../../../explanation/schema_and_validation.md) and [Generate JSON Schemas](../generate-json-schema.ipynb). + +Currently these tools are available: + +- [Use ConfigurationSchema](./use-configuration-schema.ipynb) objects to create the configuration in Python and export it as a dictionary or text file. This allows to program the configuration. + +- [Use a JSON Schema in VS Code](./use-vscode-json-schema.md) + +- [Use the MetaConfigurator](./use-meta-configurator.md), a form-based editor in the browser + +```{tip} +AI coding assistants can also help: supply for example a lattice file, a description of the naming conventions of your control system, and the JSON Schema of the pyAML configuration and ask it to write the configuration for you. +``` + +```{toctree} +:maxdepth: 1 +:hidden: + +use-configuration-schema +use-vscode-json-schema +use-meta-configurator +``` diff --git a/docs/source/how-to/configuration/use-configuration-schema.ipynb b/docs/source/how-to/configuration/tools/use-configuration-schema.ipynb similarity index 99% rename from docs/source/how-to/configuration/use-configuration-schema.ipynb rename to docs/source/how-to/configuration/tools/use-configuration-schema.ipynb index a46f34c..695faf5 100644 --- a/docs/source/how-to/configuration/use-configuration-schema.ipynb +++ b/docs/source/how-to/configuration/tools/use-configuration-schema.ipynb @@ -7,7 +7,7 @@ "source": [ "# Generate Configuration Using Configuration Schemas\n", "\n", - "This guide shows how to use `ConfigurationSchema` to create a configuration. This allows to program the configuration if you prefer instead of using any of the [options based on a JSON Schema](.//create-configuration.md)." + "This guide shows how to use `ConfigurationSchema` to create a configuration. This allows to program the configuration if you prefer instead of using any of the [options based on a JSON Schema](./index.md)." ] }, { diff --git a/docs/source/how-to/configuration/use-meta-configurator.md b/docs/source/how-to/configuration/tools/use-meta-configurator.md similarity index 98% rename from docs/source/how-to/configuration/use-meta-configurator.md rename to docs/source/how-to/configuration/tools/use-meta-configurator.md index 49e8126..6db9eaa 100644 --- a/docs/source/how-to/configuration/use-meta-configurator.md +++ b/docs/source/how-to/configuration/tools/use-meta-configurator.md @@ -6,7 +6,7 @@ Pre-generated schemas are available in [pyaml-schemas](https://github.com/python-accelerator-middle-layer/pyaml-schemas). These include the classes that are part of the pyAML ecosystem. -If you have facility specific classes that you want to include in your configuration, you can use the `SchemaRegistry` to generate a JSON Schema including them. See [Generate JSON Schemas](../configuration/generate-json-schema) for details. +If you have facility specific classes that you want to include in your configuration, you can use the `SchemaRegistry` to generate a JSON Schema including them. See [Generate JSON Schemas](../generate-json-schema) for details. The schema must describe the document you want to create. For example, a schema for a quadrupole is suitable for editing a quadrupole object, but not for editing a complete accelerator containing controls, simulators, and devices. diff --git a/docs/source/how-to/configuration/use-vscode-json-schema.md b/docs/source/how-to/configuration/tools/use-vscode-json-schema.md similarity index 100% rename from docs/source/how-to/configuration/use-vscode-json-schema.md rename to docs/source/how-to/configuration/tools/use-vscode-json-schema.md diff --git a/docs/source/how-to/configuration/validate-configuration.ipynb b/docs/source/how-to/configuration/validate-configuration.ipynb index 06cacfd..7b3b944 100644 --- a/docs/source/how-to/configuration/validate-configuration.ipynb +++ b/docs/source/how-to/configuration/validate-configuration.ipynb @@ -7,7 +7,21 @@ "source": [ "# Validate Configuration\n", "\n", - "To make sure that the input data provided to pyAML has the correct format, validation is important. The validation is by default automatically performed when loading a configuration into the `Accelerator`, but it can also be done separately if one wishes. This guide shows how to do that.\n", + "To make sure that the input data provided to pyAML has the correct format, validation is important.\n", + "\n", + "If the classes you use have enabled validation during object creation (this is the default for all common pyAML classes), the configuration is validated as part of creating the objects: a missing required field or an unknown field raises a `PyAMLConfigException` naming the class and the field.\n", + "\n", + "The whole configuration can also be validated before anything is created when loading it:\n", + "\n", + "```python\n", + "from pyaml.accelerator import Accelerator\n", + "from pyaml.validation import SchemaRegistry\n", + "\n", + "SchemaRegistry().discover()\n", + "accelerator = Accelerator.load(\"accelerator.yaml\", validate=True)\n", + "```\n", + "\n", + "The configuration can also be validated without loading it, which is useful if you maintain it separately from pyAML. This guide shows how to do that.\n", "\n", "For more details about schemas and the different types of validation, see [Configuration Schemas and Validation](./../../explanation/schema_and_validation.md).\n", "\n", diff --git a/docs/source/how-to/index.md b/docs/source/how-to/index.md index 7a075ab..150b94b 100644 --- a/docs/source/how-to/index.md +++ b/docs/source/how-to/index.md @@ -25,10 +25,7 @@ configuration/create-configuration configuration/use-schema-registry configuration/validate-configuration configuration/generate-json-schema -configuration/use-configuration-schema -configuration/use-meta-configurator -configuration/use-vscode-json-schema - +configuration/tools/index ``` ```{toctree}