diff --git a/README.md b/README.md index dfe064a..939240f 100644 --- a/README.md +++ b/README.md @@ -75,7 +75,7 @@ The following platforms are officially supported (tested): ## CLI -This package ships the `gridpool-cli` command with three subcommands. +This package ships the `gridpool-cli` command with five subcommands. ### Setup @@ -87,6 +87,9 @@ export ASSETS_API_AUTH_KEY="..." export ASSETS_API_SIGN_SECRET="..." ``` +`FREQUENZ_API_KEY` and `FREQUENZ_API_SECRET` are accepted as fallbacks for +`ASSETS_API_AUTH_KEY` and `ASSETS_API_SIGN_SECRET`. + ### Print component formulas ```bash @@ -171,6 +174,28 @@ gridpool-cli generate-config \ --prefer-meters-in-component-formulas > microgrid.toml ``` +### Validate config files + +Check config files offline, without contacting the Assets API, and exit +non-zero on the first error, to gate config-repo CI: + +```bash +gridpool-cli validate microgrid.toml [more.toml ...] +``` + +Each file is validated on its own first, so every record names its own key and +required fields; the files are then validated merged, for the cross-record +checks. + +### Look up a gridpool's enterprise + +Print the enterprise ID that owns a gridpool, read from the config files +(merged as one stack); exits non-zero if the gridpool is not declared: + +```bash +gridpool-cli find-enterprise config.toml [more.toml ...] +``` + ## Contributing If you want to know how to build this project and contribute to it, please diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index 8b2ab23..ac9aa36 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -6,114 +6,29 @@ ## Upgrading -- `MicrogridConfig.load_from_file` is replaced by `AssetsConfig.load_from_files`, - which loads one or more files, merged into one document, and returns the whole - document rather than just its microgrids: - - ```python - configs = AssetsConfig.load_from_files(path).microgrids - ``` - - `load_configs` now returns the whole `AssetsConfig` rather than just its - microgrids, so the file layers' `relations` and `market_locations` survive the - merge; replace `load_configs(...)` with `load_configs(...).microgrids` where - only the microgrid map is needed. `load_from_files` layers files field by field - instead of replacing a complete microgrid entry; fields omitted by a later file - retain the value from the earlier layer and cannot be removed by omission. - -- `Metadata` is removed; its fields (`microgrid_id`, `name`, `gid`, coordinates, - times) now sit directly on `MicrogridConfig`: - - ```python - MicrogridConfig(microgrid_id=1, name="Grid") # was meta=Metadata(...) - ``` - - In TOML they move up one level, `assets.microgrids.1.name` rather than - `assets.microgrids.1.meta.name`. A file still nesting a microgrid's fields - under `meta` loads, lifted with a deprecation warning. - -- A microgrid's `delivery_area` is removed. Move its value to a relation's - `delivery_area.code`, and set `delivery_area.code_type` when the code is not - EIC. When relations are present, the legacy `gid` must be their sole - gridpool ID; remove it for a microgrid that participates in several gridpools. - -- `load_configs_from_files` is removed. Use `AssetsConfig.load_from_files`, - which returns the whole document rather than just its microgrids: replace - `load_configs_from_files(files)` with - `AssetsConfig.load_from_files(files).microgrids`, or use the returned - `AssetsConfig` directly to keep relations and market locations. - -- `load_configs_from_api` is now private. For an API-only load call - `load_configs(assets_client=..., microgrid_ids=...)` and read its `.microgrids`. - -- `merge_config_maps` and `merge_microgrid_configs` are removed. Layering is now - done on the raw tables before loading, inside `load_configs` and - `AssetsConfig.load_from_files`; pass all the layers to one of those instead of - merging loaded objects. - -- Relation validity bounds and `at` query instants must include a UTC offset. - -- The implementation modules `config.load` and `config.microgrid` are now - private. Import their public names from `frequenz.gridpool.config` instead. - -- Microgrids are now keyed by `int` microgrid ID, not `str`. This covers - `AssetsConfig.microgrids`, including documents returned by `load_configs`. - Index the mapping by integer ID: - - ```python - configs[1] # was configs["1"] - ``` - -- The `AC_ACTIVE_POWER` deprecation warning is dropped. Use `AC_POWER_ACTIVE` - as the formula metric key; the old name is no longer flagged on load. - + ## New Features -- `AssetsConfig` gives the `assets` namespace a type, so the entities still to - come are added as fields rather than as more dict lookups. Microgrid IDs are - checked during construction. `AssetsConfig.check()` performs the topology-wide - checks after all layers have been merged; the file loaders call it unless - `AssetsConfig.load_from_files` is passed `check=False`. - - File loaders ignore unknown entity tables with a warning, so a reader keeps - working against files that already carry newer entities. +- `gridpool-cli validate ` checks config files offline and exits + non-zero on the first error, to gate config-repo CI. Each file must be valid + on its own, so a record names its own key and required fields; the files are + then checked merged, for the cross-record checks. - The `assets` table may carry a `version`; on load a document is run through a - migration pipeline that brings older layouts up to the current format, so - legacy files keep working. The version tracks the assets format alone, not - the whole document. +- `gridpool-cli` accepts `FREQUENZ_API_KEY` and `FREQUENZ_API_SECRET` as a + fallback pair for `ASSETS_API_AUTH_KEY` and `ASSETS_API_SIGN_SECRET`. -- Market topology is described under `assets.relations`, based on the Assets - API `MarketTopologyRelation`: each record links at least two of a gridpool, a - microgrid and a market location, filed under a - `GML` key derived from its own - sides. A relation naming a gridpool sits in a `delivery_area` that rides on - the relation, so a gridpool-to-microgrid relation with no market location still - carries one. A relation's validity lives in `validity`, each entry a half-open - `[start, end)` datetime period it applies over. Use-case-specific periods - qualify a relation; separate relations let one microgrid participate in - several gridpools. The config extends the API with plain periods for relations - that do not distinguish use cases. A gridpool-free microgrid-to-market-location - relation may also carry a delivery area for a direct mapping. Market locations - live under `assets.market_locations` as self-describing entries carrying their - own identifier, how to read it (`MALO_ID` by default), and the Assets API market - area (`EU_DE` by default). A relation's `delivery_area` is a `code` plus a - `code_type` that defaults to EIC, so an EIC area is just - `delivery_area.code = "..."`; EIC codes are check-character-validated. Raw - market-location IDs must be unique within a document, including across market - areas. +- Gridpools are described under `assets.gridpools`, each entry naming the + enterprise that owns the gridpool. `AssetsConfig.find_enterprise(gridpool_id)` + returns the configured owner. + `gridpool-cli find-enterprise ` prints it from the config. + `AssetsConfig.check` enforces the one-enterprise-per-gridpool invariant: a + gridpool's microgrids may not disagree on it, and a declared enterprise must + match the inferred one. - `AssetsConfig` answers the common lookups with `find_relations` and the - projections `find_delivery_areas`, `find_market_locations` and - `find_microgrids`, each filtered by the other sides and an instant. - - Time-varying enterprise ownership is outside this change; - `MicrogridConfig.enterprise_id` remains as a scalar field. +- A config derived from the Assets API now carries each microgrid's + `enterprise_id`. ## Bug Fixes -- Layering config files no longer resets a field a later file leaves unset back - to its default. The raw tables are merged before they are loaded. - diff --git a/src/frequenz/gridpool/cli/__main__.py b/src/frequenz/gridpool/cli/__main__.py index 425d3a3..7c2659a 100644 --- a/src/frequenz/gridpool/cli/__main__.py +++ b/src/frequenz/gridpool/cli/__main__.py @@ -5,11 +5,13 @@ import os import tempfile +import tomllib from pathlib import Path import asyncclick as click from frequenz.client.assets import AssetsApiClient from frequenz.client.common.microgrid import MicrogridId +from marshmallow import ValidationError from frequenz.gridpool import ComponentGraphConfig, ComponentGraphGenerator from frequenz.gridpool.cli._dump_config import dump_map @@ -17,12 +19,37 @@ from frequenz.gridpool.cli._render_graph import ComponentGraphRenderer, RenderOptions from frequenz.gridpool.config import AssetsConfig, load_configs +_LOAD_ERRORS = (ValueError, TypeError, tomllib.TOMLDecodeError, ValidationError) +"""Exceptions raised when a config file fails to load or validate.""" + @click.group() async def cli() -> None: """CLI tool for gridpool functionality.""" +def _assets_credentials() -> tuple[str, str, str]: + """Resolve Assets API URL, auth key and sign secret from the environment. + + `FREQUENZ_API_KEY` and `FREQUENZ_API_SECRET` are accepted as a fallback pair + for `ASSETS_API_AUTH_KEY` and `ASSETS_API_SIGN_SECRET`. The key and secret + are taken as a whole from one source, never mixed across the two. + """ + url = os.environ.get("ASSETS_API_URL") + key = os.environ.get("ASSETS_API_AUTH_KEY") + secret = os.environ.get("ASSETS_API_SIGN_SECRET") + if not key and not secret: + key = os.environ.get("FREQUENZ_API_KEY") + secret = os.environ.get("FREQUENZ_API_SECRET") + if not url or not key or not secret: + raise click.ClickException( + "ASSETS_API_URL and auth credentials must be set: " + "ASSETS_API_AUTH_KEY (or FREQUENZ_API_KEY) and " + "ASSETS_API_SIGN_SECRET (or FREQUENZ_API_SECRET)." + ) + return url, key, secret + + def _graph_config(prefer_meters: bool) -> ComponentGraphConfig | None: """Build the graph config for `--prefer-meters-in-component-formulas`. @@ -55,13 +82,7 @@ async def print_formulas( prefer_meters_in_component_formulas: bool, ) -> None: """Fetch and print component graph formulas for a microgrid.""" - url = os.environ.get("ASSETS_API_URL") - key = os.environ.get("ASSETS_API_AUTH_KEY") - secret = os.environ.get("ASSETS_API_SIGN_SECRET") - if not url or not key or not secret: - raise click.ClickException( - "ASSETS_API_URL, ASSETS_API_AUTH_KEY, ASSETS_API_SIGN_SECRET must be set." - ) + url, key, secret = _assets_credentials() async with AssetsApiClient( url, @@ -90,6 +111,70 @@ async def print_formulas( ) +@cli.command() +@click.argument( + "config_files", + nargs=-1, + required=True, + type=click.Path(exists=True, dir_okay=False, path_type=Path), +) +async def validate(config_files: tuple[Path, ...]) -> None: + """Validate each config file on its own, then the merged stack. + + Each file must stand alone: every record names its own key and required + fields. The merged stack then adds the cross-record checks. Exits non-zero + on the first error, to gate CI. + """ + for config_file in config_files: + try: + AssetsConfig.load_from_files([config_file]) + except _LOAD_ERRORS as exc: + raise click.ClickException(f"{config_file}: {exc}") from exc + + try: + config = AssetsConfig.load_from_files(list(config_files)) + except _LOAD_ERRORS as exc: + raise click.ClickException(str(exc)) from exc + + click.echo( + f"OK: {len(config_files)} file(s), {len(config.microgrids)} microgrid(s), " + f"{len(config.relations)} relation(s), " + f"{len(config.market_locations)} market location(s).", + err=True, + ) + + +@cli.command("find-enterprise") +@click.argument("gridpool_id", type=int) +@click.argument( + "config_files", + nargs=-1, + required=True, + type=click.Path(exists=True, dir_okay=False, path_type=Path), +) +async def find_enterprise(gridpool_id: int, config_files: tuple[Path, ...]) -> None: + """Print the enterprise ID owning GRIDPOOL_ID, read from the config files. + + The files are read as one merged stack. The owner is taken from the + `gridpools` entry without validating the full document. Exits non-zero if + the gridpool is not configured. + """ + try: + # The declared `gridpools` entry is the source of truth for ownership, + # so read it even from a document whole-document validation would reject. + config = AssetsConfig.load_from_files(list(config_files), check=False) + enterprise_id = config.find_enterprise(gridpool_id) + except _LOAD_ERRORS as exc: + raise click.ClickException(str(exc)) from exc + + if enterprise_id is None: + raise click.ClickException( + f"Could not determine the enterprise for gridpool {gridpool_id} " + "from the given config(s)." + ) + click.echo(enterprise_id) + + @cli.command("render-graph") @click.argument("microgrid_id", type=int) @click.option( @@ -107,13 +192,7 @@ async def print_formulas( ) async def render_graph(microgrid_id: int, output: str, show: bool) -> None: """Render and save a component graph visualization for a microgrid.""" - url = os.environ.get("ASSETS_API_URL") - key = os.environ.get("ASSETS_API_AUTH_KEY") - secret = os.environ.get("ASSETS_API_SIGN_SECRET") - if not url or not key or not secret: - raise click.ClickException( - "ASSETS_API_URL, ASSETS_API_AUTH_KEY, ASSETS_API_SIGN_SECRET must be set." - ) + url, key, secret = _assets_credentials() try: async with AssetsApiClient( @@ -151,9 +230,16 @@ async def render_graph(microgrid_id: int, output: str, show: bool) -> None: "--inplace", is_flag=True, default=False, - help="Patch --default in place instead of printing to stdout. Preserves " - "existing comments, ordering and formatting in that file; only fills in " - "values it is missing. Requires --default.", + help="Patch --default in place instead of printing to stdout, refreshing " + "the managed values while keeping its comments, ordering and formatting. " + "Requires --default.", +) +@click.option( + "--fill-missing", + is_flag=True, + default=False, + help="With --inplace, only add values --default is missing, leaving the " + "values already in it untouched.", ) @click.option( "--prefer-meters-in-component-formulas", @@ -162,11 +248,13 @@ async def render_graph(microgrid_id: int, output: str, show: bool) -> None: help="Read the meter before the component in the per-category formulas. " "This is the order used before component graph v0.5.0.", ) -async def generate_config( +async def generate_config( # pylint: disable=too-many-locals microgrid_ids: tuple[int, ...], + *, default_file: Path | None, override_file: Path | None, inplace: bool, + fill_missing: bool, prefer_meters_in_component_formulas: bool, ) -> None: """Generate microgrid config from the Assets API as TOML. @@ -184,21 +272,17 @@ async def generate_config( component graph v0.5.0. With `--inplace`, `--default` is patched directly instead: candidate values - come from the Assets API (with `--override` layered on top), and only - leaves `--default` is missing are added, preserving its existing comments, - field order and formatting. If no microgrid IDs are given, every microgrid - already in `--default` is processed. + come from the Assets API (with `--override` layered on top) and refresh the + managed values, keeping the file's comments, field order and formatting. + With `--fill-missing`, only values `--default` lacks are added. If no + microgrid IDs are given, every microgrid already in `--default` is processed. """ if inplace and default_file is None: raise click.ClickException("--inplace requires --default.") + if fill_missing and not inplace: + raise click.ClickException("--fill-missing requires --inplace.") - url = os.environ.get("ASSETS_API_URL") - key = os.environ.get("ASSETS_API_AUTH_KEY") - secret = os.environ.get("ASSETS_API_SIGN_SECRET") - if not url or not key or not secret: - raise click.ClickException( - "ASSETS_API_URL, ASSETS_API_AUTH_KEY, ASSETS_API_SIGN_SECRET must be set." - ) + url, key, secret = _assets_credentials() ids = list(dict.fromkeys(microgrid_ids)) or None if inplace and ids is None: @@ -236,13 +320,25 @@ async def generate_config( if inplace: assert default_file is not None - patched = patch_file(default_file, configs) + try: + patched = patch_file(default_file, configs, fill_missing=fill_missing) + except ValueError as exc: + raise click.ClickException(str(exc)) from exc fd, tmp_name = tempfile.mkstemp( dir=default_file.parent, prefix=f".{default_file.name}." ) try: with os.fdopen(fd, "w") as tmp_file: tmp_file.write(patched) + # Guard against a patch that corrupts the file (e.g. bad splicing) + # before it overwrites the user's config. + try: + AssetsConfig.load_from_files(Path(tmp_name)) + except _LOAD_ERRORS as exc: + raise click.ClickException( + f"Refusing to write {default_file}: the patched result is " + f"invalid: {exc}" + ) from exc os.replace(tmp_name, default_file) except BaseException: os.remove(tmp_name) diff --git a/src/frequenz/gridpool/cli/_dump_config.py b/src/frequenz/gridpool/cli/_dump_config.py index e952abe..abbec28 100644 --- a/src/frequenz/gridpool/cli/_dump_config.py +++ b/src/frequenz/gridpool/cli/_dump_config.py @@ -5,9 +5,11 @@ Renders a `{microgrid_id: MicrogridConfig}` mapping as dotted-key TOML, e.g.: - 115.microgrid_id = 115 - 115.name = "Demo Grid" - 115.latitude = 52.52 + assets.version = 1 + + assets.microgrids.115.microgrid_id = 115 + assets.microgrids.115.name = "Demo Grid" + assets.microgrids.115.latitude = 52.52 This is the inverse of `AssetsConfig.load_from_files`. Value rendering (quoting, escaping, key-quoting, list/number/datetime formatting) is delegated to @@ -21,6 +23,9 @@ from tomlkit.items import Integer, Trivia from frequenz.gridpool.config import MicrogridConfig +from frequenz.gridpool.config._migrations import _CURRENT_VERSION + +_MICROGRID_PREFIX = ["assets", "microgrids"] def _is_empty(value: Any) -> bool: @@ -83,14 +88,14 @@ def dump_map(configs: dict[int, MicrogridConfig]) -> str: """ schema = MicrogridConfig.Schema() doc = tomlkit.document() + doc.append(tomlkit.key(["assets", "version"]), _CURRENT_VERSION) for mid in sorted(configs): dumped = schema.dump(configs[mid]) assert isinstance(dumped, dict) - leaves = _iter_leaves([str(mid)], dumped) + leaves = _iter_leaves([*_MICROGRID_PREFIX, str(mid)], dumped) if not leaves: continue - if doc.body: - doc.add(tomlkit.nl()) + doc.add(tomlkit.nl()) for path, value in leaves: doc.append(tomlkit.key(path), _format_value(value)) return tomlkit.dumps(doc) diff --git a/src/frequenz/gridpool/cli/_patch_config.py b/src/frequenz/gridpool/cli/_patch_config.py index ed28da8..d550691 100644 --- a/src/frequenz/gridpool/cli/_patch_config.py +++ b/src/frequenz/gridpool/cli/_patch_config.py @@ -4,8 +4,9 @@ """Patch existing dotted-key TOML files with new microgrid config values. Unlike `dump_map`, which rebuilds a TOML document from scratch, this module -only adds missing leaves and missing microgrid entries. Values already on -disk always win, so comments, field order and number formatting survive. +edits the document in place, so its comments, field order and number formatting +survive. By default it refreshes the managed leaves, overwriting them; with +`fill_missing` it only adds the leaves the document lacks. """ from collections.abc import Mapping @@ -17,33 +18,42 @@ from frequenz.gridpool.config import MicrogridConfig -from ._dump_config import _format_value, _iter_leaves +from ._dump_config import _MICROGRID_PREFIX, _format_value, _iter_leaves -def patch_file(path: Path, configs: dict[int, MicrogridConfig]) -> str: - """Patch the TOML file at `path` with any leaves missing from `configs`. +def patch_file( + path: Path, configs: dict[int, MicrogridConfig], *, fill_missing: bool = False +) -> str: + """Patch the TOML file at `path` with the values in `configs`. Args: path: Path to the existing TOML file to patch. configs: Mapping from microgrid ID to `MicrogridConfig`. + fill_missing: Only add leaves the file lacks, leaving existing values + untouched; by default the managed leaves are overwritten. Returns: The patched TOML text; the caller is responsible for writing it back. """ - return patch_text(path.read_text(), configs) + return patch_text(path.read_text(), configs, fill_missing=fill_missing) -def patch_text(original: str, configs: dict[int, MicrogridConfig]) -> str: - """Patch dotted-key TOML text with any leaves missing from `configs`. +def patch_text( + original: str, configs: dict[int, MicrogridConfig], *, fill_missing: bool = False +) -> str: + """Patch dotted-key TOML text with the values in `configs`. Args: original: The existing TOML text to patch. configs: Mapping from microgrid ID to `MicrogridConfig`. + fill_missing: Only add leaves the text lacks, leaving existing values + untouched; by default the managed leaves are overwritten. Returns: The patched TOML text. """ doc = tomlkit.parse(original) + _reject_legacy_layout(doc) schema = MicrogridConfig.Schema() # Leaves needing a whole new sub-table can't be inserted via item assignment @@ -59,12 +69,12 @@ def patch_text(original: str, configs: dict[int, MicrogridConfig]) -> str: if not leaves: continue - if mid not in doc: + if not _leaf_exists(doc, mid, []): _append_new_entry(doc, mid, leaves) continue for path, value in leaves: - if _leaf_exists(doc, mid, path): + if fill_missing and _leaf_exists(doc, mid, path): continue if not _insert_leaf(doc, mid, path, value): orphans.setdefault(mid, []).append((path, value)) @@ -75,10 +85,44 @@ def patch_text(original: str, configs: dict[int, MicrogridConfig]) -> str: return text +def _reject_legacy_layout(doc: TOMLDocument) -> None: + """Refuse a document that is not in the current `assets.microgrids` layout. + + Patching navigates by `assets.microgrids.`, so a top-level or + `meta`-nested entry would be duplicated rather than edited. Such a file + must be rebuilt with `generate-config` (without `--inplace`) first. + + Args: + doc: The parsed document to check. + + Raises: + ValueError: If the document uses a deprecated layout. + """ + hint = "rebuild it with generate-config (without --inplace) first" + if stray := sorted(k for k in doc if k != "assets"): + raise ValueError( + f"Cannot patch in place: deprecated top-level keys {stray}; {hint}." + ) + assets = doc.get("assets", {}) + microgrids = assets.get("microgrids", {}) if isinstance(assets, Mapping) else {} + if not isinstance(microgrids, Mapping): + microgrids = {} + metas = sorted( + mid + for mid, entry in microgrids.items() + if isinstance(entry, Mapping) and "meta" in entry + ) + if metas: + raise ValueError( + f"Cannot patch in place: microgrids {metas} nest fields under the " + f"deprecated `meta` table; {hint}." + ) + + def _leaf_exists(doc: TOMLDocument, mid: str, path: list[str]) -> bool: - """Whether the dotted key `mid.path...` already has a value in `doc`.""" + """Whether the dotted key `assets.microgrids.mid.path...` has a value in `doc`.""" node: Any = doc - for key in (mid, *path): + for key in (*_MICROGRID_PREFIX, mid, *path): if not isinstance(node, Mapping) or key not in node: return False node = node[key] @@ -100,14 +144,14 @@ def _insert_leaf(doc: TOMLDocument, mid: str, path: list[str], value: Any) -> bo """ node: Any = doc matched = 0 - for key in (mid, *path[:-1]): + full_path = [*_MICROGRID_PREFIX, mid, *path] + for key in full_path[:-1]: nxt = node[key] if isinstance(node, Mapping) and key in node else None if not isinstance(nxt, Mapping): break node = nxt matched += 1 - full_path = [mid, *path] if matched != len(full_path) - 1: return False node[full_path[-1]] = _format_value(value) @@ -128,14 +172,14 @@ def _append_new_entry( if doc.body: doc.add(tomlkit.nl()) for path, value in leaves: - doc.append(tomlkit.key([mid, *path]), _format_value(value)) + doc.append(tomlkit.key([*_MICROGRID_PREFIX, mid, *path]), _format_value(value)) def _render_lines(mid: str, leaves: list[tuple[list[str], Any]]) -> list[str]: - """Render `(path, value)` leaves as standalone `mid.path = value` lines.""" + """Render leaves as standalone `assets.microgrids.mid.path = value` lines.""" tmp = tomlkit.document() for path, value in leaves: - tmp.append(tomlkit.key([mid, *path]), _format_value(value)) + tmp.append(tomlkit.key([*_MICROGRID_PREFIX, mid, *path]), _format_value(value)) return tomlkit.dumps(tmp).splitlines(keepends=True) @@ -159,8 +203,8 @@ def _splice_orphans(text: str, orphans: dict[str, list[tuple[list[str], Any]]]) def _last_line_index_for_mid(lines: list[str], mid: str) -> int: - """Index of the last line belonging to `mid` (its key starts with `mid.`).""" - prefix = f"{mid}." + """Index of the last line belonging to `mid` (its key starts with the prefix).""" + prefix = f"{'.'.join(_MICROGRID_PREFIX)}.{mid}." for idx in range(len(lines) - 1, -1, -1): if lines[idx].lstrip().startswith(prefix): return idx diff --git a/src/frequenz/gridpool/config/__init__.py b/src/frequenz/gridpool/config/__init__.py index fa70a4e..05c671f 100644 --- a/src/frequenz/gridpool/config/__init__.py +++ b/src/frequenz/gridpool/config/__init__.py @@ -6,6 +6,7 @@ from frequenz.microgrid_component_graph import ComponentGraphConfig, FormulaOverrides from ._assets import AssetsConfig +from ._gridpool import GridpoolConfig from ._load import load_configs from ._microgrid import ( BatteryConfig, @@ -32,6 +33,7 @@ "ComponentTypeConfig", "DeliveryAreaConfig", "FormulaOverrides", + "GridpoolConfig", "MarketLocationConfig", "MicrogridConfig", "PVConfig", diff --git a/src/frequenz/gridpool/config/_assets.py b/src/frequenz/gridpool/config/_assets.py index ff88ae9..3b8d179 100644 --- a/src/frequenz/gridpool/config/_assets.py +++ b/src/frequenz/gridpool/config/_assets.py @@ -15,6 +15,7 @@ from marshmallow import Schema from marshmallow_dataclass import dataclass +from ._gridpool import GridpoolConfig from ._microgrid import MicrogridConfig from ._migrations import _CURRENT_VERSION, migrate from ._topology import DeliveryAreaConfig, MarketLocationConfig, RelationConfig @@ -89,6 +90,9 @@ class AssetsConfig: microgrids: dict[int, MicrogridConfig] = field(default_factory=dict) """Microgrids, keyed by microgrid ID.""" + gridpools: dict[int, GridpoolConfig] = field(default_factory=dict) + """Gridpools, keyed by gridpool ID.""" + market_locations: dict[str, MarketLocationConfig] = field(default_factory=dict) """Market locations, keyed by their identifier.""" @@ -123,6 +127,11 @@ def __post_init__(self) -> None: raise ValueError( f"Microgrid ID mismatch: key {mid} != {cfg.microgrid_id}" ) + for gpid, gridpool in self.gridpools.items(): + if int(gridpool.gridpool_id) != gpid: + raise ValueError( + f"Gridpool ID mismatch: key {gpid} != {gridpool.gridpool_id}" + ) def check(self) -> None: """Check the document as a whole, once every layer has been merged. @@ -131,8 +140,9 @@ def check(self) -> None: ValueError: If an entry's identifier disagrees with the key it is filed under, a relation names fewer than two sides, its key disagrees with its fields, a gridpool relation names no delivery - area, one market location is placed in two of them, or a legacy - microgrid gridpool ID disagrees with its relations. + area, one market location is placed in two of them, a legacy + microgrid gridpool ID disagrees with its relations, or a + gridpool's declared and inferred enterprise disagree. """ for key, location in self.market_locations.items(): if location.id is None: @@ -143,6 +153,7 @@ def check(self) -> None: ) self._check_relations() self._check_legacy_gridpool_ids() + self._check_gridpool_enterprises() def _check_relations(self) -> None: """Check the relations are complete, well-keyed and area-consistent. @@ -213,6 +224,61 @@ def _check_legacy_gridpool_ids(self) -> None: f"{sorted(relation_gids)}; remove gid when several apply" ) + def _derive_enterprise(self, gridpool_id: int) -> int | None: + """Infer a gridpool's enterprise from the microgrids its relations name. + + Args: + gridpool_id: The gridpool whose enterprise to infer. + + Returns: + The inferred enterprise ID, or `None` if none can be inferred. + + Raises: + ValueError: If the related microgrids disagree on the enterprise. + """ + enterprises: set[int] = set() + for mid in self.find_microgrids(gridpool_id=gridpool_id): + microgrid = self.microgrids.get(mid) + if microgrid is not None and microgrid.enterprise_id is not None: + enterprises.add(microgrid.enterprise_id) + if not enterprises: + return None + if len(enterprises) > 1: + raise ValueError( + f"Gridpool {gridpool_id}: its microgrids disagree on the owning " + f"enterprise: {sorted(enterprises)}" + ) + return enterprises.pop() + + def _check_gridpool_enterprises(self) -> None: + """Check declared and inferable gridpool enterprises agree. + + A gridpool owns one enterprise, so its microgrids must not disagree on + it, and a declared `gridpools` entry must match what they imply. + + Raises: + ValueError: If a gridpool's microgrids disagree on the enterprise, + or a declared enterprise differs from the inferred one. + """ + gridpool_ids = set(self.gridpools) | { + relation.gridpool_id + for relation in self.relations.values() + if relation.gridpool_id is not None + } + for gpid in gridpool_ids: + inferred = self._derive_enterprise(gpid) + declared = self.gridpools.get(gpid) + if ( + declared is not None + and inferred is not None + and declared.enterprise_id != inferred + ): + raise ValueError( + f"Gridpool {gpid}: declared enterprise " + f"{declared.enterprise_id} disagrees with its microgrids' " + f"enterprise {inferred}" + ) + def find_relations( self, *, @@ -350,6 +416,18 @@ def find_microgrids( ) ) + def find_enterprise(self, gridpool_id: int) -> int | None: + """Find the configured enterprise owning `gridpool_id`. + + Args: + gridpool_id: The gridpool to look up. + + Returns: + The owning enterprise ID, or `None` when the gridpool is not configured. + """ + gridpool = self.gridpools.get(gridpool_id) + return gridpool.enterprise_id if gridpool is not None else None + @classmethod def _warn_unknown_entities(cls, assets: dict[str, Any], source: Path) -> None: """Warn about entity tables that this version drops on load.""" diff --git a/src/frequenz/gridpool/config/_gridpool.py b/src/frequenz/gridpool/config/_gridpool.py new file mode 100644 index 0000000..2bdc4e5 --- /dev/null +++ b/src/frequenz/gridpool/config/_gridpool.py @@ -0,0 +1,17 @@ +# License: MIT +# Copyright © 2026 Frequenz Energy-as-a-Service GmbH + +"""Data model for a gridpool.""" + +from marshmallow_dataclass import dataclass + + +@dataclass +class GridpoolConfig: + """Configuration of a gridpool.""" + + gridpool_id: int + """ID of the gridpool.""" + + enterprise_id: int + """Enterprise that owns the gridpool.""" diff --git a/src/frequenz/gridpool/config/_load.py b/src/frequenz/gridpool/config/_load.py index 9a48a03..0dc5a01 100644 --- a/src/frequenz/gridpool/config/_load.py +++ b/src/frequenz/gridpool/config/_load.py @@ -219,6 +219,9 @@ async def _build_config_from_metadata( location = mgrid.location if mgrid.location else None return MicrogridConfig( microgrid_id=microgrid_id, + # The Assets API always sets an EnterpriseId; an unassigned enterprise + # comes back as the proto default 0, which we treat as absent. + enterprise_id=int(mgrid.enterprise_id) or None, latitude=location.latitude if location else None, longitude=location.longitude if location else None, ) diff --git a/tests/test_cli.py b/tests/test_cli.py index 4e48557..782746f 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -3,6 +3,7 @@ """Tests for the gridpool CLI.""" +from pathlib import Path from unittest.mock import AsyncMock, MagicMock, patch from asyncclick.testing import CliRunner @@ -28,7 +29,9 @@ def _mock_client() -> MagicMock: """Mock an Assets API client: grid 1 -> meter 2 -> solar inverter 4.""" client = MagicMock(spec=AssetsApiClient) - client.get_microgrid = AsyncMock(return_value=MagicMock(location=None)) + client.get_microgrid = AsyncMock( + return_value=MagicMock(location=None, enterprise_id=7) + ) client.list_microgrid_electrical_components = AsyncMock( return_value=[ GridConnectionPoint( @@ -85,10 +88,246 @@ async def test_generate_config_prefer_meters_flips_the_order() -> None: assert result.exit_code == 0, result.output assert ( - '10.ctype.pv.formula.AC_POWER_ACTIVE = "COALESCE(#2, #4, 0.0)"' in result.output + 'assets.microgrids.10.ctype.pv.formula.AC_POWER_ACTIVE = "COALESCE(#2, #4, 0.0)"' + in result.output + ) + + +async def test_freq_api_env_vars_are_accepted_as_fallbacks() -> None: + """`FREQUENZ_API_{KEY,SECRET}` stand in when the `ASSETS_API_*` vars are unset.""" + env = { + "ASSETS_API_URL": "grpc://localhost", + "ASSETS_API_AUTH_KEY": None, + "ASSETS_API_SIGN_SECRET": None, + "FREQUENZ_API_KEY": "freq-key", + "FREQUENZ_API_SECRET": "freq-secret", + } + client = _patched_client() + with patch("frequenz.gridpool.cli.__main__.AssetsApiClient", client): + result = await CliRunner().invoke(cli, ["print-formulas", "10"], env=env) + + assert result.exit_code == 0, result.output + client.assert_called_once_with( + "grpc://localhost", auth_key="freq-key", sign_secret="freq-secret" ) +async def test_partial_assets_credentials_are_not_mixed_with_fallbacks() -> None: + """A half-set `ASSETS_API_*` pair is not completed from `FREQUENZ_API_*`.""" + env = { + "ASSETS_API_URL": "grpc://localhost", + "ASSETS_API_AUTH_KEY": "key", + "ASSETS_API_SIGN_SECRET": None, + "FREQUENZ_API_KEY": "freq-key", + "FREQUENZ_API_SECRET": "freq-secret", + } + result = await CliRunner().invoke(cli, ["print-formulas", "10"], env=env) + + assert result.exit_code != 0, result.output + + +async def test_missing_credentials_fail_with_a_readable_message() -> None: + """With no credentials set the command exits non-zero and names the vars.""" + env = { + "ASSETS_API_URL": "grpc://localhost", + "ASSETS_API_AUTH_KEY": None, + "ASSETS_API_SIGN_SECRET": None, + "FREQUENZ_API_KEY": None, + "FREQUENZ_API_SECRET": None, + } + result = await CliRunner().invoke(cli, ["print-formulas", "10"], env=env) + + assert result.exit_code != 0 + assert "FREQUENZ_API_KEY" in result.output + + def test_graph_config_is_none_without_the_flag() -> None: """With no flag no config is built, so the library's defaults apply.""" assert _graph_config(False) is None + + +async def test_inplace_refuses_to_write_an_invalid_patch() -> None: + """A patch that yields invalid TOML must not overwrite the target file.""" + with CliRunner().isolated_filesystem(): + original = "assets.microgrids.10.microgrid_id = 10\n" + Path("cfg.toml").write_text(original, encoding="utf-8") + with ( + patch("frequenz.gridpool.cli.__main__.AssetsApiClient", _patched_client()), + patch( + "frequenz.gridpool.cli.__main__.patch_file", + return_value="this is = = not valid toml", + ), + ): + result = await CliRunner().invoke( + cli, + ["generate-config", "10", "--inplace", "--default", "cfg.toml"], + env=_ENV, + ) + + assert result.exit_code != 0, result.output + assert "invalid" in result.output.lower() + assert Path("cfg.toml").read_text(encoding="utf-8") == original + + +async def test_inplace_refuses_to_write_an_inconsistent_patch() -> None: + """A patch that conflicts with its gridpool must not replace the target.""" + with CliRunner().isolated_filesystem(): + original = ( + "assets.gridpools.80.gridpool_id = 80\n" + "assets.gridpools.80.enterprise_id = 42\n" + "assets.microgrids.10.microgrid_id = 10\n" + "assets.relations.G80M10.gridpool_id = 80\n" + "assets.relations.G80M10.microgrid_id = 10\n" + 'assets.relations.G80M10.delivery_area.code = "10YDE-RWENET---I"\n' + ) + Path("cfg.toml").write_text(original, encoding="utf-8") + with patch("frequenz.gridpool.cli.__main__.AssetsApiClient", _patched_client()): + result = await CliRunner().invoke( + cli, + ["generate-config", "10", "--inplace", "--default", "cfg.toml"], + env=_ENV, + ) + + assert result.exit_code != 0, result.output + assert "disagrees" in result.output + assert Path("cfg.toml").read_text(encoding="utf-8") == original + + +async def test_find_enterprise_prints_the_owner() -> None: + """The command reads a gridpool's owning enterprise from the config.""" + with CliRunner().isolated_filesystem(): + Path("gp.toml").write_text( + "assets.gridpools.80.gridpool_id = 80\n" + "assets.gridpools.80.enterprise_id = 42\n", + encoding="utf-8", + ) + result = await CliRunner().invoke(cli, ["find-enterprise", "80", "gp.toml"]) + + assert result.exit_code == 0, result.output + assert result.output.strip() == "42" + + +async def test_find_enterprise_requires_a_gridpool_entry() -> None: + """Related microgrids are not used as a fallback.""" + with CliRunner().isolated_filesystem(): + Path("cfg.toml").write_text( + "assets.microgrids.10.microgrid_id = 10\n" + "assets.microgrids.10.enterprise_id = 7\n" + "assets.relations.G80M10.gridpool_id = 80\n" + "assets.relations.G80M10.microgrid_id = 10\n", + encoding="utf-8", + ) + result = await CliRunner().invoke(cli, ["find-enterprise", "80", "cfg.toml"]) + + assert result.exit_code != 0 + assert "80" in result.output + + +async def test_find_enterprise_skips_consistency_validation() -> None: + """Deployment lookup reads the gridpool even if other entries conflict.""" + with CliRunner().isolated_filesystem(): + Path("cfg.toml").write_text( + "assets.gridpools.80.gridpool_id = 80\n" + "assets.gridpools.80.enterprise_id = 42\n" + "assets.microgrids.10.microgrid_id = 10\n" + "assets.microgrids.10.enterprise_id = 7\n" + "assets.relations.G80M10.gridpool_id = 80\n" + "assets.relations.G80M10.microgrid_id = 10\n", + encoding="utf-8", + ) + result = await CliRunner().invoke(cli, ["find-enterprise", "80", "cfg.toml"]) + + assert result.exit_code == 0, result.output + assert result.output.strip() == "42" + + +async def test_find_enterprise_fails_for_an_unknown_gridpool() -> None: + """A gridpool with no entry exits non-zero with a readable message.""" + with CliRunner().isolated_filesystem(): + Path("gp.toml").write_text( + "assets.gridpools.80.gridpool_id = 80\n" + "assets.gridpools.80.enterprise_id = 42\n", + encoding="utf-8", + ) + result = await CliRunner().invoke(cli, ["find-enterprise", "99", "gp.toml"]) + + assert result.exit_code != 0 + assert "99" in result.output + + +async def test_validate_accepts_a_valid_stack() -> None: + """A well-formed document validates with a zero exit code.""" + with CliRunner().isolated_filesystem(): + Path("good.toml").write_text( + "[assets.relations.G80M241L10208446344]\n" + "gridpool_id = 80\n" + "microgrid_id = 241\n" + 'market_location_id = "10208446344"\n' + 'delivery_area.code = "10YDE-RWENET---I"\n', + encoding="utf-8", + ) + result = await CliRunner().invoke(cli, ["validate", "good.toml"]) + + assert result.exit_code == 0, result.output + + +async def test_validate_reports_a_bad_eic_code() -> None: + """A malformed EIC code fails with a non-zero exit and a readable message.""" + with CliRunner().isolated_filesystem(): + Path("bad.toml").write_text( + "[assets.relations.G80M241L10208446344]\n" + "gridpool_id = 80\n" + "microgrid_id = 241\n" + 'market_location_id = "10208446344"\n' + 'delivery_area.code = "10YDE-RWENET---X"\n', + encoding="utf-8", + ) + result = await CliRunner().invoke(cli, ["validate", "bad.toml"]) + + assert result.exit_code != 0 + assert "valid EIC code" in result.output + + +async def test_validate_rejects_a_partial_file_even_in_a_stack() -> None: + """Each file must stand alone; a partial record is not completed by a merge.""" + with CliRunner().isolated_filesystem(): + Path("base.toml").write_text( + "[assets.relations.G80M241L10208446344]\n" + "gridpool_id = 80\n" + "microgrid_id = 241\n" + 'market_location_id = "10208446344"\n', + encoding="utf-8", + ) + Path("override.toml").write_text( + "[assets.relations.G80M241L10208446344]\n" + 'delivery_area.code = "10YDE-RWENET---I"\n', + encoding="utf-8", + ) + runner = CliRunner() + alone = await runner.invoke(cli, ["validate", "base.toml"]) + stacked = await runner.invoke(cli, ["validate", "base.toml", "override.toml"]) + + assert alone.exit_code != 0, alone.output + assert stacked.exit_code != 0, stacked.output + + +async def test_validate_accepts_a_stack_of_complete_files() -> None: + """Several files, each self-valid, validate together.""" + with CliRunner().isolated_filesystem(): + Path("relation.toml").write_text( + "[assets.relations.G80M241L10208446344]\n" + "gridpool_id = 80\n" + "microgrid_id = 241\n" + 'market_location_id = "10208446344"\n' + 'delivery_area.code = "10YDE-RWENET---I"\n', + encoding="utf-8", + ) + Path("microgrid.toml").write_text( + "assets.microgrids.241.microgrid_id = 241\n", + encoding="utf-8", + ) + result = await CliRunner().invoke( + cli, ["validate", "relation.toml", "microgrid.toml"] + ) + + assert result.exit_code == 0, result.output diff --git a/tests/test_config.py b/tests/test_config.py index 9c4fbd5..ca8bd87 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -14,6 +14,7 @@ from frequenz.gridpool.config import ( AssetsConfig, ComponentTypeConfig, + GridpoolConfig, MicrogridConfig, load_configs, ) @@ -286,6 +287,126 @@ def test_assets_config_rejects_mismatched_id() -> None: AssetsConfig.Schema().load({"microgrids": {"23": {"microgrid_id": 99}}}) +def test_gridpool_names_its_owning_enterprise() -> None: + """A gridpool entry records the enterprise that owns it, keyed by its ID.""" + config = AssetsConfig.Schema().load( + {"gridpools": {"80": {"gridpool_id": 80, "enterprise_id": 42}}} + ) + assert config.gridpools[80] == GridpoolConfig(gridpool_id=80, enterprise_id=42) + assert config.find_enterprise(80) == 42 + + +def test_find_enterprise_unknown_gridpool_is_none() -> None: + """Looking up a gridpool with no entry yields `None`, not an error.""" + assert AssetsConfig().find_enterprise(999) is None + + +def test_gridpool_rejects_mismatched_id() -> None: + """A gridpool filed under the wrong ID is rejected.""" + with pytest.raises(ValueError, match="Gridpool ID mismatch"): + AssetsConfig.Schema().load( + {"gridpools": {"80": {"gridpool_id": 99, "enterprise_id": 42}}} + ) + + +def test_gridpool_requires_an_enterprise() -> None: + """A gridpool must name its enterprise; the invariant is not optional.""" + with pytest.raises(ValidationError, match="enterprise_id"): + AssetsConfig.Schema().load({"gridpools": {"80": {"gridpool_id": 80}}}) + + +def test_find_enterprise_requires_a_gridpool_entry() -> None: + """Related microgrids are not an authoritative source for lookup.""" + config = AssetsConfig.Schema().load( + { + "microgrids": {"10": {"microgrid_id": 10, "enterprise_id": 7}}, + "relations": {"G80M10": {"gridpool_id": 80, "microgrid_id": 10}}, + } + ) + assert config.find_enterprise(80) is None + + +def test_find_enterprise_reads_the_gridpool_entry() -> None: + """The configured gridpool enterprise is authoritative for lookup.""" + config = AssetsConfig.Schema().load( + { + "gridpools": {"80": {"gridpool_id": 80, "enterprise_id": 42}}, + "microgrids": {"10": {"microgrid_id": 10, "enterprise_id": 7}}, + "relations": {"G80M10": {"gridpool_id": 80, "microgrid_id": 10}}, + } + ) + assert config.find_enterprise(80) == 42 + + +def test_find_enterprise_ignores_disagreeing_microgrids() -> None: + """Lookup is independent of whole-document validation.""" + config = AssetsConfig.Schema().load( + { + "gridpools": {"80": {"gridpool_id": 80, "enterprise_id": 7}}, + "microgrids": { + "10": {"microgrid_id": 10, "enterprise_id": 7}, + "11": {"microgrid_id": 11, "enterprise_id": 8}, + }, + "relations": { + "G80M10": {"gridpool_id": 80, "microgrid_id": 10}, + "G80M11": {"gridpool_id": 80, "microgrid_id": 11}, + }, + } + ) + assert config.find_enterprise(80) == 7 + + +def _gridpool_relation(gridpool_id: int, microgrid_id: int) -> dict[str, Any]: + """Return a minimal gridpool-to-microgrid relation with a delivery area.""" + return { + f"G{gridpool_id}M{microgrid_id}": { + "gridpool_id": gridpool_id, + "microgrid_id": microgrid_id, + "delivery_area": {"code": "10YDE-RWENET---I"}, + } + } + + +def test_check_rejects_a_declared_enterprise_the_microgrids_contradict() -> None: + """A `gridpools` enterprise must match what the gridpool's microgrids imply.""" + config = AssetsConfig.Schema().load( + { + "gridpools": {"80": {"gridpool_id": 80, "enterprise_id": 42}}, + "microgrids": {"10": {"microgrid_id": 10, "enterprise_id": 7}}, + "relations": _gridpool_relation(80, 10), + } + ) + with pytest.raises(ValueError, match="declared enterprise 42 disagrees"): + config.check() + + +def test_check_passes_when_declared_matches_the_microgrids() -> None: + """A declared enterprise agreeing with the microgrids is accepted.""" + config = AssetsConfig.Schema().load( + { + "gridpools": {"80": {"gridpool_id": 80, "enterprise_id": 7}}, + "microgrids": {"10": {"microgrid_id": 10, "enterprise_id": 7}}, + "relations": _gridpool_relation(80, 10), + } + ) + config.check() + + +def test_check_rejects_a_gridpool_whose_microgrids_disagree() -> None: + """The one-enterprise invariant is enforced across the whole document.""" + config = AssetsConfig.Schema().load( + { + "microgrids": { + "10": {"microgrid_id": 10, "enterprise_id": 7}, + "11": {"microgrid_id": 11, "enterprise_id": 8}, + }, + "relations": _gridpool_relation(80, 10) | _gridpool_relation(80, 11), + } + ) + with pytest.raises(ValueError, match="disagree on the owning enterprise"): + config.check() + + def test_microgrid_delivery_area_removed() -> None: """Delivery areas live on topology relations, not on a microgrid.""" with pytest.raises(ValidationError, match="Unknown field"): diff --git a/tests/test_dump_config.py b/tests/test_dump_config.py index 9a85edc..f5f245f 100644 --- a/tests/test_dump_config.py +++ b/tests/test_dump_config.py @@ -26,13 +26,16 @@ def test_dump_map_round_trips() -> None: ) } - parsed = tomllib.loads(dump_map(configs)) + parsed = tomllib.loads(dump_map(configs))["assets"] - assert parsed["10"]["microgrid_id"] == 10 - assert parsed["10"]["name"] == "Demo" - assert parsed["10"]["latitude"] == 52.5 - assert parsed["10"]["ctype"]["pv"]["meter"] == [2] - assert parsed["10"]["ctype"]["pv"]["formula"]["AC_POWER_ACTIVE"] == "#2" + assert parsed["version"] == 1 + assert parsed["microgrids"]["10"]["microgrid_id"] == 10 + assert parsed["microgrids"]["10"]["name"] == "Demo" + assert parsed["microgrids"]["10"]["latitude"] == 52.5 + assert parsed["microgrids"]["10"]["ctype"]["pv"]["meter"] == [2] + assert ( + parsed["microgrids"]["10"]["ctype"]["pv"]["formula"]["AC_POWER_ACTIVE"] == "#2" + ) def test_dump_map_omits_empty_and_none() -> None: @@ -41,12 +44,12 @@ def test_dump_map_omits_empty_and_none() -> None: text = dump_map(configs) - assert text == "7.microgrid_id = 7\n" + assert text == "assets.version = 1\n\nassets.microgrids.7.microgrid_id = 7\n" def test_dump_map_empty() -> None: - """An empty mapping serializes to an empty string.""" - assert dump_map({}) == "" + """An empty mapping serializes to just the version stamp.""" + assert dump_map({}) == "assets.version = 1\n" def test_dump_map_renders_whole_floats_as_underscored_ints() -> None: @@ -61,7 +64,7 @@ def test_dump_map_renders_whole_floats_as_underscored_ints() -> None: text = dump_map(configs) - assert "10.pv.1.peak_power = 1_736_680\n" in text - assert "10.pv.1.rated_power = 1_400_000\n" in text + assert "assets.microgrids.10.pv.1.peak_power = 1_736_680\n" in text + assert "assets.microgrids.10.pv.1.rated_power = 1_400_000\n" in text # Genuinely fractional floats are left alone. - assert "10.latitude = 52.5\n" in text + assert "assets.microgrids.10.latitude = 52.5\n" in text diff --git a/tests/test_load.py b/tests/test_load.py index 27615eb..5548acb 100644 --- a/tests/test_load.py +++ b/tests/test_load.py @@ -32,7 +32,9 @@ def _mock_client() -> MagicMock: """Mock an Assets API client for one microgrid: grid -> meter -> PV, + a meter.""" client = MagicMock(spec=AssetsApiClient) - client.get_microgrid = AsyncMock(return_value=MagicMock(location=None)) + client.get_microgrid = AsyncMock( + return_value=MagicMock(location=None, enterprise_id=7) + ) client.list_microgrid_electrical_components = AsyncMock( return_value=[ GridConnectionPoint( @@ -70,6 +72,7 @@ async def test_load_microgrids_from_api_derives_formulas_and_ids() -> None: configs = await _load_microgrids_from_api(_mock_client(), [10]) cfg = configs[10] + assert cfg.enterprise_id == 7 assert cfg.ctype["pv"].formula == {"AC_POWER_ACTIVE": "COALESCE(#4, #2, 0.0)"} assert cfg.ctype["pv"].inverter == [4] assert cfg.ctype["pv"].meter == [2] @@ -78,6 +81,18 @@ async def test_load_microgrids_from_api_derives_formulas_and_ids() -> None: assert set(cfg.ctype) == {"grid", "consumption", "pv"} +async def test_load_microgrids_from_api_allows_missing_enterprise_id() -> None: + """An unassigned enterprise (proto default 0) remains unset.""" + client = _mock_client() + client.get_microgrid = AsyncMock( + return_value=MagicMock(location=None, enterprise_id=0) + ) + + configs = await _load_microgrids_from_api(client, [10]) + + assert configs[10].enterprise_id is None + + async def test_load_microgrids_from_api_honours_the_component_graph_config() -> None: """A component graph config reaches the derived formulas.""" configs = await _load_microgrids_from_api( diff --git a/tests/test_patch_config.py b/tests/test_patch_config.py index 89ed162..0d8d708 100644 --- a/tests/test_patch_config.py +++ b/tests/test_patch_config.py @@ -3,6 +3,8 @@ """Tests for in-place patching of existing dotted-key TOML config files.""" +import pytest + from frequenz.gridpool.cli._patch_config import patch_text from frequenz.gridpool.config import ( ComponentTypeConfig, @@ -12,20 +14,20 @@ _ORIGINAL = """# EID 6 TOML configuration -40.name = "Bona - Auf dem Aurain" #grid_side True -40.gid = 6 -40.enterprise_id = 6 -40.microgrid_id = 40 -40.latitude = 50.39567065 -40.longitude = 8.083947042665976 -40.ctype.grid.meter = [87] -40.pv.1.peak_power = 616_140 -40.pv.1.rated_power = 480_000 # https://example.com/technischedaten +assets.microgrids.40.name = "Bona - Auf dem Aurain" #grid_side True +assets.microgrids.40.gid = 6 +assets.microgrids.40.enterprise_id = 6 +assets.microgrids.40.microgrid_id = 40 +assets.microgrids.40.latitude = 50.39567065 +assets.microgrids.40.longitude = 8.083947042665976 +assets.microgrids.40.ctype.grid.meter = [87] +assets.microgrids.40.pv.1.peak_power = 616_140 +assets.microgrids.40.pv.1.rated_power = 480_000 # https://example.com/technischedaten """ -def test_patch_is_a_noop_when_nothing_changed() -> None: - """Patching with values already on disk leaves the file byte-identical.""" +def test_fill_missing_is_a_noop_when_nothing_changed() -> None: + """With `fill_missing`, values already on disk leave the file byte-identical.""" configs = { 40: MicrogridConfig( microgrid_id=40, @@ -34,7 +36,31 @@ def test_patch_is_a_noop_when_nothing_changed() -> None: ) } - assert patch_text(_ORIGINAL, configs) == _ORIGINAL + assert patch_text(_ORIGINAL, configs, fill_missing=True) == _ORIGINAL + + +def test_patch_overwrites_existing_leaf_by_default() -> None: + """By default a managed leaf already on disk is refreshed to the new value.""" + configs = { + 40: MicrogridConfig(microgrid_id=40, pv={"1": PVConfig(rated_power=500_000.0)}) + } + + patched = patch_text(_ORIGINAL, configs) + + assert "assets.microgrids.40.pv.1.rated_power = 500_000" in patched + assert "480_000" not in patched + + +def test_fill_missing_keeps_an_existing_leaf() -> None: + """With `fill_missing`, an existing leaf keeps its on-disk value.""" + configs = { + 40: MicrogridConfig(microgrid_id=40, pv={"1": PVConfig(rated_power=500_000.0)}) + } + + patched = patch_text(_ORIGINAL, configs, fill_missing=True) + + assert "assets.microgrids.40.pv.1.rated_power = 480_000" in patched + assert "500_000" not in patched def test_patch_inserts_missing_leaf_next_to_existing_table() -> None: @@ -48,10 +74,11 @@ def test_patch_inserts_missing_leaf_next_to_existing_table() -> None: lines = patched.splitlines() assert 'name = "Bona - Auf dem Aurain" #grid_side True' in lines[2] # A genuinely fractional value is left alone. - assert lines[3] == "40.altitude = 45.5" + assert lines[3] == "assets.microgrids.40.altitude = 45.5" # Untouched lines are unchanged, including comments. assert ( - "40.pv.1.rated_power = 480_000 # https://example.com/technischedaten" in patched + "assets.microgrids.40.pv.1.rated_power = 480_000 # " + "https://example.com/technischedaten" in patched ) assert "# EID 6 TOML configuration" in patched @@ -66,7 +93,8 @@ def test_patch_appends_new_microgrid_at_the_end() -> None: assert patched.startswith(_ORIGINAL) assert patched[len(_ORIGINAL) :] == ( - '\n9999.microgrid_id = 9_999\n9999.name = "Brand New"\n' + "\nassets.microgrids.9999.microgrid_id = 9_999\n" + 'assets.microgrids.9999.name = "Brand New"\n' ) @@ -81,12 +109,15 @@ def test_patch_inserts_new_subtable_next_to_its_microgrid() -> None: patched = patch_text(_ORIGINAL, configs) - assert patched == _ORIGINAL + "40.pv.2.peak_power = 50_000\n" + assert patched == _ORIGINAL + "assets.microgrids.40.pv.2.peak_power = 50_000\n" def test_patch_inserts_new_subtables_for_multiple_microgrids() -> None: """Each microgrid's new sub-table lands next to its own lines, not all at the end.""" - original = _ORIGINAL + '\n41.name = "Other Grid"\n41.microgrid_id = 41\n' + original = _ORIGINAL + ( + '\nassets.microgrids.41.name = "Other Grid"\n' + "assets.microgrids.41.microgrid_id = 41\n" + ) configs = { 40: MicrogridConfig( microgrid_id=40, @@ -103,11 +134,28 @@ def test_patch_inserts_new_subtables_for_multiple_microgrids() -> None: lines = patched.splitlines() assert lines[ lines.index( - "40.pv.1.rated_power = 480_000 # https://example.com/technischedaten" + "assets.microgrids.40.pv.1.rated_power = 480_000 # " + "https://example.com/technischedaten" ) + 1 - ] == ("40.pv.2.peak_power = 50_000") - assert lines[-1] == "41.ctype.grid.meter = [1]" + ] == ("assets.microgrids.40.pv.2.peak_power = 50_000") + assert lines[-1] == "assets.microgrids.41.ctype.grid.meter = [1]" + + +def test_patch_refuses_top_level_layout() -> None: + """A deprecated top-level file is refused, not duplicated into the new layout.""" + with pytest.raises(ValueError, match="top-level"): + patch_text("1.microgrid_id = 1\n", {1: MicrogridConfig(microgrid_id=1)}) + + +def test_patch_refuses_meta_layout() -> None: + """A file nesting fields under `meta` is refused rather than patched alongside it.""" + original = ( + "assets.microgrids.1.microgrid_id = 1\n" + 'assets.microgrids.1.meta.name = "Old"\n' + ) + with pytest.raises(ValueError, match="meta"): + patch_text(original, {1: MicrogridConfig(microgrid_id=1)}) def test_patch_formats_new_numeric_leaves() -> None: @@ -118,4 +166,4 @@ def test_patch_formats_new_numeric_leaves() -> None: patched = patch_text(_ORIGINAL, configs) - assert "5555.enterprise_id = 1_234_567\n" in patched + assert "assets.microgrids.5555.enterprise_id = 1_234_567\n" in patched