Repository navigation
Feat: copy automation button - #2591
Conversation
…ne type Context: - The listing returned every data source the caller may read, with no way to search it or to ask for one kind. A picker offering a choice of forecasters therefore had to fetch them all and sift through them in the browser. Change: - Add a `filter` of space-separated search terms, matched against a source's name, its model and its id prefix, the way the sensors listing already searches, and a `type`, which narrows the listing to one source type. - Move the rule about which sources a user may read into the data-sources service, as `get_readable_source_account_ids` and `user_may_read_source`, so that it can be applied outside this endpoint without being restated. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
…nder Context: - A data source stores a data generator and the configuration it runs under, and the results computed under it are recorded on it. `flexmeasures add automation` has taken `--source` since the command was written, so several automations can share one generator and one lineage of data, but the API could only be handed a class and a config, which reaches an existing source by accident, when what it builds happens to hash to the same attributes. Change: - Accept a `source` on the creation endpoint, deserialized to the DataSource the service already knows how to take. It cannot be combined with `data-generator` or `config`, which the source itself determines, and a schedule automation cannot name one, since it re-resolves its source from the asset and the flex config on every run and would move off the named one at the next run. - Require that the caller may read the source, by the same rule the sources listing applies. A source is otherwise a way to compute under, and record on, a generator configuration belonging to another organisation. The CLI runs without a user and stays trusted, as it already is for sensors. - Let DataSourceIdField dump None, so that the field's absent default serializes rather than failing the OpenAPI generation. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
…tores Context: - The creation form asked for a data generator class and a config as free text, with no way to say "the one that automation already uses", and nothing showed what a generator would be configured with until after the automation existed. Change: - Add a search box above those two fields, which lists the data sources of the automation's own type, matched by id, by name and by generator class, and lists them on focus so they can be browsed without guessing a term. - Show the picked source's description and the configuration it stores, and disable the class and config fields while it is picked, since the source already determines both. Clearing it hands them back. - Drop the whole section for a schedule automation, which has no generator to name, and clear a selection when the type changes, as the source type does. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
The Automations page could create an automation from scratch or edit one's recurrence, but a variation on an existing automation, such as the same forecast on a different schedule, had to be typed out again from its details. Each row's Actions menu now offers Copy, which opens the creation form with the original's type, recurrence, timezone and parameters filled in. Nothing is created until the form is submitted, so the copy is a starting point that can still be changed, rather than a command of its own. The copy is filled in inactive, so that a variation being worked on does not start running while it is still being edited, and so that two automations writing the same results never appear from a single click. Its name carries a (copy) suffix, cut to the 80 characters the field and the API allow, so that a long name loses its tail rather than the suffix that marks it. A forecast or report copy reuses the data source the original computes under, which already stores the data generator and its configuration, rather than naming that class and config again. A schedule automation resolves its source afresh on every run and cannot name one, so a copy of one names none. The parameters are carried over exactly as stored, which is safe here because the copy stays on the same asset, so the sensors they name stay valid and the creation endpoint checks them against the user's permissions as it would for any new automation. The creation form is now shared between New automation and Copy, so opening it blank clears what a copy left in it, rather than inheriting the last copy's settings. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Context: - #2554 rebuilt the same creation form this branch extends, and #2565 renamed flexmeasures/api/dev to flexmeasures/api/ui. Change: - Take main's creation form and re-apply the picker onto it: the picker joins the .chooses-generator fields rather than carrying its own wrapper, clearing a selection folds into main's type-change handler, and the submit path goes through main's readJsonField. - Assemble the POST payload before sending it, since a reused source replaces both the generator and the config, and follow test_asset_crud's payload assertions onto the new lines, adding one for the source branch. - Keep both sides' tests in test_automations_api, restoring the two parametrize decorators the merge separated from their functions. - Take v3.0-39 in the API change log, as main claimed v3.0-38. - Share test_automation_actions.automation_script in the picker's browser checks, which renders all five of the page's Jinja values rather than three. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
…tton The base branch merged main, which brought in the rebuilt creation form: the generator fields are now marked `.chooses-generator` and shown by type, where they used to sit in an `#automationGeneratorSection` toggled by `applyAutomationTypeToSourcePicker`, which no longer exists. The copy button filled the form in through that helper, so it needed rewiring rather than a textual conflict resolution: - `typeChoosesGenerator` and `showGeneratorFields` move from inside the ready callback to module scope, beside `clearSelectedSource`, which the ready block already calls the same way. Filling the form in from an existing automation applies the same type rule, so it is stated once rather than copied. - Resetting and prefilling the form now clear the selected source and show the generator fields for the chosen type, instead of toggling a wrapper. - The copy checks share `automation_script` from the action tests, which renders every Jinja value the page reads and asserts that none is left behind. The merge added two such values, which the file's own renderer would have passed through to the browser as a syntax error. - The checks cover the new mechanism: a schedule copy hides the generator fields, a report copy keeps them, and opening a blank form brings them back. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
…er' into feat/copy-automation-button
Automations have not shipped, so the changelog says what the feature is rather than how it got there: the entry that already covers creating and managing automations in the UI names Copy among the row's controls and carries this PR's number, instead of a bullet of its own. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Documentation build overview
46 files changed ·
|
… covers it The automations feature has not shipped yet, so the changelog describes where it ends up rather than the steps it took to get there. Someone reading the v1.1.0 notes never saw a version of the *New automation* form without a data source to pick, and a bullet announcing that one arrived tells them about a past they have no memory of. The per-PR list under *Automations, in detail* is the place where a contribution is still worth naming, so this PR's number joins the entry that already describes the creation form and the data source it records under. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
…es it ``POST /assets/<id>/automations`` arrived at v3.0-37, inside this same unreleased cycle, so no API consumer has ever called it without a ``source``. The release will simply introduce the endpoint, with that field on it, and an entry saying it "now accepts" one describes a revision nobody integrated against. The ``GET /sources`` entry stays. That listing predates this cycle and is already in users' hands, so a new way to search and narrow it is a change they need told about. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
…mplies The field's description had grown into a paragraph, restating what a data source stores, how the results are attributed to it, and why a schedule automation may not name one. All three are already in the automations documentation, where they have room to be explained, and none of them help someone reading the OpenAPI reference to decide what to put in the field. One sentence is left: it is the ID of a data source to reuse, in place of naming a generator and a config. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
…aSourceIdField alone Giving the field a ``load_default`` of None put a default into the OpenAPI spec, which the generator then pushed back through the field's own serializer. A field that turns an ID into a DataSource has nothing to do with None, so it raised, and the way out taken at the time was to teach it to swallow None -- changing the field for every schema that uses it, to accommodate one field's setup. The field is simply not required, so say that instead. Nothing then asks for a default, the serializer goes back to doing one thing, and the published schema improves as well: ``source`` is now an integer that may be omitted, rather than a nullable integer defaulting to null. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
The API changelog entries of the two lines each keep a section of their own, and the page entry is main's, which now covers what both say. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0129WrXeJ5gia2pctFH93BqC Signed-off-by: F.N. Claessen <claessen@seita.nl>
Main asks the server about one automation only when its panel is opened (#2299), where this branch's copy button expected every row to have been read on page load, and would have said its details were still loading. A copy now reads that automation itself when the panel has not been opened. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0129WrXeJ5gia2pctFH93BqC Signed-off-by: F.N. Claessen <claessen@seita.nl>
…ll request Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0129WrXeJ5gia2pctFH93BqC Signed-off-by: F.N. Claessen <claessen@seita.nl>
The creation form carried nine blocks of help text, several of them three to five lines, so the fields it was meant to explain were spread thin between paragraphs and the modal read as a page of prose with inputs in it. Each field's label now carries the info icon the sensor creation form already uses, with the same text behind it on hover. Two hints stay on the page, because they are the ones you act on while typing rather than read once: that the generator config can be left empty, and the offset example worth copying into the parameters. Bootstrap only turns those icons into tooltips when asked, so the page now initialises them, as the chart pages do. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Bootstrap appends a tooltip to the document body, where it sits at a z-index of 1080. The theme gives a modal 999993. A tooltip opened from a field inside the creation form was therefore built, shown and positioned correctly, and painted behind the modal, so hovering an info icon appeared to do nothing at all. Anchoring each tooltip to the modal that holds its icon puts the two in one stacking context, and the tooltip comes out on top. Icons outside a modal keep the body as their container. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
…eps jobs The jobs column counts what the job cache returns, and that cache holds only the jobs Redis still has: FLEXMEASURES_JOB_TTL expires them, and the cache drops the ids it can no longer fetch. A daily automation two weeks old therefore shows a week of jobs on an instance that keeps them for a week, which reads as runs that never happened rather than as records that have been cleaned up. The column and the info panel now say so, naming the window this instance is configured for rather than pointing at the setting. The phrasing works for either shape the value takes, since a one-day retention reads as "a day" and not "1 day". The browser checks render the page's script through a bare Jinja environment, so that environment now carries the filter and the configuration value the page reads, keeping its promise that nothing is left unrendered. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <149331360+BelhsanHmida@users.noreply.github.com>
The base moved the creation form's reference text onto info icons and raised their tooltips above the modal. Copy fills the same fields by id and reads the same helpers, so nothing here needed adapting this time: the merge is a plain sync, and the copy checks still bind against the merged form. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
…pes unnarrowed A search from the automation form asked for every source a user may read, in whatever order Postgres returned them, so an instance with many sources sent its whole source table to the browser, and the sources shown could differ between two searches for the same term. The listing is now ordered with the most recently created source first, and takes a limit. The source types in the response are read from every source the user may read, rather than from the sources the call returns, so that narrowing the listing no longer narrows the types a client can offer to narrow it by. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0129WrXeJ5gia2pctFH93BqC Signed-off-by: F.N. Claessen <claessen@seita.nl>
The search now asks for one source more than the list shows, which is what tells the list that there are more to narrow down to, and which keeps an instance with many sources from sending all of them to the browser on every pause in typing. The list therefore says that it leaves sources out rather than how many, which is no longer a number it has. A source picked and then left behind by closing the form used to still disable the data generator fields when the form was opened again, with nothing on screen to say why, so closing the form now clears the picked source. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0129WrXeJ5gia2pctFH93BqC Signed-off-by: F.N. Claessen <claessen@seita.nl>
A client which fills in the whole creation form and leaves the source field empty sends a null source, which was refused with 'Field may not be null' rather than taken as the automation setting up its own data generator. Our own form works around this by leaving the field out altogether. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0129WrXeJ5gia2pctFH93BqC Signed-off-by: F.N. Claessen <claessen@seita.nl>
…urces Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0129WrXeJ5gia2pctFH93BqC Signed-off-by: F.N. Claessen <claessen@seita.nl>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0129WrXeJ5gia2pctFH93BqC Signed-off-by: F.N. Claessen <claessen@seita.nl>
…nisation alone Almost every data generator's source belongs to no organisation, because nothing on the creation path passes one, and a source naming neither an organisation nor a user was readable by every authenticated user. The read check this pull request added therefore hardly ever refused anything. A source is now readable when it belongs to an organisation the user may read, when an automation they may read computes under it, or when it has recorded data on a sensor they may read, which is what makes it possible to ask what produced a number one is allowed to see. The first two are what makes a source one to work with: those are the sources the listing holds, because reusing a source means recording under its lineage, which is more than reading what it computed. The stored configuration follows the stricter rule as well, since a data generator's configuration names the sensors it runs on, which can be sensors of an organisation whose data the user cannot see at all. Two tests created sources belonging to no organisation to exercise version collapsing rather than visibility; their fixtures name an organisation now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0129WrXeJ5gia2pctFH93BqC Signed-off-by: F.N. Claessen <claessen@seita.nl>
…o work with Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0129WrXeJ5gia2pctFH93BqC Signed-off-by: F.N. Claessen <claessen@seita.nl>
…ad it The listing a source is picked from holds the sources the user may work with, while reading one source is allowed more widely, for a source which recorded data on a sensor they may read. Creating the automation checked the wider of the two, so a source that had recorded on one of the user's own sensors could be named for an automation of theirs, which records under that source and so writes into another organisation's lineage. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0129WrXeJ5gia2pctFH93BqC Signed-off-by: F.N. Claessen <claessen@seita.nl>
…omment Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0129WrXeJ5gia2pctFH93BqC Signed-off-by: F.N. Claessen <claessen@seita.nl>
|
@BelhsanHmida apologies — I closed this by accident about an hour ago, with no word to you, and I have reopened it. What happened: this PR targets What still needs doing, and I am happy to do it if you would rather not: #2584 was squash-merged, so One thing from #2584 that touches this PR: the automation listing no longer pre-loads each row's details, so the copy button has to fetch the source it copies from on demand. That is what Sorry for the noise. 🤖 Generated with Claude Code |
…er' into feat/copy-automation-button
…y-automation-button
The merge of main brought in the entry PR #2563 had extended, beside this branch's own edit of the same line. Neither conflicted with the other, so both survived: the section carried the same 900-character bullet twice, differing only in whether it mentioned Copy and which pull requests it cited. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0129WrXeJ5gia2pctFH93BqC Signed-off-by: F.N. Claessen <claessen@seita.nl>
Flix6x
left a comment
There was a problem hiding this comment.
Reviewed the restacked branch. The feature itself reads well and its tests are the right ones; I found one thing the restack left behind, fixed it on the branch, and there is one claim in the description that no longer holds.
Fixed in edec10f: the changelog carried the same bullet twice. Merging main brought in the entry #2563 had extended, beside this branch's own edit of the same line. Neither conflicted with the other, so both survived — two near-identical 900-character bullets, differing only in whether they mention Copy and which PRs they cite (main has one). I merged them into a single entry citing #2294, #2563 and #2591. This is the one failure mode a -X ours restack cannot catch, a non-conflicting double-add, so it is worth knowing to grep for after any restack. I checked for others of the same kind on this branch — duplicated JS functions, duplicated test names, duplicated element ids — and found none.
The description is now wrong on one point. It says "There is no extra request either — loadAutomationDetails already fetches every row's details on page load and discarded the response; it is now cached". Since #2299 the page asks about an automation only when its panel is opened, so for a row whose panel was never opened there is one extra request. The code is right — the handler fetches on demand, caches, and the comment beside it says exactly that — it is only the description that predates the rebase. Worth correcting so the next reader is not looking for a cache that is not the point.
DCO is red, on two merge commits from the restack (110ed1eb8, 94d427343) that carry no sign-off. That wants a maintainer override rather than a history rewrite, since rewriting would throw away the restack and the review history with it.
What I checked and found sound:
- A schedule copy names no source:
SOURCE_TYPE_PER_AUTOMATION_TYPE["scheduling"]is undefined, sosourceIdfalls to null rather than being refused later byvalidate_generator_is_named_once. - The name truncation leaves room for the suffix against the API's own limit (
validate.Length(max=80)onname), so a long name loses its tail rather than the marker. - The failure path when the source cannot be read tells the user what will happen otherwise, instead of silently falling back to a default generator — that is the right call, and unusual enough to be worth praising.
- 136 JavaScript checks pass on the branch, including the four new ones. They cover what the feature promises: settings carried over, inactive by default, a source only where one can be named, the long-name case, and a blank form clearing what a copy left behind.
One thing I did not test, and nor does the suite: the copy carries parameters exactly as stored, so an automation created before the window work — one whose parameters still hold a fixed start or end — would produce a copy that the creation endpoint refuses with a 422 from refuse_fixed_moments. The user sees the error and can edit the field, which seems the right outcome, but if any such automations exist in the wild it is worth a sentence in the docs rather than a surprise.
🤖 Generated with Claude Code
Flix6x
left a comment
There was a problem hiding this comment.
Approving, with the changelog duplicate fixed on the branch and one sentence in the description corrected — it claimed the copy costs no extra request, which stopped being true when #2299 made the details panel load on demand. The code was always right; I corrected the description because a squash merge carries it into the history.
The feature does what it says, its four JavaScript checks cover the cases that matter, and the whole UI suite passes on the branch. A schedule copy names no source, a long name keeps the suffix that marks it as a copy, and a source that cannot be read says what will happen instead of silently falling back.
Description
The Automations page could create an automation from scratch, or edit an existing one's name, recurrence, timezone and activation. It could not start from an automation that already exists. A variation on one — the same forecast on a different recurrence, the same reporter over a different period — had to be read off the details modal and typed back into the creation form by hand.
Each row's Actions menu now offers Copy, which opens the creation form with that automation's settings filled in.
(copy)suffix, cut to the 80 characters the field and the API allow, so a long name loses its tail rather than the suffix that marks it as a copy.sourcefield, used from the page itself.validate_generator_is_named_once), so a copy of one names none.This is entirely client-side, and adds no API change of its own: the submit posts the
sourcefield that #2584 adds, and nothing else is new. The page asks about one automation only when its panel is opened, so a copy of a row whose panel was never opened reads that automation once, and caches it; a second copy of the same row costs nothing.Prefilling Data generator and Data generator config instead would work too, and would not need #2584:
get_or_create_sourcematches on the attributes hash, so an unchanged copy lands on the original's source without naming it. On this base the picker is already in the form, so Copy fills it in rather than leaving it blank, and the source id says what is meant rather than arriving there by a hash match. Discussed on #2588.A side effect worth naming: an automation's parameters cannot be edited after creation, by design, so that the sensors it involves stay the ones its creator was checked against. Copy-edit-delete is now a supported way to arrive at the same place, with the new automation's sensors checked against whoever makes the copy.
documentation/changelog.rst, under Automations, in detail.The second commit merges #2584's rebuilt creation form. That merge is not a formality: the generator fields moved from an
#automationGeneratorSectionwrapper to.chooses-generatorfields shown by type, andapplyAutomationTypeToSourcePickerno longer exists, so the prefill had to be rewired onto the new mechanism.typeChoosesGeneratorandshowGeneratorFieldsmove to module scope besideclearSelectedSource, so that filling the form in from an existing automation applies the same type rule rather than restating it.Look & Feel
Actions → Copy opens the creation form, already filled in.
A forecast copy: name suffixed, recurrence and parameters carried over, the original's data source preselected — so Data generator and config are disabled, as whenever a source is picked — and Active unticked.
A report copy keeps its reporter's source, which a report automation requires.
A schedule copy names no source and the generator fields are gone: a schedule automation resolves its own on every run.
The form is shared with New automation, so opening it blank clears what a copy left in it. The grey text is placeholder, not carried-over values.
A long name is cut to 73 characters before the
(copy)suffix. If the source cannot be read, the form still opens and says so rather than silently falling back to a default generator.How to test
flexmeasures/ui/tests/js/test_automation_copy.py:test_a_copy_carries_the_settings_over_and_starts_inactivetest_a_copy_only_reuses_a_data_source_where_one_can_be_namedtest_a_long_name_keeps_the_suffix_that_marks_the_copytest_opening_a_blank_form_clears_what_a_copy_left_in_ittest_copy_is_offered_to_managers_onlyFurther Improvements
Related Items
sourcefield this uses.Sign-off