Skip to content

evetest: list parameters declared through a helper - #6600

Merged
milan-zededa merged 2 commits into
lf-edge:masterfrom
eriknordmark:evetest-listtests-params
Sep 28, 2026
Merged

milan-zededa merged 2 commits into
lf-edge:masterfrom
eriknordmark:evetest-listtests-params

Conversation

@eriknordmark

@eriknordmark eriknordmark commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Description

make list-tests is where a user learns what a test can be configured
with. A test whose DefineTestParameters call goes through a
package-local helper advertised nothing at all. The ten conversion tests
under evetest/tests/konvert are the only ones in the tree that declare
parameters that way, and every one of their rows was empty, so the axes
they accept -- the released version the device starts on, which way a
repartition must free space, why a conversion must be refused -- were
invisible to anyone reading the listing.

The first commit teaches list-tests to follow a call from a test into a
package-local function, binding the definitions the call site passes for
that function's formal parameters, and to resolve a definition list
through local variables, append, slice literals, and functions
returning a slice of definitions. The output diff against master is
confined to TestKonvertSuite and those ten tests; every other row is
byte-identical.

The second commit concerns one displayed default. RAM_SIZE_MB and
CPUS were shown as resolving to the framework's 8192 MiB and 4 vCPUs
when left unset, while a test is free to raise an unset value to a floor
of its own -- which the doc comment on each constructor already says.
Naming the floor keeps the string a user sees from promising a number
that need not hold.

The two commits are independent; neither depends on the other.

Introduced in #6539.

How to test and validate this PR

Static analysis only -- no EVE build and no device are involved, and
nothing a test does at run time changes.

cd evetest && make list-tests

Every konvert test now carries its parameters, e.g.

TestKvmToKRepartition  [EVETEST_EVE_VERSION(string), default: HEAD of the checked-out EVE repo …;
  EVETEST_HYPERVISOR…; EVETEST_TPM…; EVETEST_DISK_SIZE_MB…; EVETEST_RAM_SIZE_MB…; EVETEST_CPUS…;
  EVETEST_INITIAL_EVE_VERSION, default: 10.1.0; EVETEST_INITIAL_EVE_REPO…; EVETEST_INITIAL_HYPERVISOR…;
  EVETEST_EXPECT_DECISION(shrink|grow), default: shrink; EVETEST_FILL_PERSIST_GIB, default: 33]

and each set matches that test's own call site: the cross-flavor tests
get ALT_EVE_VERSION, the refused test REFUSE_REASON.

Covered by an automated test: GOWORK=off go test ./cmd/list-tests/
runs the resolver over a fixture package shaped like tests/konvert
(helper indirection, variadic pass-through, append, and a
slice-returning function). Against the tool as it stands on master that
test fails with got 0 parameters [], want 3.

Changelog notes

No user-facing changes.

PR Backports

  • 17.0-stable: No. list-tests exists there, but the branch has neither
    the tests/konvert package nor the RAM_SIZE_MB / CPUS parameters,
    so neither commit has a subject on it.
  • 16.0-stable: No, evetest is not on that branch.
  • 14.5-stable: No, evetest is not on that branch.
  • 13.4-stable: No, evetest is not on that branch.

Checklist

  • I've provided a proper description
  • I've added the proper documentation
  • I've tested my PR on amd64 device
  • I've tested my PR on arm64 device
  • I've written the test verification instructions
  • I've set the proper labels to this PR

And the last but not least:

  • I've checked the boxes above, or I've provided a good reason why I didn't
    check them.

No device was used because none can be: the change is a static-analysis
CLI plus one description string, neither of which runs on a device. The
README already documents make list-tests and the parameter mechanism,
and this changes neither.

🤖 Generated with Claude Code

@eriknordmark
eriknordmark marked this pull request as ready for review September 18, 2026 22:51
eriknordmark and others added 2 commits September 18, 2026 16:05
A test whose DefineTestParameters call goes through a package-local
helper advertised no parameters at all in `make list-tests`. The ten
conversion tests under tests/konvert are the only ones in the tree that
declare them that way, and every one of their rows was empty, so the
axes they accept -- the released version the device starts on, which
way a repartition must free space, why a conversion must be refused --
were invisible to anyone reading the listing.

list-tests now follows a call from a test into a package-local
function, binding the definitions the call site passes for that
function's formal parameters, and resolves a definition list through
local variables, append, slice literals, and functions returning a
slice of definitions.

Signed-off-by: eriknordmark <erik@zededa.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The listing told the reader that RAM_SIZE_MB and CPUS resolve to the
framework's 8192 MiB and 4 vCPUs when left unset, while a test is free
to raise an unset value to a floor of its own -- which the doc comment
on each constructor already says. Naming the floor in the displayed
default keeps the one string a user sees from promising a number that
need not hold.

Signed-off-by: eriknordmark <erik@zededa.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@eriknordmark
eriknordmark force-pushed the evetest-listtests-params branch from 7c243cc to 9bcd5b1 Compare September 18, 2026 23:14
@rene rene added the testing New test or test framework changes label Sep 21, 2026

@rene rene left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codecov

codecov Bot commented Sep 27, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 26.56%. Comparing base (8272a00) to head (9bcd5b1).
⚠️ Report is 118 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6600      +/-   ##
==========================================
+ Coverage   26.24%   26.56%   +0.31%     
==========================================
  Files         515      525      +10     
  Lines       94872    96584    +1712     
==========================================
+ Hits        24902    25655     +753     
- Misses      68046    68821     +775     
- Partials     1924     2108     +184     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@milan-zededa
milan-zededa merged commit e5ddc76 into lf-edge:master Sep 28, 2026
53 of 56 checks passed
@eriknordmark
eriknordmark deleted the evetest-listtests-params branch October 8, 2026 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing New test or test framework changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants