Skip to content

Update URM Test Runner #568

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Srikanth Muppandam (smuppand) merged 1 commit into from
Sep 11, 2026
+108 −11
Merged

Update URM Test Runner #568

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 27 additions & 5 deletions Runner/suites/Performance/userspace-resource-manager/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,17 @@ export URM_REQUIRE_COMMON_FILES="InitConfig.yaml PropertiesConfig.yaml Resources
export URM_REQUIRE_TEST_FILES="InitConfig.yaml PropertiesConfig.yaml ResourcesConfig.yaml SignalsConfig.yaml TargetConfig.yaml ExtFeaturesConfig.yaml Baseline.yaml"
```

### 3) Test test nodes
`/etc/urm/tests/nodes` must exist and be non‑empty for **`/usr/bin/UrmIntegrationTests`** and **`/usr/bin/UrmComponentTests`**. If missing/empty → **SKIP only that suite**.
### 3) Test nodes
The runner resolves the test-nodes source directory in the following priority order:

1. **`URM_TEST_NODES_DIR`** – explicit operator override (takes precedence over everything).
2. **`URM_CONFIG_DIR`** – legacy compatibility fallback (same variable honoured by the config roots).
3. **`/var/lib/urm/tests/nodes`** – runtime/writable location, used only when the directory exists **and is non-empty**.
4. **`/usr/share/urm/tests/nodes`** – package-installed default (Yocto and Debian).

Before tests run, the resolved nodes are **copied** into a private temporary directory created with `mktemp -d`, under the path `<tmpdir>/urm/tests/nodes`. This path is passed to each test binary via the `--npath` argument, so the binaries write to a fresh exclusively-owned location rather than a shared fixed path. The temporary directory is removed automatically on exit, interrupt, or termination.

The resolved source directory must exist and be non-empty for **`/usr/bin/UrmIntegrationTests`** and **`/usr/bin/UrmComponentTests`**. If missing, empty, or the copy fails → **SKIP only that suite**.

### 4) Base tools
Requires: `awk`, `grep`, `date`, `printf`. If missing → **overall SKIP**.
Expand Down Expand Up @@ -100,8 +109,11 @@ Per‑suite default timeouts (if helper is present):
## Environment overrides

- `SERVICE_NAME`: systemd unit to check (default: `urm.service`)
- `URM_CONFIG_DIR`: root of config tree (default: `/etc/urm`)
- `URM_REQUIRE_COMMON_FILES`, `URM_REQUIRE_TEST_FILES`: *space‑separated* filenames that must exist in `common/` / `tests/` respectively to treat that tree as present.
- `URM_CONFIG_DIR`: **legacy** compatibility override – sets the root for `common/`, `tests/configs/`, and `tests/nodes/` when the per-root variables below are not set (default: unset; built-in defaults are `/etc/urm`, `/usr/share/urm`, and `/usr/share/urm` respectively).
- `URM_COMMON_CONFIG_DIR`: root for `common/` configs (default: `/etc/urm`). Takes precedence over `URM_CONFIG_DIR`.
- `URM_TESTS_CONFIG_DIR`: root for `tests/configs/` (default: `/usr/share/urm`). Takes precedence over `URM_CONFIG_DIR`.
- `URM_TEST_NODES_DIR`: explicit root for `tests/nodes/` (default: auto-resolved; see [Test nodes](#3-test-nodes) above). Takes precedence over the automatic candidate search.
- `URM_REQUIRE_COMMON_FILES`, `URM_REQUIRE_TEST_FILES`: *space‑separated* filenames that must exist in `common/` / `tests/configs/` respectively to treat that tree as present.

---

Expand All @@ -122,11 +134,21 @@ List suites and presence coverage:
./run.sh --list
```

Use a different config root:
Use a different config root (legacy, applies to both common and tests-config roots):
```bash
URM_CONFIG_DIR=/opt/rt/etc ./run.sh
```

Override individual roots:
```bash
URM_COMMON_CONFIG_DIR=/opt/rt/etc/urm URM_TESTS_CONFIG_DIR=/opt/rt/usr/share/urm ./run.sh
```

Point to a custom test-nodes directory:
```bash
URM_TEST_NODES_DIR=/opt/rt/usr/share/urm ./run.sh
```

---

## Exit status
Expand Down
87 changes: 81 additions & 6 deletions Runner/suites/Performance/userspace-resource-manager/run.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,13 +71,15 @@ if command -v flock >/dev/null 2>&1; then
echo "$TESTNAME SKIP" >"$RES_FILE"
exit 0
fi
lock_flock=1
trap 'exec 9>&-' EXIT INT TERM
else
if ! mkdir "$LOCKDIR" 2>/dev/null; then
log_warn "Another ${TESTNAME} run is active or stale fallback lockdir exists: $LOCKDIR"
echo "$TESTNAME SKIP" >"$RES_FILE"
exit 0
fi
lock_flock=0
trap 'rmdir "$LOCKDIR" 2>/dev/null || true' EXIT INT TERM
fi

Expand Down Expand Up @@ -275,10 +277,50 @@ else
fi

# ---------- Config preflight (check both common/ and tests/) ----------
URM_CONFIG_DIR="${URM_CONFIG_DIR:-/etc/urm}"
COMMON_CONFIGS_DIR="$URM_CONFIG_DIR/common"
TEST_CONFIGS_DIR="$URM_CONFIG_DIR/tests/configs"
TEST_NODES_DIR="$URM_CONFIG_DIR/tests/nodes"
# Resolution order for each root (first match wins):
# 1. Explicit per-root override (URM_COMMON_CONFIG_DIR / URM_TESTS_CONFIG_DIR)
# 2. Legacy URM_CONFIG_DIR (compatibility fallback for existing callers)
# 3. Built-in default
#
# For test-nodes the candidate order is:
# 1. Explicit URM_TEST_NODES_DIR override
# 2. Legacy URM_CONFIG_DIR (compatibility fallback for existing callers)
# 3. /var/lib/urm — runtime/writable location, only when non-empty
# 4. /usr/share/urm — package-installed location (default)

# Resolve common-config root
if [ -n "${URM_COMMON_CONFIG_DIR:-}" ]; then
common_root="$URM_COMMON_CONFIG_DIR"
elif [ -n "${URM_CONFIG_DIR:-}" ]; then
common_root="$URM_CONFIG_DIR"
else
common_root="/etc/urm"
fi

# Resolve tests-config root
if [ -n "${URM_TESTS_CONFIG_DIR:-}" ]; then
tests_root="$URM_TESTS_CONFIG_DIR"
elif [ -n "${URM_CONFIG_DIR:-}" ]; then
tests_root="$URM_CONFIG_DIR"
else
tests_root="/usr/share/urm"
fi

# Resolve test-nodes root: explicit override > legacy URM_CONFIG_DIR > populated runtime dir > package dir
if [ -n "${URM_TEST_NODES_DIR:-}" ]; then
nodes_root="$URM_TEST_NODES_DIR"
elif [ -n "${URM_CONFIG_DIR:-}" ]; then
nodes_root="$URM_CONFIG_DIR"
elif [ -d "/var/lib/urm/tests/nodes" ] && \
[ "$(find "/var/lib/urm/tests/nodes" -mindepth 1 -maxdepth 1 -type f 2>/dev/null | wc -l | awk '{print $1}')" -gt 0 ]; then
nodes_root="/var/lib/urm"
else
nodes_root="/usr/share/urm"
fi

COMMON_CONFIGS_DIR="$common_root/common"
TEST_CONFIGS_DIR="$tests_root/tests/configs"
TEST_NODES_DIR="$nodes_root/tests/nodes"

COMMON_CONFIGS_OK=1
TEST_CONFIGS_OK=1
Expand Down Expand Up @@ -397,6 +439,39 @@ if [ -z "$TESTS" ]; then
exit 0
fi

# ---------- Stage test nodes into a private temporary directory ----------
# UrmComponentTests and UrmIntegrationTests accept a --npath argument that
# tells them where to find the writable test nodes. The package installs
# read-only node files under /usr/share/urm/tests/nodes (or /var/lib/urm),
# so we copy them into a mktemp-owned directory before running the tests.
# mktemp -d guarantees a fresh unique base exclusively owned by this run;
# the trap registered immediately after removes only that base directory
# on exit, interrupt, or termination.
RUNTIME_NODES_DIR=""
if [ "$TEST_NODES_OK" -eq 1 ]; then
nodes_tmp_base="$(mktemp -d)"

@smuppand Srikanth Muppandam (smuppand) Sep 9, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mktemp -d is unchecked. If it fails, nodes_tmp_base is empty and line 458 turns the destination into /urm/tests/nodes; a root-run test will then create and copy files outside test-owned temporary state. The cleanup trap removes "$nodes_tmp_base"—an empty path—not that accidental directory.

Check mktemp -d before deriving RUNTIME_NODES_DIR; log a clean skip/failure on error, and register cleanup only for a confirmed private directory.

@kartnema Kartik Nema (kartnema) Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed now

@kartnema Kartik Nema (kartnema) Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also added checks for verifying if mkdir -p "$RUNTIME_NODES_DIR"; succeeded.

if [ -z "$nodes_tmp_base" ] || [ ! -d "$nodes_tmp_base" ]; then
log_warn "[NODES] mktemp -d failed — suites requiring nodes will SKIP"
TEST_NODES_OK=0
else
if [ "$lock_flock" -eq 1 ]; then
trap 'rm -rf "$nodes_tmp_base"; exec 9>&-' EXIT INT TERM
else
trap 'rm -rf "$nodes_tmp_base"; rmdir "$LOCKDIR" 2>/dev/null || true' EXIT INT TERM
fi
RUNTIME_NODES_DIR="$nodes_tmp_base/urm/tests/nodes"
if ! mkdir -p "$RUNTIME_NODES_DIR"; then
log_warn "[NODES] Failed to create staging directory $RUNTIME_NODES_DIR — suites requiring nodes will SKIP"
TEST_NODES_OK=0
elif cp -r "$TEST_NODES_DIR/"* "$RUNTIME_NODES_DIR/"; then
log_info "[NODES] Staged test nodes from $TEST_NODES_DIR to $RUNTIME_NODES_DIR"
else
log_warn "[NODES] Failed to stage test nodes into $RUNTIME_NODES_DIR — suites requiring nodes will SKIP"
TEST_NODES_OK=0
fi
fi
fi

# ---------- Execute ----------
PASS=0
FAIL=0
Expand Down Expand Up @@ -440,7 +515,7 @@ run_one() {

log_info "--- Running $bin ---"
log_info "[CI] Logging to $tlog"
run_cmd_maybe_timeout "$bin" >"$tlog" 2>&1
run_cmd_maybe_timeout "$bin" --npath "$RUNTIME_NODES_DIR" >"$tlog" 2>&1
rc=$?

case $rc in
Expand Down Expand Up @@ -508,7 +583,7 @@ log_info "Overall counts: PASS=$PASS FAIL=$FAIL SKIP=$SKIP"
# - Else -> overall SKIP (everything skipped)
if [ "$FAIL" -gt 0 ]; then
echo "$TESTNAME FAIL" >"$RES_FILE"
exit 1
exit 1
fi
if [ "$PASS" -gt 0 ]; then
echo "$TESTNAME PASS" >"$RES_FILE"
Expand Down
Loading