Skip to content

feat: add Google Benchmark bench subcommand - #79

Open
zhangzhenghao wants to merge 4 commits into
mainfrom
feat/bench
Open

zhangzhenghao wants to merge 4 commits into
mainfrom
feat/bench

Conversation

@zhangzhenghao

@zhangzhenghao zhangzhenghao commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Add clap bench for serial text, audio embeddings and waveform-frame inference using pinned Google Benchmark v1.9.4.
  • Forward native benchmark options after --, including filtering, repetitions, listing and JSON/CSV output; fail on inference/report errors and unmatched filters.
  • Measure full synchronous public C API calls (preprocessing and output allocation/free included), excluding weight loading, audio decoding and one warmup; report wall-clock milliseconds and item throughput.
  • Keep Google Benchmark static/PIC and private to the CLI; core-only builds and exported core SDK dependencies remain unaffected.
  • Add generated tiny-model CLI and packaging regressions (README unchanged).

Verification

All local builds and compilation temporary files are on /dev/sdb1 under /mnt/openclaw/clap-bench-dev.

  • Final static full CTest suite: 37/37 passed; new benchmark CLI and packaging tests separately re-run after test-portability fixes.
  • Shared build and final full CTest suite: 36/36 passed.
  • Core-only configuration, build and packaging/install checks verified locally.
  • Installed shared CLI exercised with real generated-model inference.
  • No published-checkpoint performance or GPU measurements claimed; numerical/inference checks use deterministic generated fixtures.

Remote CI

All 9 checks passed on head e222bf04eef7ac4a9a41ac80a34f604ffba05ca7 (run 42550921790): formatting; native build/test/install and applicable Python/Node tests on Linux x64/ARM64, macOS ARM64, Windows x64/ARM64; Rust checks on Linux, macOS and Windows.

Earlier CI findings were fixed in the new test infrastructure: honor custom executable install directories and avoid minimum-duration assertions against native benchmark calibration. The transient Hugging Face download failure in the unchanged Node suite did not recur on the latest head. README remains unchanged.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants