Skip to content

Add a FrontierHarness Eval adapter under contrib/frontierharness (ENG-4328) - #806

Merged
ZoranPandovski merged 1 commit into
stagingfrom
feature/frontierharness-adapter
Nov 16, 2026
Merged

ZoranPandovski merged 1 commit into
stagingfrom
feature/frontierharness-adapter

Conversation

@ZoranPandovski

Copy link
Copy Markdown
Member

FrontierHarness Eval only lists a harness after its maintainers rerun it themselves on Runta, so they need to install anton and its benchmark adapter from a public repo. This adds that adapter under contrib/frontierharness/: the Harbor and Pier agent classes, the install and run scripts their workflow calls, and a small patch for their cost script. The README has the exact commands.

Nothing outside contrib/ changes, and the folder is not in the wheel or the test suite.

Tested on local Docker: the same adapter ran all 30 benchmark tasks, and run-task.sh ran end to end on a non-benchmark task. It has not run on Runta yet.

…-4328)

Lets the benchmark maintainers rerun anton on Runta from this repo at a pinned commit: Harbor and Pier agent classes, the install and run scripts their workflow calls, and the patch their cost script needs to read anton's usage.

@ianu82 ianu82 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

@ZoranPandovski
ZoranPandovski merged commit 4fa1f7c into staging Nov 16, 2026
15 checks passed
@ZoranPandovski
ZoranPandovski deleted the feature/frontierharness-adapter branch November 16, 2026 15:56
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 16, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants