Sign Parabricks skill by ohadmo · Pull Request #51 · NVIDIA-BioNeMo/bionemo-agent-toolkit

ohadmo · 2026-10-02T00:05:03Z

No description provided.

Signed-off-by: Ohad Mosafi <omosafi@nvidia.com>

ohadmo · 2026-10-02T00:05:10Z

@greptileai

ohadmo · 2026-10-02T00:05:17Z

/nvskills-ci

greptile-apps · 2026-10-02T00:07:07Z

Confidence Score: 1/5

[Medium impact] Updates documentation and test data for genomics skill.

The PR is not safe to merge while plugin synchronization fails and the PON and readiness guidance regressions remain.

Findings

Nested payload fails sync ▶
Probe timeout is ignored ▶
Translator fails on Python 3.7 ▶
Evaluation metadata conflicts ▶
Neutral verdict missing from card ▶
Neutral verdict cites missing decline ▶

Summary

The PR refreshes Parabricks command references and evaluations, replaces the Python readiness helper with Bash, and updates publication artifacts. The generated plugin payload is not synchronized with its source, the revised PON guidance conflicts with the supported linear-reference workflow, and the Bash helper can lose its timeout guarantee.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  S[Source Parabricks skill] --> C[Plugin sync]
  C --> P[Flat distributed skill]
  P --> A[Card, benchmark and signature]
  S --> R[Tool references and readiness helper]

_{Reviews (17) · Last reviewed commit: "Merge remote-tracking branch 'origin/par..." · Reviewed by Greptile}

Signed-off-by: Ohad Mosafi <omosafi@nvidia.com>

ohadmo · 2026-10-02T00:56:56Z

/nvskills-ci

Signed-off-by: nvskills-svc-account <svc-nvskills-signing@nvidia.com>

Signed-off-by: Ohad Mosafi <omosafi@nvidia.com>

ohadmo · 2026-10-02T01:22:42Z

/nvskills-ci

Signed-off-by: nvskills-svc-account <svc-nvskills-signing@nvidia.com>

Signed-off-by: Ohad Mosafi <omosafi@nvidia.com>

ohadmo · 2026-10-02T17:45:41Z

/nvskills-ci

Signed-off-by: nvskills-svc-account <svc-nvskills-signing@nvidia.com>

Signed-off-by: Ohad Mosafi <omosafi@nvidia.com>

ohadmo · 2026-10-02T18:39:31Z

/nvskills-ci

greptile-apps · 2026-10-02T18:54:40Z

+## Evaluation Results: <br>
+| Measure | Claude Code (Baseline → Skill Uplift) | Codex (Baseline → Skill Uplift) |
+|---|---:|---:|
+| Overall | 97.1% | 92.6% |


Neutral verdict missing from card

The refreshed card lists strong evaluation scores but omits the benchmark’s new NEUTRAL publication verdict and its recommendation to collect more evidence before deciding to publish. Someone using the card as a release summary could mistake the scores for a positive publication recommendation. Please include the verdict or link to the benchmark alongside the results.

Knowledge Base Used: Skill quality and compliance controls

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Signed-off-by: Ohad Mosafi <omosafi@nvidia.com>

ohadmo · 2026-10-02T18:57:00Z

/nvskills-ci

Signed-off-by: nvskills-svc-account <svc-nvskills-signing@nvidia.com>

Signed-off-by: Ohad Mosafi <omosafi@nvidia.com>

ohadmo · 2026-10-02T21:37:57Z

/nvskills-ci

Signed-off-by: Ohad Mosafi <omosafi@nvidia.com>

ohadmo · 2026-10-02T22:09:58Z

/nvskills-ci

Signed-off-by: nvskills-svc-account <svc-nvskills-signing@nvidia.com>

Signed-off-by: Ohad Mosafi <omosafi@nvidia.com>

ohadmo · 2026-10-07T18:17:38Z

/nvskills-ci

greptile-apps · 2026-10-07T18:21:49Z

+            result["issues"].append("Parabricks will generate read groups. Confirm sample metadata if read-group identity matters; no metadata was inferred by this helper.")
+        result["notes"].append("Draft only: paths, reference/index compatibility, numeric ranges, runtime readiness, and biological output parity are not validated.")
+        result.update(status="needs_review" if result["issues"] else "draft",
+                      draft_argv=argv, command=shlex.join(argv))


Translator fails on Python 3.7

If the host uses Python 3.7, every otherwise valid translation reaches shlex.join, which is unavailable in that version. The helper then raises an uncaught AttributeError instead of producing a command, even though its reference says only that Python 3 is required. Specify Python 3.8 or newer for this helper, or use a compatible command formatter.

Signed-off-by: nvskills-svc-account <svc-nvskills-signing@nvidia.com>

greptile-apps · 2026-10-07T19:15:29Z

@@ -0,0 +1,153 @@
+# Skill Benchmark: parabricks
+
+> **Overall verdict: NEUTRAL — One or more dimensions remain below PASS**


Neutral verdict cites missing decline

The refreshed benchmark says its NEUTRAL verdict is because one or more dimensions are below PASS. But every reported Codex dimension exceeds the report’s 50% PASS threshold, and every Claude Code score is unavailable. This explanation points readers to a low score the results do not show, making it harder to tell whether the limitation is missing evaluation evidence instead.

Knowledge Base Used: Skill quality and compliance controls

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

- Update tool references and command conventions for Parabricks 4.7.1 - Improve planning triggers, option translation, and input safeguards - Replace the Python readiness helper and adapt tests and eval fixtures Signed-off-by: Angel Pizarro <apizarro@nvidia.com>

…s-skill Signed-off-by: Angel Pizarro <apizarro@nvidia.com>

greptile-apps · 2026-10-08T21:00:05Z

@@ -0,0 +1,218 @@
+---


Nested payload fails sync

The new parabricks/parabricks/ directory exists in the generated plugin payload but not in the source skill. The plugin freshness check treats its files as extra content and fails. The top-level distributed SKILL.md also remains at version 1.2.1 while the source is 1.2.4, so users receive older guidance. Regenerate the flat payload from the source instead of adding a second skill directory inside it.

Knowledge Base Used: Plugin synchronization and generated skills

greptile-apps · 2026-10-08T21:00:05Z

+  else
+    "$@" >"$TMP_DIR/out" 2>"$TMP_DIR/err" </dev/null
+    CMD_RC=$?


Probe timeout is ignored

If neither timeout nor gtimeout is installed, this fallback runs external probes without a time limit. A stalled Docker command can therefore leave the readiness check hanging indefinitely even when the caller supplied --timeout, rather than returning a report.

Knowledge Base Used: Genomic acceleration library integrations

greptile-apps · 2026-10-08T21:00:05Z

+- Tasks: 18 evaluation tasks (17 positive, 1 negative)
+- Dataset digest: `sha256:6e9efe466a1409e4cbcd58924a921028b12d3af7cce343c86b975cfbf78f9f1b` (skill-evaluator-dataset-snapshot/1)


Evaluation metadata conflicts

The refreshed results cover seven live tasks, but this metadata still says 18 tasks and gives a different dataset digest from the seven-task skill card. Readers cannot tell which dataset produced the published scores and verdict, making the release evidence harder to assess.

Knowledge Base Used: Skill quality and compliance controls

greptile-apps · 2026-10-08T21:00:14Z

Comments Outside Diff

These findings could not be posted inline.

Linear PON workflow rejected library-skills/parabricks/references/pbrun-prepon.md:70 ▶

For a standard GRCh38 mutectcaller panel-of-normals workflow, this new guardrail tells agents not to use prepon and the preceding guidance asks for pangenome resources. But the mutectcaller reference requires prepon to prepare the PON resource. The postpon reference likewise forbids standard linear-reference post-processing, so following these instructions can prevent users from preparing and annotating a valid PON workflow.

Sign Parabricks skill

a646daa

Signed-off-by: Ohad Mosafi <omosafi@nvidia.com>

Deduplicate Parabricks skill references and fix CI validation issues

1208967

Signed-off-by: Ohad Mosafi <omosafi@nvidia.com>

Attach NVSkills validation signatures

9875a9b

Signed-off-by: nvskills-svc-account <svc-nvskills-signing@nvidia.com>

greptile-apps Bot reviewed Oct 2, 2026

View reviewed changes

Comment thread skills/bionemo-agent-toolkit/skills/parabricks/skill-card.md Outdated

Clarify Parabricks runtime requirements and NGC authentication

e2fadf2

Signed-off-by: Ohad Mosafi <omosafi@nvidia.com>

greptile-apps Bot reviewed Oct 2, 2026

View reviewed changes

Comment thread skills/bionemo-agent-toolkit/skills/parabricks/SKILL.md

Comment thread library-skills/parabricks/SKILL.md Outdated

Attach NVSkills validation signatures

960ad38

Signed-off-by: nvskills-svc-account <svc-nvskills-signing@nvidia.com>

greptile-apps Bot reviewed Oct 2, 2026

View reviewed changes

Comment thread skills/bionemo-agent-toolkit/skills/parabricks/BENCHMARK.md

Fix Parabricks GPU detection and improve skill evaluation

d08cf9a

Signed-off-by: Ohad Mosafi <omosafi@nvidia.com>

greptile-apps Bot reviewed Oct 2, 2026

View reviewed changes

Comment thread library-skills/parabricks/evals/evals.json Outdated

Comment thread library-skills/parabricks/references/runtime-environment.md

Attach NVSkills validation signatures

254203a

Signed-off-by: nvskills-svc-account <svc-nvskills-signing@nvidia.com>

greptile-apps Bot reviewed Oct 2, 2026

View reviewed changes

Comment thread skills/bionemo-agent-toolkit/skills/parabricks/skill-card.md Outdated

Fix truncated dataset digests in generated skill cards

d813c47

Signed-off-by: Ohad Mosafi <omosafi@nvidia.com>

greptile-apps Bot reviewed Oct 2, 2026

View reviewed changes

Document release-compatible CUDA probes for Parabricks 4.2

1494369

Signed-off-by: Ohad Mosafi <omosafi@nvidia.com>

ohadmo force-pushed the parabricks-skill branch from 9daa040 to 1494369 Compare October 2, 2026 18:56

greptile-apps Bot reviewed Oct 2, 2026

View reviewed changes

Comment thread skills/bionemo-agent-toolkit/skills/parabricks/skill.oms.sig Outdated

svc-nvskills-signing and others added 2 commits October 2, 2026 19:07

Attach NVSkills validation signatures

5308876

Signed-off-by: nvskills-svc-account <svc-nvskills-signing@nvidia.com>

Avoid sensitive host probes in Parabricks readiness checks

60fede8

Signed-off-by: Ohad Mosafi <omosafi@nvidia.com>

Fix Parabricks Tier 1 false positive in Docker socket guidance

53b7d6e

Signed-off-by: Ohad Mosafi <omosafi@nvidia.com>

svc-nvskills-signing and others added 2 commits October 2, 2026 22:22

Attach NVSkills validation signatures

e5c239f

Signed-off-by: nvskills-svc-account <svc-nvskills-signing@nvidia.com>

Add STAR translation helper and strengthen Parabricks evaluations

159fa23

Signed-off-by: Ohad Mosafi <omosafi@nvidia.com>

ohadmo force-pushed the parabricks-skill branch from e5c239f to 159fa23 Compare October 7, 2026 18:16

greptile-apps Bot reviewed Oct 7, 2026

View reviewed changes

Attach NVSkills validation signatures

689e4e3

Signed-off-by: nvskills-svc-account <svc-nvskills-signing@nvidia.com>

greptile-apps Bot reviewed Oct 7, 2026

View reviewed changes

delagoya added 2 commits October 8, 2026 13:18

Merge remote-tracking branch 'origin/parabricks-skill' into parabrick…

b92e22f

…s-skill Signed-off-by: Angel Pizarro <apizarro@nvidia.com>

greptile-apps Bot reviewed Oct 8, 2026

View reviewed changes

		@@ -0,0 +1,153 @@
		# Skill Benchmark: parabricks

		> Overall verdict: NEUTRAL — One or more dimensions remain below PASS

		- Tasks: 18 evaluation tasks (17 positive, 1 negative)
		- Dataset digest: `sha256:6e9efe466a1409e4cbcd58924a921028b12d3af7cce343c86b975cfbf78f9f1b` (skill-evaluator-dataset-snapshot/1)

Conversation

ohadmo commented Oct 2, 2026

Uh oh!

ohadmo commented Oct 2, 2026

Uh oh!

ohadmo commented Oct 2, 2026

Uh oh!

greptile-apps Bot commented Oct 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 1/5

Findings

Summary

Diagram

Uh oh!

ohadmo commented Oct 2, 2026

Uh oh!

Uh oh!

ohadmo commented Oct 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ohadmo commented Oct 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ohadmo commented Oct 2, 2026

Uh oh!

greptile-apps Bot Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

ohadmo commented Oct 2, 2026

Uh oh!

Uh oh!

ohadmo commented Oct 2, 2026

Uh oh!

ohadmo commented Oct 2, 2026

Uh oh!

ohadmo commented Oct 7, 2026

Uh oh!

greptile-apps Bot Oct 7, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot Oct 7, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Oct 8, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Oct 8, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Oct 8, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot commented Oct 8, 2026

Comments Outside Diff

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

greptile-apps Bot commented Oct 2, 2026 •

edited

Loading