Skip to content

Commit a80eddf

Browse files
sanchitrkclaude
andauthored
fix(harness): run the bash tool's command under bash (#78)
The command ran under each backend's `sh`, which is dash on Debian and Ubuntu, so bash syntax failed there. It was also pasted into the output wrapper, so a bash-only construct broke the wrapper's own parse. The wrapper now carries the command as a quoted string and runs it with `bash -c` when bash is on PATH, else `eval` under `sh`. The shell rides on the wrapper's report, and a failure under `sh` says so to the model. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent e7c141b commit a80eddf

8 files changed

Lines changed: 129 additions & 14 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,16 @@ This file is the canonical source for unreleased changes and published release n
66

77
- Add in-progress work under `## [Unreleased]
88

9+
## [Unreleased]
10+
11+
### Fixed
12+
13+
- The `bash` tool now runs the model's command under bash on every sandbox. It ran under `sh`, which is dash on Debian and Ubuntu, so bash syntax such as arrays, `[[ ]]`, `{1..3}`, here-strings and `source` worked on a Mac but failed on Linux and in Debian-based containers, and broke the tool's own output handling, so the model got an error pointing into CodeWork's script rather than its command. Where bash is not on `PATH`, the command runs under `sh`, and a failure says so: `Command exited with code 2 (ran under sh: bash is not on PATH in this sandbox)`.
14+
15+
### Internal
16+
17+
- Added E2E tests that run bash-only syntax through the `bash` tool on the host, just-bash, Daytona and Vercel sandboxes and check they print the same output, and that a host without bash runs the command under `sh` and says so when it fails.
18+
919
## [@codeworksh/cli@0.3.0] - 2026-10-13
1020

1121
### Added

‎packages/harness/src/plugin/builtin/tool/bash.ts‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,8 @@ const BashSuccess = Schema.Struct({ ...outputFields, exitCode: Schema.Finite });
5353
class BashFailed extends Schema.TaggedError<BashFailed>()("BashFailed", {
5454
...outputFields,
5555
exitCode: Schema.Finite,
56+
/** The shell that ran the command, when the sandbox reported it. */
57+
shell: Schema.optional(Schema.Literals(["bash", "sh"])),
5658
}) {}
5759

5860
/** Deadline exceeded — expected, model-visible. Carries the partial output produced so far. */
@@ -71,7 +73,7 @@ const withStatus = (failure: BashFailureError): string => {
7173
? `Command timed out after ${failure.timeoutSeconds} seconds`
7274
: failure.exitCode === -1
7375
? "Command terminated without an exit code"
74-
: `Command exited with code ${failure.exitCode}`;
76+
: `Command exited with code ${failure.exitCode}${failure.shell === "sh" ? " (ran under sh: bash is not on PATH in this sandbox)" : ""}`;
7577
return failure.output ? `${failure.output.replace(/\n$/, "")}\n\n${status}` : status;
7678
};
7779

@@ -134,6 +136,8 @@ const present = (snap: OutputSnapshot, lastLineBytes: number, saved: string | un
134136

135137
const utf8 = new TextEncoder();
136138

139+
const shellOf = (report: Output.Report | undefined) => (report !== undefined ? { shell: report.shell } : {});
140+
137141
/**
138142
* A command cut off by its deadline never reached the wrapper's own cleanup: kill
139143
* what is left of it and remove its side files, keeping the output file only when
@@ -195,7 +199,7 @@ const runBuffered = (
195199
acc.finish();
196200
const presented = present(acc.snapshot(), acc.getLastLineBytes(), window?.kept === true ? file : undefined);
197201
if (result.exitCode !== 0) {
198-
return yield* new BashFailed({ exitCode: result.exitCode, ...presented });
202+
return yield* new BashFailed({ exitCode: result.exitCode, ...presented, ...shellOf(window) });
199203
}
200204
return { exitCode: result.exitCode, ...presented } satisfies typeof BashSuccess.Type;
201205
});
@@ -275,7 +279,7 @@ const runStreaming = (
275279
const code = yield* Ref.get(exitCode);
276280
// No Exit event (e.g. killed before reporting one) → treat as a failure.
277281
if (code === null || code !== 0) {
278-
return yield* new BashFailed({ exitCode: code ?? -1, ...presented });
282+
return yield* new BashFailed({ exitCode: code ?? -1, ...presented, ...shellOf(report) });
279283
}
280284
return { exitCode: code, ...presented } satisfies typeof BashSuccess.Type;
281285
});

‎packages/harness/src/tool/output.ts‎

Lines changed: 20 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -49,10 +49,15 @@ const script = (
4949
) =>
5050
[
5151
`f=${quote(file)}`,
52+
// The command stays a quoted string, so syntax `sh` lacks cannot break the
53+
// wrapper: it runs under bash where the sandbox has one, else under `sh`.
54+
`c=${quote(command)}`,
55+
"if command -v bash >/dev/null 2>&1; then s=bash; else s=sh; fi",
5256
// A function whose body is a subshell: `exit` in the command ends only it,
53-
// and the command is written once for both branches below.
57+
// and the command is written once for both branches below. No `exec`:
58+
// just-bash carries on past it.
5459
"run() (",
55-
command,
60+
'\tif [ "$s" = bash ]; then bash -c "$c"; else eval "$c"; fi',
5661
")",
5762
// The side files are created the same exclusive way as the output file.
5863
// `umask` may be missing where modes mean nothing (an in-process shell over a virtual filesystem).
@@ -83,21 +88,24 @@ const script = (
8388
"run 2>&1",
8489
].join("\n");
8590

86-
/** The sandbox's own account of the output: its raw size, and whether the file was kept. */
91+
/** The sandbox's own account of the output: its raw size, whether the file was kept, and the shell that ran it. */
8792
export interface Report {
8893
/** Bytes and newlines in the whole output. */
8994
readonly bytes: number;
9095
readonly newlines: number;
9196
/** Whether the sandbox kept the output file — decided on raw bytes, which the harness only sees decoded. */
9297
readonly kept: boolean;
98+
/** `sh` when the sandbox has no bash. */
99+
readonly shell: "bash" | "sh";
93100
}
94101

95-
const header = new RegExp(`^${HEADER} (\\d+) (\\d+) ([01])(?: (\\d+))?$`, "m");
102+
const header = new RegExp(`^${HEADER} (\\d+) (\\d+) ([01]) (bash|sh)(?: (\\d+))?$`, "m");
96103

97104
const parse = (match: RegExpExecArray): Report => ({
98105
bytes: Number(match[1]),
99106
newlines: Number(match[2]),
100107
kept: match[3] === "1",
108+
shell: match[4] === "sh" ? "sh" : "bash",
101109
});
102110

103111
/**
@@ -107,7 +115,7 @@ const parse = (match: RegExpExecArray): Report => ({
107115
*/
108116
export const streaming = (command: string, file: string, limits: Limits) =>
109117
script(command, file, limits, (source) => `${source} | tee -a "$f"`, [
110-
`printf '${HEADER} %s %s %s\\n' "$b" "$n" "$keep" >&2`,
118+
`printf '${HEADER} %s %s %s %s\\n' "$b" "$n" "$keep" "$s" >&2`,
111119
]);
112120

113121
/** Read {@link streaming}'s stderr: the report, if it got that far, and anything else the wrapper said. */
@@ -134,7 +142,11 @@ export const buffered = (command: string, file: string, limits: Limits) => {
134142
// The redirect sits on the whole group, not on `cat`: an in-process shell
135143
// (just-bash) re-encodes text that a command reading a pipe writes to a file.
136144
(source) => `{ ${source} | cat; } >> "$f"`,
137-
[`w=$(${tail} | wc -c | awk '{ print $1 }')`, `printf '${HEADER} %s %s %s %s\\n' "$b" "$n" "$keep" "$w"`, tail],
145+
[
146+
`w=$(${tail} | wc -c | awk '{ print $1 }')`,
147+
`printf '${HEADER} %s %s %s %s %s\\n' "$b" "$n" "$keep" "$s" "$w"`,
148+
tail,
149+
],
138150
);
139151
};
140152

@@ -149,8 +161,8 @@ export interface Window extends Report {
149161
export const window = (stdout: string): Window | undefined => {
150162
const newline = stdout.indexOf("\n");
151163
const match = header.exec(newline === -1 ? stdout : stdout.slice(0, newline));
152-
if (match === null || match[4] === undefined) return undefined;
153-
return { ...parse(match), text: stdout.slice(newline + 1), textBytes: Number(match[4]) };
164+
if (match === null || match[5] === undefined) return undefined;
165+
return { ...parse(match), text: stdout.slice(newline + 1), textBytes: Number(match[5]) };
154166
};
155167

156168
/**
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"status": "completed",
3+
"text": "array b
4+
1 2 3
5+
here-string
6+
source x=1
7+
shell bash
8+
",
9+
}

‎packages/harness/test/fixtures/bash.spec.ts‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ import type { SandboxDriver } from "../../src/sandbox/driver.ts";
1111
import { SandboxIO } from "../../src/sandbox/io.ts";
1212
import { bashPlugin } from "../../src/plugin/builtin/tool/bash.ts";
1313
import { defaultPromptPlugin } from "../../src/plugin/builtin/prompt/default.ts";
14-
import { pendingCall } from "../tools.fixture.ts";
14+
import { BASH_SYNTAX, pendingCall } from "../tools.fixture.ts";
1515
import { toolTurn } from "./llm.ts";
1616
import { remoteSuite } from "./live.ts";
1717
import { withSettings } from "./settings.ts";
@@ -112,6 +112,20 @@ export const bashPluginSpec = (options: {
112112
180_000,
113113
);
114114

115+
it(
116+
"runs bash-only syntax under bash, as every local backend does",
117+
() =>
118+
withSettings(async (input) => {
119+
const { call } = await exchange(input, BASH_SYNTAX);
120+
if (call.status !== "completed" && call.status !== "error") throw new Error("bash was not settled");
121+
const text = call.result.content.flatMap((part) => (part.type === "text" ? [part.text] : [])).join("");
122+
await expect({ status: call.status, text }).toMatchFileSnapshot(
123+
"./__artifacts__/tools.bash.syntax.json",
124+
);
125+
}),
126+
180_000,
127+
);
128+
115129
it(
116130
"truncates real output and preserves the entire spill file",
117131
() =>

‎packages/harness/test/tools.fixture.ts‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,3 +12,12 @@ export const pendingCall = (
1212
status: "pending",
1313
time: { start: 1, end: 1 },
1414
});
15+
16+
/** Syntax that `sh` (dash, ash) rejects: every backend must run it under bash. */
17+
export const BASH_SYNTAX = [
18+
'arr=(a b c); [[ ${#arr[@]} -eq 3 ]] && echo "array ${arr[1]}"',
19+
"echo {1..3}",
20+
"cat <<< here-string",
21+
`printf 'x=1\\n' > bash-syntax.env && source bash-syntax.env && echo "source x=$x"`,
22+
'echo "shell ${BASH_VERSION:+bash}"',
23+
].join("; ");

‎packages/harness/test/tools.output.test.ts‎

Lines changed: 58 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
import { Effect, Fiber, Layer, Stream } from "effect";
22
import type { ChildProcessSpawner } from "effect/process";
3+
import { existsSync } from "node:fs";
34
import fs from "node:fs/promises";
45
import { basename } from "node:path";
56
import { describe, expect, it } from "vite-plus/test";
@@ -14,7 +15,7 @@ import * as Executor from "../src/tool/executor.ts";
1415
import { fromSandboxShell, ToolShell } from "../src/tool/shell.ts";
1516
import * as Tool from "../src/tool/tool.ts";
1617
import { tmpdir } from "./fixtures/tempdir.ts";
17-
import { pendingCall } from "./tools.fixture.ts";
18+
import { BASH_SYNTAX, pendingCall } from "./tools.fixture.ts";
1819

1920
/**
2021
* The bash tool end to end over two real backends — the host shell, which
@@ -33,6 +34,8 @@ import { pendingCall } from "./tools.fixture.ts";
3334
* - a backend that cannot cut a command short leaves it running after the deadline
3435
* - a backend that hands text rather than bytes reports a file the sandbox removed,
3536
* or numbers the shown lines wrongly
37+
* - bash-only syntax fails where `sh` is not bash, or breaks the wrapper itself
38+
* - a sandbox without bash fails the command without saying it ran under `sh`
3639
*/
3740

3841
interface Details {
@@ -172,6 +175,23 @@ describe.each([
172175
if (backend !== "justbash") await fs.rm(result.details.fullOutputPath!);
173176
});
174177

178+
it("runs bash-only syntax under bash", async () => {
179+
const { dir, layer } = await mount();
180+
await using _ = dir;
181+
const result = await run(layer, BASH_SYNTAX);
182+
await expect({ status: result.status, text: result.text }).toMatchFileSnapshot(
183+
"./__artifacts__/tools.bash.syntax.json",
184+
);
185+
});
186+
187+
it("keeps a syntax error inside the command", async () => {
188+
const { dir, layer } = await mount();
189+
await using _ = dir;
190+
const result = await run(layer, "echo before; if then");
191+
expect(result.details).toMatchObject({ _tag: "BashFailed", exitCode: 2, shell: "bash" });
192+
expect(result.text).toMatch(/\n\nCommand exited with code 2$/);
193+
});
194+
175195
it("tells the model when there is no output, and why a silent command failed", async () => {
176196
const { dir, layer } = await mount();
177197
await using _ = dir;
@@ -291,6 +311,43 @@ describe.each([
291311
}
292312
});
293313

314+
// A host without bash: the tools the wrapper needs, and nothing else.
315+
const withoutBash = async (dir: string) => {
316+
const bin = `${dir}/bin`;
317+
await fs.mkdir(bin);
318+
for (const tool of ["awk", "cat", "mkdir", "ps", "rm", "tail", "tee", "wc"]) {
319+
const found = (process.env.PATH ?? "")
320+
.split(":")
321+
.map((path) => `${path}/${tool}`)
322+
.find((path) => existsSync(path));
323+
if (found === undefined) throw new Error(`${tool} not on PATH`);
324+
await fs.symlink(found, `${bin}/${tool}`);
325+
}
326+
return HostExe.layer({ shell: "/bin/sh", env: { PATH: bin } });
327+
};
328+
329+
describe.each([
330+
["streams", (shell: Layer.Layer<Shell, never, ChildProcessSpawner.ChildProcessSpawner>) => shell],
331+
[
332+
"buffers",
333+
(shell: Layer.Layer<Shell, never, ChildProcessSpawner.ChildProcessSpawner>) =>
334+
Layer.effect(
335+
Shell,
336+
Effect.map(Shell, (inner) => Shell.of({ exec: inner.exec, execArgv: inner.execArgv })),
337+
).pipe(Layer.provide(shell)),
338+
],
339+
] as const)("bash tool on a host without bash (%s)", (_mode, backend) => {
340+
it("runs the command under sh, and says so when it fails", async () => {
341+
await using dir = await tmpdir();
342+
const layer = host(dir.path, backend(await withoutBash(dir.path)));
343+
const ok = await run(layer, "echo ok");
344+
expect(ok).toMatchObject({ status: "completed", text: "ok\n" });
345+
const failed = await run(layer, "echo no; exit 3");
346+
expect(failed.details).toMatchObject({ _tag: "BashFailed", exitCode: 3, shell: "sh" });
347+
expect(failed.text).toBe("no\n\nCommand exited with code 3 (ran under sh: bash is not on PATH in this sandbox)");
348+
});
349+
});
350+
294351
describe("ToolShell over the host sandbox shell", () => {
295352
it("force-kills a command that ignores SIGTERM at its deadline", async () => {
296353
await using dir = await tmpdir();

‎sites/www/content/docs/sandboxes/local.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ description: The default sandbox. Your folder and your real shell.
55

66
## How It Works
77

8-
`local` is the default driver. It binds the agent directly to your machine: file tools use your real filesystem, starting in the current directory or in `--cwd`, and `bash` runs real commands through `sh` with the environment CodeWork started with.
8+
`local` is the default driver. It binds the agent directly to your machine: file tools use your real filesystem, starting in the current directory or in `--cwd`, and `bash` runs real commands through bash (`sh` when bash is not on `PATH`) with the environment CodeWork started with.
99

1010
Anything you have installed, such as `git`, `node`, or your test runner, the agent can use. There's nothing to provision, so it's the fastest sandbox, and the only one that works on your repository without copying anything.
1111

0 commit comments

Comments
 (0)