kyanite

A parallel command runner written in Rust, in the spirit of GNU parallel and xargs -P. kyanite reads lines from stdin and runs a command template once per line, with field access, sed-style substitution, and regex captures built into the template syntax.

Features

• Parallel execution: one job per CPU by default (-j to change)
• Template expansion: whole line, fields, field ranges, sed-style substitution, and regex captures
• Safe by default: substituted values are shell-quoted, so filenames with spaces or $(...) are passed as plain arguments
• NUL-separated input: -0 pairs with find -print0 for filenames containing anything, even newlines
• Custom placeholders: use [], @, <<>>, or anything else when {} clashes with your command
• Ordered output: -k prints results in input order
• Clean output: each job's output is printed in one piece, stdout and stderr stay on their own streams
• Meaningful exit status: non-zero when any job fails
• Graceful stop: Ctrl+C or SIGTERM stops new jobs and lets running ones finish; a second signal kills them
• Pipe friendly: stops early when the reader goes away (kyanite ... | head)
• Dry run: -n prints the commands instead of running them

Install

cargo install --path .

Prebuilt binaries for Linux (static x86_64, aarch64), macOS, and Windows are attached to each GitHub release, along with a SHA256SUMS file. Commands run through sh -c, so on Windows sh must be on your PATH (e.g. from Git for Windows or MSYS2).

Quick start

# Convert every mp4 in the current directory
ls *.mp4 | kyanite 'ffmpeg -i {} {s/\.mp4$/.mp3/}'

# Same thing with a regex capture
ls *.mp4 | kyanite 'ffmpeg -i {} {/(.*)\.mp4$/1}.mp3'

# Four jobs at a time
ls *.txt | kyanite -j 4 'wc -l {}'

# Fields, with output kept in input order
ps aux | kyanite -k 'echo pid={2} cmd={11+}'

# Any filename, even with spaces or newlines
find . -name '*.log' -print0 | kyanite -0 'gzip {}'

# Preview the commands without running them
ls *.mp4 | kyanite -n 'ffmpeg -i {} {s/\.mp4$/.mp3/}'

Template syntax

Pattern	Expands to	Example
{}	The whole input line	echo {}
{N}	Field N, counting from 1	echo {1} {2}
{N+}	Field N through the last field	echo {3+}
{N-}	Fields 1 through N	echo {3-}
{s/RE/REPL/F}	The line with RE replaced by REPL; flags g (all), i (case)	{s/\.mp4$/.mp3/}
{/RE/N}	Capture group N of RE matched against the line (0 = whole match)	{/(.+)\.(.+)/2}

• Fields are split on runs of whitespace, like awk. Use --field-separator to split on an exact string instead.
• Regexes use Rust regex syntax. In REPL, refer to groups as $1 or ${name} (not \1).
• Write \/ for a literal slash inside RE or REPL.
• A missing field or a regex that doesn't match expands to an empty value.
• Text that isn't valid template syntax is left alone, so awk '{print $1}' still works.
• A malformed regex or unknown flag is reported before any job runs.

Write templates in single quotes so your shell passes them through untouched.

Quoting

Every substituted value is shell-quoted before the command runs, the way GNU parallel does it:

$ printf 'my file.txt\n' | kyanite -n 'cat {}'
cat 'my file.txt'

Because of this, don't wrap placeholders in quotes yourself ("{}" would pass the quote characters through). If you really want values inserted verbatim, for example to splice in a shell fragment, pass --no-quote.

Custom placeholders

Change the placeholder with -I / --input. A one-character placeholder wraps expressions on both sides; a longer one must have an even length and is split in half to get its opening and closing delimiters.

Placeholder	Whole line	Field 1	Substitution
{}	{}	{1}	{s/a/b/}
[]	[]	[1]	[s/a/b/]
@	@	@1@	@s/a/b/@
<<>>	<<>>	<<1>>	<<s/a/b/>>

ls *.mp4 | kyanite -I '[]' 'ffmpeg -i [] [s/\.mp4$/.mp3/]'

Options

Flag	Description
-j, --jobs <N>	Number of jobs to run at once (default: CPU count)
-k, --keep-order	Print output in input order instead of completion order
-n, --dry-run	Print the expanded commands instead of running them
-v, --verbose	Log job progress to stderr
-0, --null	Split input on NUL bytes instead of newlines
--max-lines <N>	Stop after reading N non-blank input lines (default: 0, no limit; --max-jobs still works)
-I, --input <PH>	Placeholder for the input line (default: {})
--field-separator <SEP>	Split fields on this exact string instead of whitespace
--no-quote	Insert substituted values verbatim instead of shell-quoting

kyanite --help shows the same reference, including template syntax and examples.

Behavior

• Blank (or whitespace-only) input lines are skipped; with -0, only empty items are. Input that isn't valid UTF-8 is skipped with a warning.
• Each job's stdout and stderr are captured and printed once the job finishes, so output from parallel jobs never interleaves.
• When a job fails, kyanite reports it on stderr and keeps running the remaining jobs.
• On Ctrl+C or SIGTERM, no new jobs start and running jobs are waited for. A second signal kills the running jobs and exits. Killing a job stops its sh, and with it a single-command job, but processes started by a compound command (a; b) may keep running.
• If stdout is closed early (for example kyanite ... | head -1), no new jobs start and kyanite exits once the running ones finish.

Exit status

Code	Meaning
0	Every job succeeded
1	At least one job failed, or an input line was skipped
2	Invalid arguments or template
130	Interrupted (Ctrl+C / SIGINT)
141	Stdout was closed early (e.g. piped into head)
143	Terminated (SIGTERM)

Examples

Batch downloads

cat urls.txt | kyanite -j 8 'curl -sSLo {/[^\/]+$/0} {}'

CSV fields

cat users.csv | kyanite --field-separator , 'echo name={1} email={2}'

Log fields

cat access.log | kyanite -k 'echo ip={1} time={4}'

Extensions

ls | kyanite 'echo {/(.+)\.(.+)/1} has extension {/(.+)\.(.+)/2}'

Development

cargo build
cargo test
cargo clippy --all-targets -- -D warnings
cargo fmt --check

License

MIT