S-exp GPU
All pages
Docs · The command lineMarkdown

The command line

One binary, sexpgpu, does four things: author a run (new, fmt, variants, watch), check it (check, explain, diff, ir, eval), run it (run, sweep), evaluate it from its checkpoints (evaluate) and pack it for another machine (bundle). doctor says whether a run can start here, and checkpoint what a checkpoint holds. There is no run management: a run is one process, and whatever starts it (a terminal, a job runner, a scheduler) owns the rest.

Verbs

verbargumentswhat it doespage
check<file.sx> [--json]compile and report diagnosticscheck
watch<file.sx>check again on every change, until Ctrl-Ccheck
explain<file.sx> [--device cuda] [--json]what the compiler made of the file, and with a device what the executor makes of itexplain
diff<a.sx> <b.sx> [--json]what changed between two runs, or two selections of oneexplain
ir<file.sx>the experiment IR as JSONexplain
eval<file.sx>the value of each top-level formexplain
fmt<file.sx>... [--check] [--stdout]rewrite files in the one fixed layoutnew
variants<file.sx>the knobs, variants and sweeps the file declaresnew
new<target.sx> --from <file> [--knob k=v]...start a run from a template or another runnew
run<file.sx>run the experimentrun
evaluate<file.sx> --checkpoint <dir|s3://...|latest> [--pass P]... [--follow] [--poll S]run the file's passes from a run's checkpoints, into the run's metrics at each checkpoint's stepevaluation
sweep<file.sx> <sweep> [--dry-run] [--only N]run every point of a declared sweepsweep
bundle<file.sx> -o <dir> [--binary <sexpgpu>]one directory that runs the experiment anywherebundle
checkpoint<dir>|s3://bucket/prefix/step-<n>what a checkpoint holds: its experiment, step and tensorscheckpoints
doctor—can a run start here, and with whatdoctor

The selection

check, explain, diff, watch, ir, run and bundle take:

flagmeaning
--variant <name>select a declared variant
--set <knob>=<value>set a knob, repeatable
--diagnostics <glob,glob>replaces defrun :diagnostics; '' selects none. See diagnostics

For diff the selection applies to b only. sweep takes all three: --set pins a knob across every point (pinning an axis of the sweep exits 2), --variant applies when the sweep names no variant, --diagnostics applies to every point. See knobs.

Standard streams

Standard output carries a verb's answer: JSON, IR, formatted source, a table. Standard error carries what a run says while it works: diagnostics, progress, the timing block, the done: block. run keeps standard output free unless asked for data: --json puts the summary there, and --metrics stdout makes it the metrics stream. With both, the summary is the stream's last line.

Exit codes

The same for every verb:

codewhen
0it worked
1the thing failed: diagnostics (an unknown --variant or --set knob among them), a run that could not start or died, an evaluate refused by its checkpoint or asked for an undeclared pass, a fmt --check that would rewrite, a doctor that says a run cannot start, a bundle whose data is not on this machine or whose run would not read its bundled data
2the command line was wrong: an unknown verb, a missing argument, a --set without =, a --steps or --eval-every that is not a whole number, an --evaluate other than inline or offline or one whose checkpoints are too coarse for a pass, a sweep the file does not declare or an --only past its end, a pinned sweep axis, an evaluate without --checkpoint or with a --poll that is not a positive number, a --knob the file does not declare, a new or bundle target that exists
128 + nrun or evaluate stopped by signal n: 129 SIGHUP, 130 SIGINT, 143 SIGTERM. Any other verb whose standard output closes early, sexpgpu ir x.sx | head, ends by SIGPIPE, 141, as a Unix tool does; run, evaluate and sweep fail on the write instead, and run and sweep say so in their status file

Paths

A path inside a .sx file never depends on the working directory: a require resolves against the file that holds it, a loader path against the experiment file. Only paths on the command line are relative to where you are.

Related: environment, working as an agent.