All pages
The command line
One binary, sexpgpu, does four things: author a run (new, fmt,
variants, watch), check it (check, explain, diff, ir, eval),
run it (run, sweep), evaluate it from its checkpoints (evaluate) and pack it for another machine (bundle).
doctor says whether a run can start here, and checkpoint what a
checkpoint holds. There is no run management: a
run is one process, and whatever starts it (a terminal, a job runner, a
scheduler) owns the rest.
Verbs
| verb | arguments | what it does | page |
|---|---|---|---|
check | <file.sx> [--json] | compile and report diagnostics | check |
watch | <file.sx> | check again on every change, until Ctrl-C | check |
explain | <file.sx> [--device cuda] [--json] | what the compiler made of the file, and with a device what the executor makes of it | explain |
diff | <a.sx> <b.sx> [--json] | what changed between two runs, or two selections of one | explain |
ir | <file.sx> | the experiment IR as JSON | explain |
eval | <file.sx> | the value of each top-level form | explain |
fmt | <file.sx>... [--check] [--stdout] | rewrite files in the one fixed layout | new |
variants | <file.sx> | the knobs, variants and sweeps the file declares | new |
new | <target.sx> --from <file> [--knob k=v]... | start a run from a template or another run | new |
run | <file.sx> | run the experiment | run |
evaluate | <file.sx> --checkpoint <dir|s3://...|latest> [--pass P]... [--follow] [--poll S] | run the file's passes from a run's checkpoints, into the run's metrics at each checkpoint's step | evaluation |
sweep | <file.sx> <sweep> [--dry-run] [--only N] | run every point of a declared sweep | sweep |
bundle | <file.sx> -o <dir> [--binary <sexpgpu>] | one directory that runs the experiment anywhere | bundle |
checkpoint | <dir>|s3://bucket/prefix/step-<n> | what a checkpoint holds: its experiment, step and tensors | checkpoints |
doctor | — | can a run start here, and with what | doctor |
The selection
check, explain, diff, watch, ir, run and bundle take:
| flag | meaning |
|---|---|
--variant <name> | select a declared variant |
--set <knob>=<value> | set a knob, repeatable |
--diagnostics <glob,glob> | replaces defrun :diagnostics; '' selects none. See diagnostics |
For diff the selection applies to b only. sweep takes all three:
--set pins a knob across every point (pinning an axis of the sweep exits
2), --variant applies when the sweep names no variant, --diagnostics
applies to every point. See knobs.
Standard streams
Standard output carries a verb's answer: JSON, IR, formatted source, a
table. Standard error carries what a run says while it works: diagnostics,
progress, the timing block, the done: block. run keeps standard output
free unless asked for data: --json puts the summary there, and
--metrics stdout makes it the metrics stream. With both, the summary is
the stream's last line.
Exit codes
The same for every verb:
| code | when |
|---|---|
0 | it worked |
1 | the thing failed: diagnostics (an unknown --variant or --set knob among them), a run that could not start or died, an evaluate refused by its checkpoint or asked for an undeclared pass, a fmt --check that would rewrite, a doctor that says a run cannot start, a bundle whose data is not on this machine or whose run would not read its bundled data |
2 | the command line was wrong: an unknown verb, a missing argument, a --set without =, a --steps or --eval-every that is not a whole number, an --evaluate other than inline or offline or one whose checkpoints are too coarse for a pass, a sweep the file does not declare or an --only past its end, a pinned sweep axis, an evaluate without --checkpoint or with a --poll that is not a positive number, a --knob the file does not declare, a new or bundle target that exists |
128 + n | run or evaluate stopped by signal n: 129 SIGHUP, 130 SIGINT, 143 SIGTERM. Any other verb whose standard output closes early, sexpgpu ir x.sx | head, ends by SIGPIPE, 141, as a Unix tool does; run, evaluate and sweep fail on the write instead, and run and sweep say so in their status file |
Paths
A path inside a .sx file never depends on the working directory: a
require resolves against the file that holds it, a loader path against
the experiment file. Only paths on the command line are relative to where
you are.
Related: environment, working as an agent.