new, fmt and variants

The authoring verbs: start a file from another, keep it in the one layout, and list what it lets a tool vary.

new

$ sexpgpu new my-run.sx --from base.sx --knob layers=8 --knob width=512
wrote my-run.sx from base.sx: 116 parameters, 300 steps

new <target.sx> --from <file> [--knob k=v]... copies a run or a template:

• rewrites the defknobs defaults named with --knob;
• notes the source and the date in a header;
• recomputes every relative require and loader path for the new directory;
• compiles the result before reporting it, so a successful new is a file that passes check.

Source edits use the compiler's reader. Comments and documentation strings are not knob declarations, and quoted or nested defaults are replaced as whole forms. Path edits preserve comments and decode escaped filenames.

--from is required, and a target that exists is refused (exit 2), as is a --knob the file does not declare. A copy that does not compile is written anyway, reported as a failure (exit 1) and left on disk to look at.

Copy only when the change is not a knob of an existing run; when it is, --set or a defvariant is the answer. See knobs.

fmt

$ sexpgpu fmt my-run.sx lib/mine.sx
$ sexpgpu fmt --check my-run.sx; echo $?
0

Rewrites each named file in the one layout there is; every non-flag argument is a file. --check prints each file that would change and exits 1 without touching anything. --stdout prints one file's formatted text instead of writing it.

The layout: two spaces, 80 columns. A form goes on one line when it fits with the brackets that close after it; otherwise its head decides. A body goes two spaces under its head. A call with keyword arguments takes one :key value pair per line; a call without them fills the line. A form that breaks puts one binding or one vector element per line. Comments stay where they were, and one blank line inside a body is kept. There is nothing to configure.

A rewrite is refused as E-FMT-001, and the file left as it was, unless formatting the result again leaves it alone and it reads as the same datums. That is a formatter bug, not a problem with your file.

variants

$ sexpgpu variants my-run.sx
knobs:
  lr      0.0015  peak learning rate of the block matrices, the default group
  layers  4       transformer blocks
  width   256     model width
  seq     1024    sequence length in tokens; a window holds seq + 1
  seed    1       the run seed; every random stream folds it in
variants:
  small  layers=2 width=128
  smoke  layers=2 width=128 seq=64
sweeps:
  seeds  3 points over seed

Everything the file lets a tool vary, on standard output: every knob with its default and doc, every variant with its overrides, every sweep with its size. See knobs, variants and sweeps.

Related: the command line, check.