Skip to content

Offline scoring

Run the scorer on a complete saved recording, writing to a new output file:

tower score /path/to/run --task lateral-insertion-2 --ruleset bench-2026.11 --input-path files --output /path/to/new-score.json

Keep the input recording unchanged. The file loader validates and hashes its recording inputs; reports identify the ruleset, input path, evaluator version and canonical task id. Missing or incomplete evidence must not be reported as successful evaluation. See scoring reference.

Each run needs episode_spec.json, status.json, episode/states.hdf5, finger-link-poses.npz, finger-feedback.npz and finger_geometry.json, including each finger’s distal_link_corners_m. Timestamps must align and gaps cannot exceed 100 ms. Inputs are opened read-only. Existing automatic events and human ratings are never detector inputs.

Aggregate saved benchmark runs

Use tools/simulator/benchmark_run.py aggregate --runs DIR... --output DIR with complete run roots or individual batch directories. Keep the entire run bundle, including its selected-set manifest and retained attempt inputs. Older recordings without these records require explicit migration and cannot establish requested coverage with this command. The command checks score and state hashes, computes metrics over complete episodes, excludes infrastructure failures and writes informational posture flags under the chosen output only. Input evidence is opened read-only; relocation uses retained hashes, never stale absolute paths. It does not rewrite the per-scene scorer reports. The standalone benchmark.py score export adds provenance fields and a public envelope, so its full JSON bytes differ from a per-scene report even when scorer content matches.