Skip to content

Public benchmark runtime

The public runtime uses tower_sim.bench, benchmark 2026.11.1, data version benchmark-2026.11.1, and request schema tower-public-benchmark-v2. The three wheels are tower-sim==0.1.0, tower-eval==0.1.0, and nero-kinematics==0.2.0. A release manifest identifies the exact build.

Install the pinned SDK

Run from the checkout root. Install uv and Micromamba first (to create the separate conda-forge prefixes). Isaac uses Python 3.12:

export PYTHONDONTWRITEBYTECODE=1
uv venv --python 3.12 .venv-isaac61
uv pip install --python .venv-isaac61/bin/python "isaacsim[all,extscache]==6.1.0" --default-index https://pypi.org/simple --index https://pypi.nvidia.com --index-strategy unsafe-best-match

Both public indexes and unsafe-best-match are required: first-index resolution can hide the required mujoco-usd-converter==0.5.0 candidate on PyPI. Review and accept NVIDIA's license as described in the Isaac guide.

X11 system libraries

On Debian/Ubuntu, install the system prerequisite with sudo apt-get install libxt6. It supplies libXt.so.6, needed by the MaterialX/USD plugins even on headless hosts. After installing it, check the interpreter:

uv run python tools/simulator/benchmark.py check-environment --isaac-python .venv-isaac61/bin/python

Without root, install it in a separate conda-forge prefix:

micromamba create --prefix "$PWD/.isaac-system-libs" --override-channels -c conda-forge xorg-libxt -y
mkdir -p .isaac-x11
ln -sfn "$PWD/.isaac-system-libs/lib/libXt.so.6" .isaac-x11/libXt.so.6
LD_LIBRARY_PATH="$PWD/.isaac-x11" ldd .isaac-x11/libXt.so.6
LD_LIBRARY_PATH="$PWD/.isaac-x11" uv run python tools/simulator/benchmark.py check-environment --isaac-python .venv-isaac61/bin/python

If ldd reports missing X11 dependencies, add only those libraries as symlinks from .isaac-system-libs/lib into .isaac-x11, then repeat ldd and the check. For example, a missing libSM.so.6 needs ln -sfn "$PWD/.isaac-system-libs/lib/libSM.so.6" .isaac-x11/libSM.so.6. Do not expose the entire conda library directory: it also contains compiler runtimes such as libgcc and libgomp that can override Isaac libraries.

Use this X11-only, process-scoped LD_LIBRARY_PATH for benchmark invocations. The IK child inherits it too; validate the native acceptance tests and release identity probe with this same value. Do not add it to shell startup files. The check loads libMaterialXRenderHw.so, libMaterialXRenderGlsl.so and libusd_usdBakeMtlx.so without starting Kit or creating GPU contexts. A metadata-only version match is insufficient.

The selected GPU chooses the render and physics device. The launcher deliberately unsets CUDA_VISIBLE_DEVICES; the SDK can create small CUDA contexts on other visible GPUs. On shared hosts, use whole-host reservation or container-level GPU isolation; CUDA_VISIBLE_DEVICES alone does not isolate this runtime.

For the independent native solver, follow the complete IK install and acceptance recipe.

Build and verify

Build from a fresh, clean committed checkout. Keep caches outside bound trees:

export PYTHONDONTWRITEBYTECODE=1
GIT_LFS_SKIP_SMUDGE=1 git submodule update --init third_party/XPolicyLab
uv build --wheel --out-dir dist packages/nero-kinematics
uv build --wheel --out-dir dist packages/tower-sim
uv build --wheel --out-dir dist src/tower-eval
uv run python -m scripts.maintainer.check_public_tree --wheel dist/tower_sim-0.1.0-py3-none-any.whl
uv run python -m scripts.maintainer.candidate_release --output dist/release-manifest.json

Install all three wheels in isolated package directories used by both the Isaac interpreter and the independent IK interpreter. Calibration, runtime profiles, shared IK specification and camera configuration are installed package data. Physical assets and evaluation scenes use explicit external roots.

The release binds the tracked tools/simulator/ and scripts/ files and every wheel payload. It also binds third_party/XPolicyLab/client_server/ at the pinned gitlink, reading committed bytes from the submodule repository. An initialized submodule with its own Git top-level and the exact pinned revision is required; an empty submodule directory is rejected explicitly. All tracked files must remain clean. Untracked files, including bytecode, are rejected inside src/, packages/, tools/simulator/ and scripts/. Environments and outputs can live outside those bound trees. Installed package file sets and bytes must equal their wheels. Startup validates loaded sources before GPU work; final source receipts repeat that validation. Unsupported planning entry points, private archive compatibility modules and private identifiers are rejected in the release tree and wheel payloads.

Interpreter and SDK roots are explicit. They cannot overlap bound source trees. Third-party packages under those roots are admitted by location; the dependency manifest independently binds selected numerical and SDK dependencies. The IK manifest pins Pinocchio 3.8.0, CasADi 3.7.2, pin-pink 3.4.0 and NumPy 1.26.4. Conda distributions are checked against their installed package manifests.

Evaluation data

python tools/simulator/benchmark_assets.py download --assets /path/to/assets
python tools/simulator/benchmark.py verify --assets /path/to/assets
python tools/simulator/benchmark.py describe
python tools/simulator/benchmark_portable.py prepare-request --help
python tools/simulator/benchmark.py run --help

Each task has fifty fixed scenes, scene-000 through scene-049. The request field seeds selects integer case numbers 0 through 49. It does not create scenes. The default data directory is <assets>/benchmark-2026.11.1. Downloads use the anonymous pinned TOWER-Assets revision in the release lock. benchmark.py verify checks physical assets and evaluation data at that default location. In contrast, benchmark_assets.py verify --assets ... checks only physical assets unless --data-root is supplied. Use --data-root to select an existing evaluation-data directory at a different location. CPU verification without assets checks metadata and reports zero verified payloads.

Every trial records its benchmark, data version, request schema, package versions, release manifest hash, commit, and data manifest hash. Offline commands leave release identifiers null because they do not authorize simulation.

Control and review contract

Preparation binds protocol, scenes, assets, policy identity, release, dependencies and, for maintainer qualification, current posture-review evidence. Reject reverse 180-degree symmetric grasps; never retry with a reversed target. Preserve each task's contact geometry. Review planned approach/carry trajectories and recorded approach phases before accepting a posture-reviewed run. An old pass or the absence of a symmetry flag is insufficient. Maintainers supply the posture-review document with --pilot-review.

Command bounds remain the real normal IK soft bounds intersected with simulation URDF bounds, inset by 1.5 degrees, with J2 Tower hard limits inset by 5 degrees. J2 and coordinate conversion, the separate 0.5-degree observation tolerance, and real hardware protections retain their existing meanings. Learned-policy runs reject command_replay. The exact-flip tripwire cannot replace posture review.

Option abbreviations are disabled. Runtime evidence does not establish suitability for hardware use. Public scene v2 and historical recording readers preserve their physical/scoring contracts. Private source history and maintainer replay are not public CLI features.

See the canonical control safety contract for the scope of runtime posture checks and the distinct control limits.

The WebSocket extra requires pydantic>=2.11,<3, whose dependencies include typing-inspection. This makes the installed dependency manifest consistent with the supported extra; older Pydantic installations must be upgraded.

Public benchmark runs do not require --pilot-review. Recorded TCP posture is reported in posture-flags.json, independently of scores and action mode. The automatic flag also covers joint commands and near-half-turn grasps; it does not replace maintainer contact and approach/carry review. See posture flags.