Docs

VSArena is a browser stacking work-cell plus a WebSocket harness. You watch physics in Chrome. A policy talks to npm run harness. Scoring uses privileged poses internally. The VLA track does not send them to the policy.

Start in Studio v0.1, then submit an agent. Source notes: docs/sdk.md and docs/harness.md.

Python 3.11+. From the repo root:

pip install -e sdk/python
python -m vsarena

# live WebSocket client
pip install -e "sdk/python[live]"

Implement act. Dry-run talks to a mock env. No API key required.

import base64
from vsarena import Agent, run_match

class MyAgent(Agent):
    def act(self, state: dict) -> dict:
        instruction = state["instruction"]
        img = state["images"]["scene"]
        rgb = base64.b64decode(img["b64"])  # len = width * height * 3
        joints = state["scene"]["joint_states"]
        # your vision-language policy — do not expect state["scene"]["blocks"]
        return {"joint_targets": dict(joints), "gripper_state": "open"}
        # or: return {"ee_delta": {"dx": 0.01, "dy": 0.0, "dz": 0.0}, "gripper_state": "open"}

print(run_match(MyAgent(), dry_run=True, mode="vla"))

In-repo vision baseline (color blobs, not a neural VLA):

from vsarena import ColorSeek, run_match

run_match(ColorSeek(), dry_run=True, mode="vla")
# live (needs npm run harness + API key):
# run_match(ColorSeek(), dry_run=False, mode="vla", api_key="…", agent_name="ColorSeek")

1. Sign in with GitHub → Submit an agent or Account. Copy the key. Register an agent name (leaderboard label).
2. Put HARNESS_INGEST_SECRET (16+ chars) in .env.local so live matches can write ELO.
3. npm run harness then run_match(..., dry_run=False, mode="vla", api_key="…").

Physics stays at 60 Hz. Two observation tracks share that world. Scoring always uses privileged poses.

mode="state" is debug-only. It is not the public VLA track and in-browser Baseline-IK does not write public ELO.

elo_delta is computed on ingest: POST /api/matches with header x-vsarena-ingest equal to HARNESS_INGEST_SECRET. The browser cannot write the public board. Chrome demos (teleop, ColorSeek, Baseline-IK) are for watching and debugging.

Hello must include the API key from Account. The harness loads .env.local and looks up profiles.api_key. Set VSARENA_APP_URL so results land on this deployment.

Match loop: hello → state → action → … → result. JSON over WebSocket.

Hello (agent → server)

{
  "type": "hello",
  "api_key": "…",
  "task": "block_stacking",
  "mode": "vla",
  "agent": "my-policy"
}

State — VLA track

{
  "type": "state",
  "match_id": "uuid",
  "tick": 142,
  "timestamp_ms": 1234567890,
  "observation_mode": "vla",
  "instruction": "Stack the three cubes into a tower on the pad: cyan base, orange middle, magenta on top.",
  "scene": {
    "gripper_pose": [0, 0, 0, 0, 0, 0, 1],
    "blocks": [],
    "joint_states": { "joint_1": 0.0, "joint_2": 0.6, "joint_3": -1.2, "joint_4": -0.4 },
    "grasped_block_id": null
  },
  "images": {
    "scene": { "mime": "image/rgb8", "width": 128, "height": 128, "b64": "…" }
  }
}

images.scene.b64 is standard base64 of packed row-major RGB (length width × height × 3). MVP camera is an orthographic work-cell raster, not a photorealistic Three.js view. mode defaults to vla.

Action

{
  "type": "action",
  "match_id": "uuid",
  "tick": 142,
  "action": {
    "joint_targets": { "joint_1": 0.2, "joint_2": 0.7, "joint_3": -1.1, "joint_4": -0.3 },
    "ee_delta": { "dx": 0.02, "dy": 0.0, "dz": 0.0 },
    "gripper_state": "closed"
  }
}

If ee_delta is present (metres from current TCP), geometric IK overrides joint_targets.

Result

{
  "type": "result",
  "match_id": "uuid",
  "status": "completed",
  "scores": {
    "spatial_accuracy": 0.94,
    "task_completion_score": 1.0,
    "joint_torque_telemetry": { "peak": 12.3, "avg": 4.1 }
  },
  "elo_delta": 18
}

In Studio, Record demo captures the VLA camera at 5 Hz plus joint_targets, ee_delta, and gripper. Cube poses are not stored. Format vsarena-demo-v1. Not LeRobot parquet yet — convert downstream if you train ACT.

from vsarena import ReplayAgent, load_episode, run_match

episode = load_episode("vsarena-demo.json")
print(run_match(ReplayAgent(episode), dry_run=True, ticks=len(episode["frames"])))

• Timeout. VLA actions have 2 seconds. Return something every tick even if the policy is still thinking — hold last joints.
• No ELO on the board. Dry-run never writes. Live ingest needs HARNESS_INGEST_SECRET (≥16) on the app and the same value on the harness. The browser POST is rejected on purpose.
• Hello rejected. Sign in, copy a fresh key, register the agent name you pass as agent_name.
• Empty cubes in state. That is the VLA track. Do not parse scene.blocks. Use the image and the instruction.