Open validation evidence · v0.3.1

Know your robot data
before it trains your model.

A transparent health index for robotics datasets and recordings. Every outcome links back to deterministic Physlint rules—never a mystery score.

Explore evidence ↓Run it locally ↗

23observations

237rule runs

7,047real ROS messages

Evidence index · 31 Aug 2026

Robot data health, in public.

Release-gate rows (Public or Controlled) are the scoped claim. Survey rows are a stratified Hub sample, including classified false positives and fail-closed v2 screens. “Passed” means the applicable checks found no configured blocking issues.

#	Dataset / recording	Profile	Scale	Checks	Findings	Outcome	Evidence
01	Robomimic CAN PH ↗Panda · Public evidence	LeRobot	10 eps · 1.2k frames	11	0	Passed	Report ↗
02	Scout Earth Rover Mini ↗Earth Rover Mini · Public evidence	LeRobot	3 eps · 4.2k frames	14	0	Passed	Report ↗
03	SVLA SO-101 Pick & Place ↗SO-101 · Public evidence	LeRobot	50 eps · 11.9k frames	14	0	Passed	Report ↗
04	Sentinel Demo 09 ↗YAM humanoid · Public evidence	LeRobot	11 eps · 14.0k frames	14	0	Passed	Report ↗
05	Foxglove MCAP TenMessages conformance case ↗Generic channel · Public evidence	MCAP	10 messages	7	5	Issues found	Report ↗
06	RobotisAI ARX X5 elevator-button episode ↗ARX X5 · Public evidence	ROS 2	7047 messages	12	0	Passed	Report ↗
07	Controlled ROS 2 JointState recordingROS 2 fixture · Controlled evidence	ROS 2	3 messages	12	0	Passed	Report ↗
08	Controlled ROS 2 cadence and dimensions corruptionROS 2 fixture · Controlled evidence	ROS 2	3 messages	13	2	Issues found	Report ↗
09	PushT ↗2D pusher · Survey evidence · Likely frozen-frame false positive on a low-texture 2D sim.	LeRobot	206 eps · 25.6k frames	15	50	Issues found	Report ↗
10	xArm Lift Medium ↗xArm · Survey evidence	LeRobot	800 eps · 20.0k frames	15	0	Passed	Report ↗
11	Unitree H1 Two-Robot Greeting ↗Unitree H1 · Survey evidence	LeRobot	30 eps · 3.8k frames	15	0	Passed	Report ↗
12	SVLA SO-100 Pick & Place ↗SO-100 · Survey evidence	LeRobot	50 eps · 19.6k frames	15	0	Passed	Report ↗
13	Aloha Static Battery (episodes 0-4) ↗ALOHA · Survey evidence	LeRobot	5 eps · 3.0k frames	15	0	Passed	Report ↗
14	UR10e Robotiq 2F-85 ↗UR10e + Robotiq · Survey evidence · Four 1-second holds during motion; review, not a gate claim.	LeRobot	55 eps · 21.4k frames	15	4	Issues found	Report ↗
15	SO-101 Teleop Pick-and-Place ↗SO-101 · Survey evidence · Wrist-camera freezes at the 50-finding cap; likely static close-ups.	LeRobot	50 eps · 18.2k frames	15	50	Issues found	Report ↗
16	Reachy 2 Head Sample ↗Reachy 2 · Survey evidence · Confirmed defect: episode metadata names parquet shards the Hub does not publish.	LeRobot	10 eps · 3.1k frames	5	3	Issues found	Report ↗
17	LeKiwi Go2 ↗LeKiwi · Survey evidence	LeRobot	40 eps · 14.4k frames	15	0	Passed	Report ↗
18	LIBERO Spatial ↗Franka · Survey evidence · One 6-frame freeze at episode start; review, not a gate claim.	LeRobot	432 eps · 53.0k frames	15	1	Issues found	Report ↗
19	Koch arm gripper (v2.1) ↗Koch · Survey evidence · Koch embodiment is published as v2.1; keep as fail-closed evidence.	LeRobot	v2.1 metadata	0	0	Rejected	Report ↗
20	Koch pick-and-place Lego (v2.0) ↗Koch · Survey evidence · Official-style Koch teleop remains v2.0 on the Hub.	LeRobot	v2.0 metadata	0	0	Rejected	Report ↗
21	SO-101 pi0.5 base (v2.1) ↗SO-101 · Survey evidence · Large community SO-101 corpus still on v2.1; metadata-only screen.	LeRobot	v2.1 metadata	0	0	Rejected	Report ↗
22	Reachy 2 kitchen multimodal (v2.0) ↗Reachy 2 · Survey evidence · Full Reachy kitchen set is v2.0; the v3 head sample is in the check list.	LeRobot	v2.0 metadata	0	0	Rejected	Report ↗
23	RoboTwin 3D (v2.1) ↗ALOHA / RoboTwin · Survey evidence · Bimanual Aloha conversion at 50 FPS remains v2.1.	LeRobot	v2.1 metadata	0	0	Rejected	Report ↗

Order is the catalog, not a ranking. Release-gate rows are the public claim. Survey rows are compatibility observations and may include classified false positives.

Data-engineer workflow

A quality contract that can live in CI.

compare

Diff two dataset versions or JSON reports. Exit 1 only on new blocking findings.

physlint compare before/ after/

baseline

Accept a known fingerprint with author, reason, and optional expiry. New instances of the same rule still fail.

physlint baseline . --author ada --reason known

CI reports

JSON plus JUnit, SARIF, and a local HTML file. No images or raw samples are embedded.

--junit-output --sarif-output --html-output

plugins

Load a file or entry-point rule. Task-specific idle detection stays out of the default contract.

plugins: [idle_prefix.py:IdlePrefixRule]

How to read the index

Evidence over
leaderboard theater.

Profile-aware

LeRobot, generic MCAP, and ROS 2 use separate contracts. Survey rows never collapse into the release-gate claim or a Hub quality score.

Reproducible

Sources are revision-pinned. Reports preserve configuration digests, rule versions, and SHA-256 fingerprints.

Privacy-safe

Only sanitized evidence is published. Raw robot recordings, images, and full samples stay with their owners.

Open to challenge

Every finding identifies its rule and evidence. Disagree? Re-run it, inspect the report, or improve the rule.

Bring your own recording

Make your data quality
a visible engineering practice.

pip install physlintGet started ↗

Know your robot databefore it trains your model.