[LABS]

Where multimodal
reasoning is built.

Reka Labs is our core research engine — developing world language action models and the foundational science behind the next generation of multimodal AI.

New · PhysicalRealismBench-U

Attributable Physical Realism

Evaluation for Video World Models

Reka AI Research · April 2026 · Track U: Physical Understanding

We release PhysicalRealismBench — a physics-first benchmark for evaluating physical understanding in VLMs — built on 550 synthetic videos with known ground truth. Even state-of-the-art models fail at fundamental physical reasoning tasks: the best-performing model (Gemini 3.1 Pro) achieves an F1 of only 39.5%. Our findings show that VLMs skip frames, rely on spatial heuristics, and miss violations any human would catch immediately.

Read the blog post →

View leaderboard

Dataset & eval pipeline

[BENCHMARKS]

Dataset at a glance

Total videos

550

Synthetic, physics-correct

Matched pairs

250

Valid + violation per scene

Violation categories

5

Grounded in fundamental physics

Best model F1

39.5%

Gemini 3.1 Pro · joint metric

[PHYSICS ONTOLOGY]

Five violation categories tested

Spontaneous impulse

Motion without cause — violates Newton's first law.

e.g. object accelerates with no applied force

Random disappearance

Objects vanish unexpectedly — violates conservation and permanence.

e.g. object present in frame N, absent in N+1

Pass-through

Objects interpenetrate solid surfaces — violates impenetrability.

e.g. ball passes through a wall

Gravity violation

Objects hover or fall inconsistently — violates gravity and support.

e.g. unsupported object remains suspended

[KEY FINDINGS]

Why VLMs fail at physical reasoning

INSIGHT 01
VLMs don't examine every frame
When random numbers were overlaid on each frame, models recognised 80–100% in video mode but 100% when frames were sent as individual images. When consecutive (predictable) numbers were used, video accuracy also hit 100% — strongly suggesting the model interpolates rather than inspecting each frame.
Temporal attention
INSIGHT 02
Border proximity triggers false reasoning
When an object near the image border disappears, VLMs frequently claim it "left the scene" — even when trajectory analysis shows no outward motion, and linear extrapolation places the object still inside the frame at the moment it vanishes.
Spatial heuristics
INSIGHT 03
Position augmentation substantially helps
Providing per-frame object positions as (x, y, t) tuples in the prompt — derived from segmentation masks — substantially improves performance. This suggests VLMs' vision-based physical reasoning is markedly weaker than their text-based reasoning: the physics knowledge exists in the language model but doesn't transfer cleanly to vision.
Text vs. vision reasoning
INSIGHT 04
Motion detection has an asymmetric temporal bias
VLMs fail to detect motion when an object lingers at position A for many frames but appears at position B for only a few. The reverse (short A, long B) is detected correctly. Out of 108 tested configurations, 12 produced failures — all with this asymmetric pattern — suggesting a recency or salience bias in temporal attention.
Temporal bias