Reka Responsible AI, Model Risk, Ethics & Governance Framework

How well does the model understand the physical world?

How well does the model understand the physical world?

How well does the model understand the physical world?

EPISODE 02

·

·

13:53

Marianna and Roberto have been quietly working on something that has been bugging them for a while: how do you tell whether a model actually understands physics, or has only learned what physics tends to look like?

The answer was to build the violations on purpose — objects vanishing, bodies passing through walls, things floating without support — and then measure which ones frontier VLMs notice, and which ones they walk straight past.

[TRANSCRIPT]

Test

EPISODE DETAILS

SERIES

Working Notes

RUNTIME

13:53

PUBLISHED

IN THIS ONE

Marianna, Roberto

Physical realismEvaluation
Read the research