Can CriticGPT Replace Hand-Designed Rewards for Robot Manipulation?

Podcast: Embodied AI 101 · Shaoqing Tan

Opublikowano: 2026-10-14

Can a multimodal language model replace hand-designed rewards for robot manipulation? This episode examines what CriticGPT actually learns, how its video preferences become training rewards, and where the experimental evidence stops. We cover three questions: whether strong trajectory-ranking accuracy survives ambiguous comparisons; whether the reported policy improvements are convincing given run-to-run variation; and what transfers when the critic is reused but a new reward model still has to be trained. The method adapts LLaVA-1.5 using automatically generated supervision from Meta-World, then connects pairwise feedback to reward modeling and DrQ-v2 policy learning. The experimental platform is a simulated tabletop manipulation arm — no...