Are video world models ready for robot planning?
V-JEPA 2 plans in a learned latent space, while Cosmos generates future video that can stand in for rollouts or synthetic training data.
Has anyone used either to evaluate or plan for a real arm? What worked, and what broke?
Referenced repos
Indexed from Hugging Face and linked upstream, not hosted here.
Meta's self-supervised ViT-L video encoder, pretrained by predicting masked video regions in latent space rather than in pixels.
A 2B-parameter video world model that generates future video from text, image, or video prompts, with extra multiview checkpoints for robot and driving simulation.
A small reasoning vision-language model fine-tuned from Qwen3-VL-2B for physical common sense, such as judging object interactions, spatial relations, and whether a video is physically plausible.