Reading, Writing, Sharing, and Interacting with the World through Video

This survey unifies video understanding and generation through world modeling. It views an observed video as a partial, lossy, temporally ordered projection of a latent temporal world, and organizes the literature around four operations: reading, writing, sharing, and interacting. It traces a shift from clip-level recognition and perceptual fidelity toward persistent state, controllable evolution, physical and causal validity, and interaction-grounded evaluation, and distills open challenges.

Key takeaways

• Organizes video research around four operations: reading, writing, sharing, and interacting with latent temporal world states.
• Distinguishes temporal coherence in video sequences from consistency in the underlying world state.
• Traces a shift in emphasis from clip-level recognition toward persistent state and controllable evolution.
• Calls for evaluation grounded in interaction and downstream decisions, and distills open challenges.

Why it matters

Framing video understanding and generation under one world-modeling lens could help researchers align evaluation and architectures across tasks that have developed separately.

Read with care

Source claims not independently checked; as a survey, its taxonomy reflects the authors' organizational choices rather than a measured result.