Video models are getting better all the time. The latest models give you plenty of control over camera movement. CineMaster lets you manipulate objects and camera in 3D space. With this approach, it is possible to make videos of men walking in front of another object, cars passing one another, a hot balloon circling a tower, and a lot more. As the researchers explain:
To achieve this, CineMaster operates in two stages. In the first stage, we design an interactive workflow that allows users to intuitively construct 3D-aware conditional signals by positioning object bounding boxes and defining camera movements within the 3D space. In the second stage, these control signals—comprising rendered depth maps, camera trajectories and object class labels—serve as the guidance for a text-to-video diffusion model, ensuring to generate the user-intended video content.
[HT]
