AI video models are getting more powerful all the time. The latest ones give you a bit of control over the camera angles. MagicMotion aims to take that to the next level. It is a controllable image-to-video- model that allows “trajectory control through three levels of conditions from dense to sparse: masks, boxes, and sparse boxes.” As the researchers explain:
MagicMotion uses a dense-to-sparse training procedure to train the model with different levels of trajectory conditions: mask, box and sparse box (less than 10 frames have box trajectories provided). Experiments show that the model can leverage the knowledge learned in the previous stage to achieve better performance than training from scratch.
MagicMotion can animate objects along your defined trajectories while maintaining consistentcy.
[HT] [credit: Quanhao Li, Zhen Xing, Rui Wang, Hui Zhang, Qi Dai, and Zuxuan Wu]
