Google DeepMind team has recently launched a new GenCeption model, attempting to reverse the traditional video generation AI into a visual analysis engine that can deeply understand the real world. Unlike previous tasks such as segmentation and depth estimation, which were independent, GenCeption can efficiently complete five core visual tasks simultaneously with a single model: depth estimation, image segmentation, 3D pose estimation, etc.

image.png

The model is trained based on Alibaba's open-source Tongyi Wanxiang Wan2.1 framework, and it can make predictions in a single forward pass, greatly improving processing speed. Users just need to input simple text prompts, and the model can accurately output rich information such as depth maps, segmentation masks, and camera motion trajectories.

Training with single-person synthetic data, astonishing details preserved to the hair

Interestingly, the training set of GenCeption only contains about 7,500 single-person synthetic videos rendered by Blender. However, the model demonstrates strong generalization ability, not only handling multi-person videos in the real world smoothly, but also easily transferring to animal and robotic categories.

In actual testing, the small model processes 81 frames in about 6 seconds, and the large model with 14 billion parameters takes about 10 seconds. Its detail restoration even exceeds the original synthetic images used for training, clearly preserving the edges of cat whiskers and individual strands of hair.