Google DeepMind has recently released a new model called GenCeption, which is built upon a pre-trained video generation model and is designed to address classic computer vision tasks such as depth estimation, image segmentation, and 3D pose estimation. It has achieved near or superior performance compared to current leading models on multiple benchmark tests. Notably, the training data required for GenCeption is significantly smaller than that of traditional specialized visual models.

Currently, the field of computer vision still primarily relies on task-specific models, such as Segment Anything for segmentation tasks and Depth Anything for depth estimation. Different tasks usually employ separate architectures. The DeepMind research team believes that large text-to-video models learn scene spatial structures, object motion patterns, and the relationship between language and vision during training, which may endow them with the potential to become general visual foundation models.

QQ20260720-091824.jpg

GenCeption is based on Alibaba's open-source video model Wan2.1 and achieves visual task prediction with a single forward pass by simplifying its architecture. The model can generate depth maps, surface normals, segmentation masks, and pose estimation results from the same video based on text prompts, and it supports non-image output tasks such as 3D key points through trainable modules.

In terms of training data, GenCeption mainly uses a synthetic dataset containing 7,500 videos, generated by combining 800 human models with 200 action sequences using Blender. Research shows that the model performs excellently in tasks such as depth estimation, surface normal prediction, 3D pose recognition, and language-guided segmentation, with a training data volume that is only one-seventh to one-hundredth of some existing models.

Moreover, GenCeption demonstrates strong generalization capabilities. Although the training data mainly comes from single-person synthetic human videos, the model can handle real-world scenarios with multiple people, animals, and humanoid robots, generating detailed visual prediction results for unseen categories.

However, the research team also pointed out that joint training of multiple tasks may affect the performance of some complex tasks, and the model's inference speed still needs optimization. Currently, processing an 81-frame video takes about 6 to 10 seconds.

The release of GenCeption indicates that video generation models are expanding from content generation tools toward visual understanding foundation models. In the future, how to enable generative models to gain more reliable physical understanding and real-world feedback will remain an important challenge in the AI research field.