Stanford University has proposed a new method called OccFusion, which aims to achieve high-fidelity rendering of occluded human figures. In other words, even if a part of the human body is obscured by other objects, OccFusion can ultimately render the complete human figure.

image.png

Product Entry: https://top.aibase.com/tool/occfusion

Traditional human rendering methods typically require that every part of the human body in a video is completely visible. However, in reality, occlusions are common, leading to the visibility of the human body being partial. OccFusion utilizes efficient 3D Gaussian slicing combined with a pre-trained 2D diffusion model for supervision to achieve efficient and high-fidelity human rendering.

The method consists of three stages: initialization, optimization, and refinement. In the initialization stage, a complete human body mask is generated from a partially visible mask; in the optimization stage, conditional score distillation sampling is used to optimize the human Gaussian; and finally, in the refinement stage, contextual patching is used to further improve rendering quality.

OccFusion has been evaluated on the ZJU-MoCap and challenging OcMotion sequences, performing exceptionally well and reaching the latest level of occluded human body rendering. The entire training process only requires 10 minutes on a single Titan RTX GPU.

Highlight:

 🌟 OccFusion is a new method designed to achieve high-fidelity rendering of occluded human figures.

🌟 The method includes three stages: initialization, optimization, and refinement, achieved through efficient 3D Gaussian slicing and 2D diffusion model supervision.

🌟 OccFusion has been assessed on the ZJU-MoCap and OcMotion sequences, performing excellently and reaching the latest level of occluded human body rendering.