MagicTryOn is released! ZJU x vivo jointly create video virtual try-on, you can perfectly change clothes while dancing!

Zhejiang University, in collaboration with vivo, has launched a revolutionary video virtual try-on model called MagicTryOn, which has sparked industry discussions due to its outstanding spatiotemporal consistency, clothing detail fidelity, and generalization ability. This innovative technology not only supports image and video try-ons but also achieves realistic clothing effects in complex scenarios and with significant movements, opening up new possibilities for e-commerce, fashion, and virtual content creation.

World's First: Video Try-On Framework Based on Diffusion Transformer

MagicTryOn abandons the traditional U-Net architecture and adopts advanced diffusion Transformer (DiT) technology, significantly enhancing the model's expressive power. Combined with full self-attention mechanisms, this framework achieves joint modeling of videos in both temporal and spatial dimensions, ensuring smooth and consistent try-on effects in dynamic scenes. Compared to traditional methods, MagicTryOn effectively avoids frame jitter and loss of clothing details, generating results that are film-quality.

Supports Diverse Try-On Scenarios, Dynamic Performance Stunning

MagicTryOn supports image try-ons, video try-ons, and custom try-ons, adapting to various scenarios ranging from static displays to dynamic performances. Especially in scenarios involving significant movements (such as dancing) or complex backgrounds, MagicTryOn can still maintain natural fitting and dynamic realism. Additionally, its strong generalization capability allows it to not only be used for human try-ons but also achieve virtual outfit changes on non-standard objects like dolls, providing more possibilities for creative content generation.

New Tool for E-commerce Advertising: High Detail Fidelity, Highlighting Commercial Value

MagicTryOn significantly enhances the fidelity of clothing textures, patterns, and outlines through coarse-to-fine clothing retention strategies and mask-aware loss optimization. Experiments show that the model outperforms existing technologies on the Video Virtual Try-On (VVT) dataset, generating realistic and stable try-on video effects that can be directly applied to e-commerce advertising and fashion display scenarios. This technology is expected to reduce physical try-ons and product returns, lowering the environmental impact of the fashion industry while enhancing consumers' online shopping experience.

Open Source Empowerment, Supporting Global Developers

MagicTryOn is available under the Apache2.0 license, with source code, pre-trained models, and Gradio demo interfaces released on the Hugging Face platform for free global developer access and use. This move not only demonstrates Zhejiang University and vivo's leadership in AI technology openness but also injects new innovation momentum into industries such as e-commerce, virtual reality, and content creation.

The release of MagicTryOn marks a new height in video virtual try-on technology. Its breakthroughs in spatiotemporal consistency, dynamic adaptation, and detail fidelity have set new benchmarks for AI-driven fashion technology. AIbase believes that MagicTryOn will not only drive the digital transformation of e-commerce and the fashion industry but also bring far-reaching impacts to virtual content creation and metaverse applications. In the future, with more technical details being made public and community participation, the potential of this model will be further unleashed.

Project Address: https://github.com/vivoCameraResearch/Magic-TryOn/