In the animation production field, traditional anime production processes are time-consuming and labor-intensive, usually requiring high-level artists to perform multiple stages such as keyframe drawing, in-betweening, and coloring. However, recently, ToonComposer, developed by a research team from The Chinese University of Hong Kong and Tencent PCG, has greatly simplified this process, transforming tedious manual operations into a seamless workflow using generative AI technology.

image.png

ToonComposer's "post-inpainting" technology allows users to provide just a sketch and one frame of a colored image, enabling the generation of a complete cartoon video, saving up to 70% of manual work time, allowing creators to focus more on the creation itself.

Regarding keyframe control, ToonComposer offers precise sketch keyframe control features, enabling artists to guide the progression of animations at any position on the timeline using sparse keyframes. Even for complex movements, the system can handle them accurately, producing results that match the quality of traditional manual in-betweening and coloring processes.

image.png

In addition, ToonComposer introduces a region control feature, allowing users to leave certain sketch areas blank and mark them with a brush tool. The system will intelligently recognize these areas and fill them using context or prompts, further improving the efficiency of creation.

The research team would like to thank B&T Studio and Gutong Animation Studio for allowing the use of animated content such as "Big Fish & the Sky" and "Mr. Cat" for academic demonstrations. They also express gratitude for the support from the video foundation model Wan and other open-source tools. It is worth noting that all images, videos, and other materials displayed on the page are for academic and research purposes only. ToonComposer, as a research project, is not intended for commercial use.

Project: https://lg-li.github.io/project/tooncomposer/

Key Points:

🎨 ToonComposer simplifies the animation production process using generative AI technology, allowing users to generate a complete animation with just a sketch and one frame of a colored image.

⏳ This system can save up to 70% of manual work time, letting creators focus on their creativity.

🖌️ The region control feature allows users to freely mark sketch areas, and the system will intelligently fill them, enhancing the efficiency of creation.