Apple Research Team Breaks Through AI Programming Bottleneck: Let Open Source Models Learn SwiftUI Interface Development

AIbase Report - In the latest published paper "UICoder: Training Large Language Models to Generate User Interface Code through Automatic Feedback Fine-tuning", Apple's research team demonstrated a remarkable technical breakthrough: successfully training an open-source model to autonomously learn SwiftUI user interface development.

Although large language models perform well in creative writing and programming, they still have significant shortcomings in generating "grammatically correct and well-designed UI code". The research team pointed out that even in carefully curated data sets, UI code examples are extremely scarce, accounting for less than 1% in some code data sets.

The research team used the open-source programming model StarChat-Beta as a base, employing a unique automatic feedback loop method: Data Generation: providing the model with a list of UI descriptions, instructing it to generate a large-scale SwiftUI program synthetic dataset Quality Screening: verifying the executability of code through the Swift compiler, using the GPT-4V visual model to compare the interface with the original description Iterative Optimization: removing outputs that failed compilation, were irrelevant or duplicated, and retaining high-quality training data for model fine-tuning.

After five iterations, the team obtained a large dataset containing 996,000 SwiftUI programs and successfully trained the UICoder model.

Test results show that UICoder significantly surpasses the base StarChat-Beta model in both automated metrics and human evaluation, reaching quality close to that of GPT-4, and even exceeding GPT-4 in compilation success rate.

An important finding from the study came from data analysis: the original training data of StarChat-Beta almost did not contain any SwiftUI content. When creating the TheStack dataset, the Swift code repository was accidentally excluded, and only one ten-thousandth of the Swift code examples were present in the OpenAssistant-Guanaco dataset.

This finding proves that the performance improvement of UICoder is not derived from reorganizing existing data, but rather from a true learning breakthrough achieved by building a self-generated curated dataset through an automatic feedback loop.

The research team stated that the success of this method verifies its potential application in other programming languages and UI toolkits, opening up new possibilities for AI-assisted software development.

Paper link: https://arxiv.org/html/2406.07739v1