Microsoft Launches Agent Lightning: A New AI Framework to Help Train Large Language Models with Reinforcement Learning

Microsoft recently released Agent Lightning, an open-source framework designed to optimize multi-agent systems through reinforcement learning (RL). Agent Lightning can convert real agent behavior into RL transitions without changing the existing agent architecture, thereby improving the performance of large-scale language models (LLM).

Agent Lightning models agents as a decision-making process, specifically formalizing agents as partially observable Markov decision processes. An agent's observation is the current input, its action is a model call, and the reward can be either a terminal or intermediate reward. The framework extracts the call logs of the agent model, along with input, output, and reward information, filtering out unnecessary noise to generate clean transition data for training.

The framework adopts a "training and deployment decoupling" approach, with the Lightning Server handling training and service, and providing an API interface compatible with OpenAI, making it easy to call updated models. Meanwhile, the Lightning Client captures call logs in the existing agent runtime and sends the data back to the server in real time. This design maintains tight integration with tools, browsers, and other dependencies, while placing GPU training on the server layer.

Agent Lightning supports two tracking paths. The default path uses OpenTelemetry for data collection, making it convenient to send agent telemetry information to a standard collector. There is also a lightweight embedded tracker suitable for teams that do not want to deploy OpenTelemetry. Ultimately, all data is stored in the same location for training purposes.

In terms of experiments, the research team evaluated three tasks: text-to-SQL, retrieval-augmented generation, and math QA. The text-to-SQL task uses the Spider benchmark, covering over 10,000 questions and 200 databases. Retrieval-augmented generation uses the MuSiQue benchmark, built on a Wikipedia-scale index containing 21 million documents. Math QA uses the Calc X dataset, performing calculations through tool calls. Training on each task showed stable reward improvements.

Paper: https://arxiv.org/abs/2508.03680v1

Key Points:
🌟 Agent Lightning is an open-source framework that optimizes multi-agent systems without restructuring the existing system.
🚀 The framework models agents as partially observable Markov decision processes, extracting clean training transition data.
📈 Experiments show significant performance improvements of Agent Lightning on tasks such as text-to-SQL, retrieval-augmented generation, and math QA.

DeepSeek-R1 Paper Appears on the Cover of Nature, Highlighting a New Breakthrough in Artificial Intelligence Reasoning

Recently, the latest issue of the journal Nature featured a cover paper that has attracted widespread attention. The research focuses on DeepSeek-R1. This study was led by Professor Liang Wenfeng's team and centers on how to enhance the reasoning capabilities of large language models (LLMs) through reinforcement learning. As early as January this year, the research was published on arXiv and received high praise from the academic community. In the cover introduction, Nature pointed out that if large models can plan the steps to solve problems, they often achieve better solutions. This

Tencent AI Lab Innovates with a Parallel Thinking Framework, Unlocking New Reasoning Methods for Large Models!

With the continuous advancement of AI technology, how to enable large models to possess 'parallel thinking' capabilities has become a hot topic among researchers. Recently, the Tencent AI Lab, in collaboration with research teams from multiple universities, introduced a new reinforcement learning (RL) framework called Parallel-R1, aimed at teaching large models how to explore multiple reasoning paths simultaneously. This innovative framework opens up new perspectives for tackling complex mathematical reasoning tasks. Traditional methods often rely on supervised fine-tuning (SFT), a method that not only requires high-quality data

Microsoft Releases Agent Lightning Reinforcement Learning Framework, Claims It Can Train Any AI Agent System

Microsoft Research launches the Agent Lightning reinforcement learning framework, innovatively using a decoupled design to solve AI agent training challenges. The framework abstracts the agent execution process into a unified interface, supporting standardized training for AI agents with different architectures. The accompanying LightningRL algorithm can effectively allocate task rewards. Testing shows that the framework performs well in tasks such as text-to-SQL and RAG question answering, supporting multi-agent collaboration and complex tool calls. Its training-agent separation architecture provides a platform for AI agent training.

Microsoft Launches Agent Lightning: A New AI Framework to Help Train Large Language Models with Reinforcement Learning

Related Recommendations

Small Model Training Efficiency Surges 100 Times! Thinking Machine Introduces Online Policy Distillation, OpenAI's Former CTO Likes It Personally

The Alarm Bell of the Era of Large Models: Richard Sutton Calls for a Rekindling of Scientific Exploration into Intelligence Understanding

DeepSeek-R1 Paper Appears on the Cover of Nature, Highlighting a New Breakthrough in Artificial Intelligence Reasoning

Tencent AI Lab Innovates with a Parallel Thinking Framework, Unlocking New Reasoning Methods for Large Models!

Microsoft Releases Agent Lightning Reinforcement Learning Framework, Claims It Can Train Any AI Agent System