Reinforcement learning (RL) has become a powerful and versatile paradigm in the era of large language models (LLMs), with applications ranging from aligning models with human preferences to improving their reasoning capabilities. More recently, as modern AI increasingly shifts toward learning from experience, RL has attracted renewed attention as a path toward more capable and adaptive intelligent systems, enabling LLM-based agents to continuously improve their decision-making through environment interaction. Despite its growing importance in LLM development, RL remains difficult for many LLM researchers to approach. Traditional RL literature often explains its concepts through robotics and control problems, making them less intuitive in the context of LLMs. To this end, this paper provides a comprehensive introduction to RL in the era of LLMs. We begin with the fundamental concepts of RL and explain key algorithms through a concrete example of training an LLM, including policy gradients, advantage estimation, importance sampling, and reward modeling. We then discuss recent advances in RL for LLMs, including improved reward construction, efficient training, and policy optimization. We further extend the discussion to reasoning models, LLM-based agents, and multimodal models, showing how RL can enhance reasoning, support learning through environment interaction, and optimize multimodal understanding and generation. Finally, we summarize promising future directions and commonly used systems and datasets. We hope this paper provides an accessible introduction to RL and helps LLM researchers better understand and apply RL techniques in the era of LLMs.
Reinforcement learning (RL) has become a powerful and versatile paradigm in the era of large language models (LLMs). It is widely used to align models with human preferences and improve their reasoning capabilities. More recently, with the rapid rise of LLM-based agents, RL has become increasingly important for enabling models to learn from experience and build more capable and adaptive intelligent systems. Despite its growing importance, RL remains difficult for many LLM researchers to approach. Traditional RL literature often explains key concepts through robotics and control problems, making them less intuitive in the context of LLMs. To bridge this gap, this paper provides a comprehensive introduction to RL in the era of LLMs. We first introduce the fundamental concepts of RL and explain key algorithms through a concrete example of training an LLM, including policy gradients, advantage estimation, importance sampling, and reward modeling. We then discuss recent advances in RL for LLMs, including improved reward construction, efficient training, and policy optimization. We further extend the discussion to reasoning models, LLM-based agents, and multimodal models. We show how RL can enhance reasoning, train agents through environment interaction, and optimize multimodal understanding and generation. Finally, we summarize promising future directions and commonly used systems and datasets. We hope this paper provides an accessible introduction to RL and helps LLM researchers better understand and apply RL techniques in modern foundation models.
\end{abstract}
...
...
@@ -245,7 +245,7 @@ The aim of this paper is to provide a comprehensive introduction to RL from the
\section*{Acknowledgements}
We would like to thank for their suggestions on improving the early version of this work.
