update.

README.md

 # RL without Tears: An Introduction in the Era of LLMs
 
-Reinforcement learning (RL) has become an important training paradigm for large language models (LLMs). It is widely used to align models with human preferences, improve reasoning capabilities, and enable agentic systems to learn from interaction and feedback. In essence, RL studies how an agent improves its behavior by receiving rewards from an environment. A standard objective can be written as:
+Reinforcement learning (RL) has become an important training paradigm for large language models (LLMs). It is widely used to align models with human preferences, improve reasoning capabilities, and enable agentic systems to learn from interaction and feedback. At its core, RL studies how an agent improves its behavior by interacting with an environment and receiving reward signals. A standard objective can be written as:
 
 ```math
-\max_{\pi} \; \mathbb{E}_{\tau \sim \pi}\left[\sum_{t=0}^{T} r_t\right],
-```
+\max_{\pi} \; \mathbb{E}_{\tau \sim \pi}
+\left[
+\sum_{t=0}^{T} r_t
+\right],
+````
+
+where $\pi$ denotes the policy, $\tau$ denotes a trajectory of interactions, and $r_t$ is the reward received at step $t$.
 
-where $\pi$ denotes the policy, $\tau$ denotes a trajectory of interactions, and $r_t$ is the reward received at step $t$. In the era of LLMs, this formulation naturally connects to language generation: an LLM can be viewed as a policy, generated tokens or responses form actions and trajectories, and reward signals evaluate whether the output is helpful, correct, safe, or aligned with human intent.
+In the era of LLMs, this formulation naturally connects to language generation. An LLM can be viewed as a policy, generated tokens or responses can be treated as actions and trajectories, and reward signals can be used to evaluate whether an output is helpful, correct, safe, or aligned with human intent.
 
-In the article **RL without Tears: An Introduction in the Era of LLMs**, we explain RL from the perspective of LLM training. We begin with the basic concepts of RL and introduce key algorithms through concrete LLM-oriented examples, including policy gradients, advantage estimation, importance sampling, and reward modeling. We then discuss recent advances in RL for LLMs, such as RLHF, DPO-style methods, reward construction, efficient training, and policy optimization. Finally, we extend the discussion to reasoning models, LLM-based agents, and multimodal models, showing how RL can enhance reasoning, support environment interaction, and optimize multimodal understanding and generation.
+In **RL without Tears: An Introduction in the Era of LLMs**, we introduce RL from the perspective of LLM research and training. We begin with the fundamental concepts of RL and explain key algorithms through concrete LLM-oriented examples, including policy gradients, advantage estimation, importance sampling, proximal policy optimization, and reward modeling. We then discuss recent advances in RL for LLMs, including improved reward modeling, advantage estimation, efficient training, and policy optimization. Beyond standard LLM training, we further extend the discussion to reasoning models, LLM-based agents, and multimodal models, showing how RL can enhance reasoning, support learning through environment interaction, and optimize multimodal understanding and generation.
 
-This article seeks to maintain a balance between depth and readability. It is written as a systematic introduction for readers who are familiar with LLMs but may find traditional RL literature difficult to connect with modern language model training. Overall, this work can serve as an accessible guide to RL and its applications in the era of foundation models.
+This paper aims to balance technical depth with accessibility. It is intended for readers who are familiar with LLMs but may find it difficult to connect traditional RL literature with modern language model training. Rather than introducing RL primarily through robotics or control problems, we explain its key concepts and algorithms using terminology and examples that are familiar to the LLM community. We hope this paper provides an accessible introduction to RL and helps LLM researchers better understand and apply RL techniques in modern foundation models.
 
 ## Resources