update.

.gitignore

@@ -6,4 +6,5 @@
 *.out
 *.out
 *.synctex.gz
 *.synctex.gz
 *.toc
 *.toc
+.chapter-build/
 # *.png
 # *.png

README.md

-# Template
+# RL without Tears: An Introduction in the Era of LLMs
 
 
-Template and style files for CoLM 2025
+Reinforcement learning (RL) has become an important training paradigm for large language models (LLMs). It is widely used to align models with human preferences, improve reasoning capabilities, and enable agentic systems to learn from interaction and feedback. In essence, RL studies how an agent improves its behavior by receiving rewards from an environment. A standard objective can be written as:
+
+```math
+\max_{\pi} \; \mathbb{E}_{\tau \sim \pi}\left[\sum_{t=0}^{T} r_t\right],
+```
+
+where $\pi$ denotes the policy, $\tau$ denotes a trajectory of interactions, and $r_t$ is the reward received at step $t$. In the era of LLMs, this formulation naturally connects to language generation: an LLM can be viewed as a policy, generated tokens or responses form actions and trajectories, and reward signals evaluate whether the output is helpful, correct, safe, or aligned with human intent.
+
+In the article **RL without Tears: An Introduction in the Era of LLMs**, we explain reinforcement learning from the perspective of LLM training. We begin with the basic concepts of RL and introduce key algorithms through concrete LLM-oriented examples, including policy gradients, advantage estimation, importance sampling, and reward modeling. We then discuss recent advances in RL for LLMs, such as RLHF, DPO-style methods, reward construction, efficient training, and policy optimization. Finally, we extend the discussion to reasoning models, LLM-based agents, and multimodal models, showing how RL can enhance reasoning, support environment interaction, and optimize multimodal understanding and generation.
+
+This article seeks to maintain a balance between depth and readability. It is written as a systematic introduction for readers who are familiar with LLMs but may find traditional RL literature difficult to connect with modern language model training. Overall, this work can serve as an accessible guide to RL and its applications in the era of foundation models.
+
+## Resources
+
+- [Online Website](https://wangclnlp.github.io/RL-without-Tears-site/)
+- [Full PDF](rl-introduction-0911.pdf)
+- [Chapter 1: Introduction](chapter-pdfs/chapter-01-introduction.pdf)
+- [Chapter 2: Preliminary](chapter-pdfs/chapter-02-preliminary.pdf)
+- [Chapter 3: Understanding RL in LLM Training](chapter-pdfs/chapter-03-understanding-rl-in-llm-training.pdf)
+- [Chapter 4: Improved RL for LLMs](chapter-pdfs/chapter-04-improved-rl-for-llms.pdf)
+- [Chapter 5: RL for LLM Reasoning](chapter-pdfs/chapter-05-rl-for-llm-reasoning.pdf)
+- [Chapter 6: Agentic RL](chapter-pdfs/chapter-06-agentic-rl.pdf)
+- [Chapter 7: Multimodal RL](chapter-pdfs/chapter-07-multimodal-rl.pdf)
+- [Chapter 8: Conclusions and Future Directions](chapter-pdfs/chapter-08-conclusions-and-future-directions.pdf)
+- [Appendix: Datasets and Systems](chapter-pdfs/appendix-datasets-and-systems.pdf)
+
+Each chapter PDF is compiled independently and includes its own bibliography.
+
+## Compile
+
+Compile the full tutorial:
+
+```powershell
+latexmk -pdf -bibtex rl-introduction.tex
+```
+
+Compile chapter-level PDFs:
+
+```powershell
+powershell -ExecutionPolicy Bypass -File scripts\build_chapter_pdfs.ps1
+```
+
+The chapter PDFs will be written to `chapter-pdfs/`. Temporary build files are written to `.chapter-build/`.
+
+## Citation
+
+```bibtex
+@misc{wang2026rlwithouttears,
+  title  = {RL without Tears: An Introduction in the Era of LLMs},
+  author = {Wang, Chenglong and Zhou, Hang and Liu, Tongran and Zhu, Jingbo and Xiao, Tong},
+  year   = {2026},
+  note   = {Tutorial manuscript}
+}
+```
+
+## License
+
+This work is released under the CC-BY 4.0 license.
+
+## Contact
+
+For questions or suggestions, please contact Chenglong Wang at `WANGCHENGLONG@MAIL.NEU.EDU.CN`.