David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et~al.
\newblock Mastering the game of go without human knowledge.
title={Learning robust autonomous navigation and locomotion for wheeled-legged robots},
author={Lee, Joonho and Bjelonic, Marko and Reske, Alexander and Wellhausen, Lorenz and Miki, Takahiro and Hutter, Marco},
journal={Science Robotics},
volume={9},
number={89},
pages={eadi9641},
year={2024},
publisher={American Association for the Advancement of Science}
}
@article{silver-etal:silver2017mastering,
title={Mastering the game of go without human knowledge},
author={Silver, David and Schrittwieser, Julian and Simonyan, Karen and Antonoglou, Ioannis and Huang, Aja and Guez, Arthur and Hubert, Thomas and Baker, Lucas and Lai, Matthew and Bolton, Adrian and others},
journal={nature},
volume={550},
number={7676},
pages={354--359},
year={2017},
publisher={Nature Publishing Group UK London}
}
@article{mnih-etal:mnih2013playing,
title={Playing atari with deep reinforcement learning},
author={Mnih, Volodymyr and Kavukcuoglu, Koray and Silver, David and Graves, Alex and Antonoglou, Ioannis and Wierstra, Daan and Riedmiller, Martin},
journal={arXiv preprint arXiv:1312.5602},
year={2013}
}
@article{wang2026msrl,
title={MSRL: Scaling Generative Multimodal Reward Modeling via Multi-Stage Reinforcement Learning},
\section{Reinforcement Learning for LLM-based Agents}
Using RL to train agents is a well-established practice in traditional machine learning. In this context, the agent has consistently served as the central decision-making entity interacting with an environment. However, the advent of LLMs has fundamentally transformed the motivations, application scenarios, and action spaces of RL-driven agents.
% Using RL to train agents is 一个标准的做法在传统的机器学习中。 in this context, the agent has always been the central decision-making entity interacting with an environment. However, the advent of LLMs has fundamentally transformed the motivation, application scenarios, and action spaces of RL-driven agents.
In classical RL, an agent is typically a domain-specific decision model trained entirely from scratch. For example, in tasks such as mastering Atari games \citep{mnih-etal:mnih2013playing}, playing Go \citep{silver-etal:silver2017mastering}, or controlling robotic locomotion \citep{lee-etal:lee2024learning}, researchers conventionally construct a relatively small-scale policy network within a highly structured simulated environment characterized by predefined state and action spaces. Through millions of trial-and-error interactions, the agent iteratively learns an optimal policy that maximizes expected rewards. Ultimately, the primary objective in these settings is to comprehensively explore the environment and synthesize this optimal policy from a randomly initialized state.
In classical RL-driven agents, an agent is typically a domain-specific decision model trained from scratch. For example, in tasks such as mastering Atari games, playing Go, or robotic locomotion, researchers construct a relatively small-scale policy network situated within a highly structured simulated environment with predefined state and action spaces. Through millions of trial-and-error interactions, the agent gradually learns to maximize cumulative rewards. In these classic settings, the primary objective of RL is to explore and synthesize an optimal execution policy from a blank slate.
