update.

section6/section6.tex

@@ -17,7 +17,7 @@ One feature of LLM-based agents is their capacity to interpret user instructions
 
 
 To accomplish this process, an LLM-based agent relies on several complementary capabilities. Some of these capabilities are already inherited from LLMs. Recent advances in large-scale pretraining and post-training have significantly improved LLMs in instruction following, reasoning, code generation, and general world knowledge. These capabilities provide a strong foundation for agentic behavior, enabling LLMs to understand complex user requests and generate coherent intermediate reasoning steps without explicit interaction with the real-world environment. The rapid improvement of these intrinsic capabilities has been a key factor driving the emergence of LLM-based agents, making it increasingly feasible for a single model to serve as the reasoning and decision-making core of an autonomous system.
 To accomplish this process, an LLM-based agent relies on several complementary capabilities. Some of these capabilities are already inherited from LLMs. Recent advances in large-scale pretraining and post-training have significantly improved LLMs in instruction following, reasoning, code generation, and general world knowledge. These capabilities provide a strong foundation for agentic behavior, enabling LLMs to understand complex user requests and generate coherent intermediate reasoning steps without explicit interaction with the real-world environment. The rapid improvement of these intrinsic capabilities has been a key factor driving the emergence of LLM-based agents, making it increasingly feasible for a single model to serve as the reasoning and decision-making core of an autonomous system.
 
 
-However, autonomous agents require capabilities that go beyond static reasoning. They must make sequential decisions over long horizons, coordinate multiple tool use, adapt to unexpected environmental feedback, and recover from execution failures. These capabilities cannot be effectively learned from static supervision alone because their quality depends on the outcomes of complete interaction trajectories rather than individual outputs. RL naturally addresses this challenge by optimizing behaviors through trial-and-error interaction, enabling agents to improve decisions based on delayed rewards and environmental feedback \citep{singh-etal:singhagentic}. As a result, RL complements the inherent reasoning abilities of LLMs by strengthening their decision-making and execution capabilities in dynamic environments. In the following sections, we discuss two representative capabilities that have become the primary focus of agentic RL: planning and tool use.
+However, autonomous agents require capabilities that go beyond static reasoning. They must make sequential decisions over long horizons, coordinate multiple tool use, adapt to unexpected environmental feedback, and recover from execution failures. These capabilities cannot be effectively learned from static supervision alone because their quality depends on the outcomes of complete interaction trajectories rather than individual outputs. RL naturally addresses this challenge by optimizing behaviors through trial-and-error interaction, enabling agents to improve decisions based on delayed rewards and environmental feedback \citep{singh-etal:singhagentic}. As a result, RL complements the inherent reasoning abilities of LLMs by strengthening their decision-making and execution capabilities in dynamic environments. In the following sections, we discuss two core capabilities that have become the primary focus of agentic RL: planning and tool use.
 
 
 
 
 \subsubsection{Planning}
 \subsubsection{Planning}
@@ -250,19 +250,16 @@ So far, we have introduced how RL can enhance the core capabilities of LLM-based
 
 
 
 
 
 
-\subsection{Learning from Experience}
-
-
+\subsection{Learning from Agentic Experience}
 
 
 \subsubsection{Memory Management}
 \subsubsection{Memory Management}
 
 
-
-
 \subsubsection{Skill Optimization}
 \subsubsection{Skill Optimization}
 
 
+\subsubsection{Trajectory Refinement}
+
 
 
 
 
-\subsubsection{Trajectory Refinement}
 
 
 
 
 \subsection{Self-Evolving Agents}
 \subsection{Self-Evolving Agents}