\bibitem[Wei et~al.(2022)Wei, Bosma, Zhao, Guu, Yu, Lester, Du, Dai, and Le]{wei-etal:2022finetuned}
\bibitem[Wei et~al.(2022{\natexlab{a}})Wei, Bosma, Zhao, Guu, Yu, Lester, Du, Dai, and Le]{wei-etal:2022finetuned}
Jason Wei, Maarten Bosma, Vincent~Y. Zhao, Kelvin Guu, Adams~Wei Yu, Brian Lester, Nan Du, Andrew~M. Dai, and Quoc~V. Le.
\newblock Finetuned language models are zero-shot learners.
\newblock In \emph{The Tenth International Conference on Learning Representations, {ICLR} 2022, Virtual Event, April 25-29, 2022}. OpenReview.net, 2022.
\newblock In \emph{The Tenth International Conference on Learning Representations, {ICLR} 2022, Virtual Event, April 25-29, 2022}. OpenReview.net, 2022{\natexlab{a}}.
@@ -246,7 +246,9 @@ The aim of this paper is to provide a comprehensive introduction to RL from the
\section*{Acknowledgements}
We would like to thank those who provided suggestions on improving the early version of this work.
\phantomsection
\addcontentsline{toc}{section}{Acknowledgements}
This work was supported in part by the National Science Foundation of China (Nos. 62276056 and U24A20334), the Yunnan Fundamental Research Projects (No.202401BC070021), the Yunnan Science and Technology Major Project (No. 202502AD080014), the Liaoning Provincial Science and Technology Plan Project (No. 2026JH40/10100033), the Fundamental Research Funds for the Central Universities (Nos. N25BSS054 and N25BSS094), and the Program of Introducing Talents of Discipline to Universities, Plan 111 (No.B16009). The authors thank Ziming Zhu, Yuzhang Wu, Yifu Huo, Xihan Yang, and Kaiwei Wang for their valuable comments and discussions, which helped improve the manuscript.
@@ -76,7 +76,7 @@ Please make the response clear, practical, and easy to follow. \\[1mm]
\vspace{0.5em}
In this example, the input itself specifies the task, and the LLM generates the answer by continuing the sequence. Prompts can also be constructed from templates. A prompt template is a piece of text containing placeholders, where each placeholder is filled with concrete information before being sent to the LLM. For example, we can use the following template to construct prompts for a homework assistant:
In this example, the prompt specifies both the task and the desired response style. The LLM then generates the answer by continuing the input sequence. Prompts can also be constructed from templates. A \textit{prompt template} contains placeholders that can be replaced with task-specific information. For example, we can use the following template for a homework assistant:
\vspace{0.1cm}
...
...
@@ -124,6 +124,34 @@ Another important concept related to prompting is in-context learning. When prom
\end{tcolorbox}
\vspace{0.5em}
When a single demonstration is provided, this setting is commonly called \textit{one-shot} prompting. When several demonstrations are included, it is referred to as \textit{few-shot} prompting. In contrast, prompting without any demonstration is usually called \textit{zero-shot} prompting.
In-context learning can also be combined with \textit{chain-of-thought} (CoT) prompting \citep{wei-etal:chain}. The basic idea is to encourage the model to generate intermediate reasoning steps before producing the final answer. This can be achieved either by explicitly asking the model to reason step by step or by providing demonstrations that contain both reasoning traces and answers. For example:
\vspace{0.1cm}
\begin{tcolorbox}[frame empty]
\begingroup
\setlength{\leftskip}{2em}
\setlength{\rightskip}{2em}
\textit{Input:} A student solves 8 math problems and gets 6 correct. What is the accuracy? Please reason step by step. \\[1mm]
\textit{Output:} The student gets 6 out of 8 problems correct. The accuracy is therefore $6/8=0.75$, or 75\%. \\[2mm]
\textit{Input:} A student solves 20 math problems and gets 17 correct. What is the accuracy? Please reason step by step. \\[1mm]
\textit{Output:}\underline{\hspace{4cm}}
\endgroup
\end{tcolorbox}
\vspace{0.5em}
Prompting adapts an LLM by changing the context used for generation. Therefore, prompt quality can strongly affect model performance. Even for the same task, we can write the prompt in many different ways. For example, ``Give me two tips to improve my English writing.'' can also be written as ``What are two simple ways to improve my English writing skills?''. The two prompts express nearly the same intent, but they may lead to noticeably different outputs. In practice, we often refine prompts through repeated trial and error for a given LLM. More advanced methods automate this process and search for better prompts using techniques such as RL \citep{deng-etal:rlprompt} or evolutionary algorithms \citep{guo-etal:connecting}.
