@@ -214,7 +214,7 @@ Although incorporating CoT rationales into generative reward models improves pre
\subsubsection{Rubric-based Reward Models}
While generative reward models have shown strong performance in preference prediction, they typically produce an overall judgment for each response or response pair. For complex open-ended tasks, such holistic evaluation may be insufficient because response quality often depends on multiple instruction-specific aspects. To address this issue, we can employ rubric-based reward modeling. The basic idea is to introduce a set of task-specific criteria that explicitly guide the reward model on \emph{what aspects to evaluate} when predicting preferences. In practice, these criteria can be organized into different rubric formats, as follows:
While generative reward models have shown strong performance in preference prediction, they typically produce an overall judgment for each response or response pair. For complex open-ended tasks, such holistic evaluation may be insufficient because response quality often depends on multiple instruction-specific aspects. To address this issue, we can employ rubric-based reward modeling. The basic idea is to introduce a set of task-specific criteria that explicitly guide the reward model on \emph{what aspects to evaluate} when predicting preferences. In practice, these criteria can be organized into different rubric formats, as illustrated below, where the underlined text in each example represents the rubric.
\begin{itemize}
\item\textbf{Holistic Rubric.}
...
...
@@ -359,7 +359,10 @@ While generative reward models have shown strong performance in preference predi
It is worth noting that reasoning can also be incorporated into rubric-based evaluation. For example, in a checklist rubric, we can ask the reward model to reason about each criterion before producing the corresponding Pass/Fail judgment.
% In rubric-based reward modeling, the quality of rubric is important. There are three approaches.
The quality of the rubric is crucial to the effectiveness of rubric-based reward modeling. As a result, recent work has devoted increasing attention to constructing high-quality rubrics for reward modeling. Similar to many other components in machine learning, rubric acquisition generally follows two approaches: manual design and automatic generation.
