update.

appendix/appendix.tex

 
 
 \clearpage
 \clearpage
-\section*{Appendix A: Useful Systems and Datasets}
+\section*{Appendix: Useful Systems and Datasets}
 \phantomsection
 \phantomsection
 \addcontentsline{toc}{section}{Appendix A: Useful Systems and Datasets}
 \addcontentsline{toc}{section}{Appendix A: Useful Systems and Datasets}
 
 

section4/section4.tex

@@ -357,10 +357,11 @@ While generative reward models have shown strong performance in preference predi
 \end{itemize}
 \end{itemize}
 
 
 
 
-It is worth noting that reasoning can also be incorporated into rubric-based evaluation. For example, in a checklist rubric, we can ask the reward model to reason about each criterion before producing the corresponding Pass/Fail judgment.
+It is worth noting that reasoning can also be incorporated into rubric-based evaluation. For example, in a checklist-style rubric, the reward model can be asked to reason about each criterion before producing the corresponding Pass/Fail judgment. Additionally, although we use pairwise evaluation, i.e., taking $(\mathbf{x}, \mathbf{y}_a, \mathbf{y}_b)$ as reward model input, as the running example, rubric-based reward modeling is equally applicable to pointwise evaluation, i.e., taking $(\mathbf{x}, \mathbf{y})$ as reward model input.
 
 
 The quality of the rubric is crucial to the effectiveness of rubric-based reward modeling. As a result, recent work has devoted increasing attention to constructing high-quality rubrics for reward modeling. Similar to many other components in machine learning, rubric acquisition generally follows two approaches: manual design and automatic generation.
 The quality of the rubric is crucial to the effectiveness of rubric-based reward modeling. As a result, recent work has devoted increasing attention to constructing high-quality rubrics for reward modeling. Similar to many other components in machine learning, rubric acquisition generally follows two approaches: manual design and automatic generation.
 
 
+For manual design, one straightforward approach is to recruit human experts to write rubrics based on task requirements and their domain knowledge. However, this way is often difficult to design a rubric that comprehensively covers the diverse cases that may arise across different inputs and tasks. This is because that evaluation criteria that are appropriate for one case may be insufficient for another. Additionally, manually designed rubrics inherit the prompt sensitivity of LLM-based evaluation: even when the underlying evaluation dimensions remain the same, small differences in wording can lead to noticeably different results.