Yuting Zhong 钟玉婷

Yuting Zhong 钟玉婷

Education教育背景

University of Macau澳门大学 2025.08 – 2027.07 (expected)(预计)
M.S. in Data Science (Artificial Intelligence)数据科学理学硕士(人工智能应用方向) Macau SAR, China中国澳门
Guangdong University of Foreign Studies广东外语外贸大学 2021.09 – 2025.07
B.Eng. in Computer Science and Technology计算机科学与技术工学学士 Guangzhou, China中国广州
  • GPA: 3.75/4.0 · Outstanding Graduate and Honours Degree优秀毕业生、荣誉学位

Research Interests研究兴趣

My research focuses on LLM evaluation, LLM-as-a-Judge, rubric-based evaluation, and machine learning. I am particularly interested in evaluating judges on long-horizon agent trajectories and building reliable reward models for complex tasks.

我的研究兴趣包括大语言模型评测、LLM-as-a-Judge、基于评分标准(rubric)的评测及机器学习,重点关注长程智能体轨迹评测,以及复杂任务中的可靠奖励模型。

News最新动态

Experience科研经历

Research Assistant研究助理 2025 – Present至今
NLP²CT Laboratory · University of Macau Macau SAR, China中国澳门
  • Conduct research on LLM reliability and safety evaluation, focusing on LLM judges, long-horizon agent trajectories, and reward models for deep research.
  • Contribute to benchmark design, dataset construction, large-scale model evaluation, statistical analysis, and manuscript writing.
  • Lead LongJudgeBench as first author and co-first-author CIFE; also co-first-authored DeepResearch RewardBench, accepted to Findings of EMNLP 2026.
  • 围绕大语言模型可靠性与安全性评测开展研究,重点关注 LLM 判官、长程智能体轨迹与深度研究奖励模型。
  • 参与基准设计、数据集构建、大规模模型评测、统计分析及论文写作。
  • 以第一作者身份主导 LongJudgeBench,并列第一作者完成 CIFE;同时作为共同第一作者完成 DeepResearch RewardBench,该论文已被 EMNLP 2026 Findings 接收。

Projects项目经历

End-to-end research involvement in all three projects: research design, implementation, experiments and analysis, and manuscript writing.

全程参与以下三项研究:覆盖研究设计、方法实现、实验与分析及论文撰写。

DeepResearch RewardBench: Evaluating Reward Models for Deep Research Reports 2025 – 2026

Co-first author · Accepted to Findings of EMNLP 2026.共同第一作者 · 已被 Findings of EMNLP 2026 接收。

  • Built a reward-model benchmark for long-form deep-research reports, with an end-to-end pipeline spanning 19 agent/LLM families; combined automatic filtering and human verification to construct preference data covering 331 tasks and 866 reports.
  • Evaluated 17 models from 7B to 235B with Best-of-N selection and full-ranking consistency metrics; analyzed URL robustness, input-order bias, and comparative versus pointwise scoring.
  • Developed a four-category judge-error taxonomy and a few-shot classifier reaching ~87% agreement with human annotation.
  • 构建面向长篇深度研究报告的奖励模型评测基准,设计覆盖 19 个智能体/大模型家族的端到端流水线,结合自动过滤与人工校验,产出包含 331 项任务、866 份报告的偏好数据。
  • 基于 Best-of-N 选择与全排序一致性指标评测 17 个 7B–235B 模型,并分析 URL 鲁棒性、输入顺序偏差及对比式与打分式范式。
  • 构建四类裁判错误体系及 few-shot 自动分类器,与人工标注一致率约 87%。
LongJudgeBench: Benchmarking LLM Judges on Agent Trajectory Error Localization and Evidence Attribution 2026

First author · Submitted to ACL Rolling Review (October 2026).第一作者 · 已投稿至 ACL Rolling Review(2026 年 10 月)。

  • Introduced a benchmark for three complementary judge capabilities: error detection, error localization, and attribution of supporting execution evidence. It contains 590 natural trajectories (NSet) and 262 constructed error cases (CSet).
  • Constructed CSet with executable interventions and autonomous agent continuation across five error families, providing explicit error locations and supporting evidence references where applicable; evaluated 10 LLM judges.
  • Found a substantial evidence-attribution gap: on CSet, Claude Opus 5.5 reached 94.6% detection accuracy and 98.8% conditional localization accuracy, but only 60.0% Evidence Hit (the highest among evaluated judges).
  • 提出评测 LLM 判官三项能力的基准:在长程智能体轨迹中发现错误、定位错误并归因支持判断的执行证据。包含 590 条自然轨迹(NSet)和 262 个构造错误案例(CSet)。
  • 通过可执行干预与智能体自主续写构造 CSet,覆盖五类错误,并在适用时提供明确的错误位置及支持证据;共评测 10 个 LLM 判官。
  • 实验发现证据归因存在明显短板:Claude Opus 5.5 在 CSet 上的错误检出准确率为 94.6%,条件定位准确率为 98.8%,但 Evidence Hit 仅为 60.0%(在参评判官中最高)。
CIFE: Glossary-Grounded, Item-Level Evaluation of Cultural Fidelity in Literary Machine Translation 2026

Co-first author · Submitted to ACL Rolling Review (October 2026).共同第一作者 · 已投稿至 ACL Rolling Review(2026 年 10 月)。

  • Introduced a reference-free metric that evaluates cultural fidelity at the glossary-item level, so culture-specific items in literary translations are assessed rather than being diluted in a sentence-level score.
  • For each item, an LLM judge assigns an MQM-style severity constrained by the named translation strategy and provides a translation quote that is automatically checked, making verdicts auditable.
  • Evaluated English translations of Journey to the West and a Telugu Ramayana. CIFE's correlation with human cultural-fidelity ratings exceeded the baselines, while experiments also exposed limitations such as penalizing some valid names and paraphrases.
  • 提出一种无需参考译文、基于术语表逐项评估文化忠实度的方法,避免文学翻译中的文化特定项被句子级总分稀释。
  • LLM 判官为每个词条给出受翻译策略约束的 MQM 风格严重度,并提供可自动核验的译文引文,使评测结论可审计。
  • 在《西游记》英文译本和泰卢固语《罗摩衍那》上进行评测。CIFE 与人工文化忠实度评分的相关性高于各基线;实验也揭示其局限,例如可能惩罚部分有效的人名译法与释义改写。

Publications & Manuscripts论文成果与在研工作

* denotes equal contribution.* 表示同等贡献。

  1. Shudong Liu*, Yuting Zhong*, Guanhua Chen, Xinyi Yang, Junnan Liu, Lidia S. Chao, Jiaheng Liu, and Derek F. Wong. (2026). DeepResearch RewardBench: Evaluating Reward Models for Deep Research Reports. Accepted to Findings of EMNLP 2026.已被 Findings of EMNLP 2026 接收。
  2. Yuting Zhong (first author). (2026).钟玉婷(第一作者)。(2026)。 LongJudgeBench: Benchmarking LLM Judges on Agent Trajectory Error Localization and Evidence Attribution. Submitted to ACL Rolling Review (October 2026).已投稿至 ACL Rolling Review(2026 年 10 月)。
  3. Bhaskar Ganesh Devalla*, Yuting Zhong*, Dejing Zhou, Junchao Wu, Shudong Liu, Lidia S. Chao, and Derek F. Wong. (2026). CIFE: Glossary-Grounded, Item-Level Evaluation of Cultural Fidelity in Literary Machine Translation. Submitted to ACL Rolling Review (October 2026).已投稿至 ACL Rolling Review(2026 年 10 月)。

Honors & Awards荣誉奖项

Relevant Coursework相关课程

Graduate: Applied Natural Language Processing, Deep Learning, Practical Machine Learning, Principles of Artificial Intelligence.

研究生: 应用自然语言处理、深度学习、实用机器学习、人工智能原理。

Undergraduate: Algorithm Design, C++ Programming, Natural Language Processing, Big Data Processing.

本科: 算法设计、C++ 程序设计、自然语言处理、大数据处理技术。

Languages语言能力

Chinese: Native · English: Fluent academic reading and writing (CET-4, CET-6)中文: 母语 · 英文: 学术读写流利(CET-4、CET-6)

Technical Skills专业技能

Programming编程语言
Python, C/C++
ML & LLM Frameworks机器学习与大模型框架
PyTorch, TensorFlow, Hugging Face Transformers, vLLM
Evaluation & Analysis评测与分析
LLM-as-a-Judge, Reward Models, Prompt/Few-shot Methods, Wilcoxon, Spearman, TF-IDF
Developer Tools开发工具
Linux, Git, LaTeX

References推荐人

Available upon request.可应要求提供。