Yuting Zhong 钟玉婷
Education教育背景
- GPA: 3.62/4.0 · Advisor:导师: Prof. Derek F. Wong
- GPA: 3.75/4.0 · Outstanding Graduate and Honours Degree优秀毕业生、荣誉学位
Research Interests研究兴趣
My research focuses on LLM evaluation, LLM-as-a-Judge, rubric-based evaluation, and machine learning. I am particularly interested in evaluating judges on long-horizon agent trajectories and building reliable reward models for complex tasks.
我的研究兴趣包括大语言模型评测、LLM-as-a-Judge、基于评分标准(rubric)的评测及机器学习,重点关注长程智能体轨迹评测,以及复杂任务中的可靠奖励模型。
News最新动态
- 2026.10 · Submitted LongJudgeBench and CIFE to the October 2026 ACL Rolling Review (ARR).
- 2026.08 · DeepResearch RewardBench was accepted to Findings of EMNLP 2026.
- 2025.08 · Started the M.S. program in Data Science at the University of Macau and joined the NLP²CT Laboratory.
- 2026.10 · LongJudgeBench 与 CIFE 已投稿至 2026 年 10 月 ACL Rolling Review(ARR)。
- 2026.08 · DeepResearch RewardBench 被 EMNLP 2026 Findings 接收。
- 2025.08 · 开始在澳门大学攻读数据科学硕士学位,并加入 NLP²CT 实验室。
Experience科研经历
- Conduct research on LLM reliability and safety evaluation, focusing on LLM judges, long-horizon agent trajectories, and reward models for deep research.
- Contribute to benchmark design, dataset construction, large-scale model evaluation, statistical analysis, and manuscript writing.
- Lead LongJudgeBench as first author and co-first-author CIFE; also co-first-authored DeepResearch RewardBench, accepted to Findings of EMNLP 2026.
- 围绕大语言模型可靠性与安全性评测开展研究,重点关注 LLM 判官、长程智能体轨迹与深度研究奖励模型。
- 参与基准设计、数据集构建、大规模模型评测、统计分析及论文写作。
- 以第一作者身份主导 LongJudgeBench,并列第一作者完成 CIFE;同时作为共同第一作者完成 DeepResearch RewardBench,该论文已被 EMNLP 2026 Findings 接收。
Projects项目经历
End-to-end research involvement in all three projects: research design, implementation, experiments and analysis, and manuscript writing.
全程参与以下三项研究:覆盖研究设计、方法实现、实验与分析及论文撰写。
Co-first author · Accepted to Findings of EMNLP 2026.共同第一作者 · 已被 Findings of EMNLP 2026 接收。
- Built a reward-model benchmark for long-form deep-research reports, with an end-to-end pipeline spanning 19 agent/LLM families; combined automatic filtering and human verification to construct preference data covering 331 tasks and 866 reports.
- Evaluated 17 models from 7B to 235B with Best-of-N selection and full-ranking consistency metrics; analyzed URL robustness, input-order bias, and comparative versus pointwise scoring.
- Developed a four-category judge-error taxonomy and a few-shot classifier reaching ~87% agreement with human annotation.
- 构建面向长篇深度研究报告的奖励模型评测基准,设计覆盖 19 个智能体/大模型家族的端到端流水线,结合自动过滤与人工校验,产出包含 331 项任务、866 份报告的偏好数据。
- 基于 Best-of-N 选择与全排序一致性指标评测 17 个 7B–235B 模型,并分析 URL 鲁棒性、输入顺序偏差及对比式与打分式范式。
- 构建四类裁判错误体系及 few-shot 自动分类器,与人工标注一致率约 87%。
First author · Submitted to ACL Rolling Review (October 2026).第一作者 · 已投稿至 ACL Rolling Review(2026 年 10 月)。
- Introduced a benchmark for three complementary judge capabilities: error detection, error localization, and attribution of supporting execution evidence. It contains 590 natural trajectories (NSet) and 262 constructed error cases (CSet).
- Constructed CSet with executable interventions and autonomous agent continuation across five error families, providing explicit error locations and supporting evidence references where applicable; evaluated 10 LLM judges.
- Found a substantial evidence-attribution gap: on CSet, Claude Opus 5.5 reached 94.6% detection accuracy and 98.8% conditional localization accuracy, but only 60.0% Evidence Hit (the highest among evaluated judges).
- 提出评测 LLM 判官三项能力的基准:在长程智能体轨迹中发现错误、定位错误并归因支持判断的执行证据。包含 590 条自然轨迹(NSet)和 262 个构造错误案例(CSet)。
- 通过可执行干预与智能体自主续写构造 CSet,覆盖五类错误,并在适用时提供明确的错误位置及支持证据;共评测 10 个 LLM 判官。
- 实验发现证据归因存在明显短板:Claude Opus 5.5 在 CSet 上的错误检出准确率为 94.6%,条件定位准确率为 98.8%,但 Evidence Hit 仅为 60.0%(在参评判官中最高)。
Co-first author · Submitted to ACL Rolling Review (October 2026).共同第一作者 · 已投稿至 ACL Rolling Review(2026 年 10 月)。
- Introduced a reference-free metric that evaluates cultural fidelity at the glossary-item level, so culture-specific items in literary translations are assessed rather than being diluted in a sentence-level score.
- For each item, an LLM judge assigns an MQM-style severity constrained by the named translation strategy and provides a translation quote that is automatically checked, making verdicts auditable.
- Evaluated English translations of Journey to the West and a Telugu Ramayana. CIFE's correlation with human cultural-fidelity ratings exceeded the baselines, while experiments also exposed limitations such as penalizing some valid names and paraphrases.
- 提出一种无需参考译文、基于术语表逐项评估文化忠实度的方法,避免文学翻译中的文化特定项被句子级总分稀释。
- LLM 判官为每个词条给出受翻译策略约束的 MQM 风格严重度,并提供可自动核验的译文引文,使评测结论可审计。
- 在《西游记》英文译本和泰卢固语《罗摩衍那》上进行评测。CIFE 与人工文化忠实度评分的相关性高于各基线;实验也揭示其局限,例如可能惩罚部分有效的人名译法与释义改写。
Publications & Manuscripts论文成果与在研工作
* denotes equal contribution.* 表示同等贡献。
- Shudong Liu*, Yuting Zhong*, Guanhua Chen, Xinyi Yang, Junnan Liu, Lidia S. Chao, Jiaheng Liu, and Derek F. Wong. (2026). DeepResearch RewardBench: Evaluating Reward Models for Deep Research Reports. Accepted to Findings of EMNLP 2026.已被 Findings of EMNLP 2026 接收。
- Yuting Zhong (first author). (2026).钟玉婷(第一作者)。(2026)。 LongJudgeBench: Benchmarking LLM Judges on Agent Trajectory Error Localization and Evidence Attribution. Submitted to ACL Rolling Review (October 2026).已投稿至 ACL Rolling Review(2026 年 10 月)。
- Bhaskar Ganesh Devalla*, Yuting Zhong*, Dejing Zhou, Junchao Wu, Shudong Liu, Lidia S. Chao, and Derek F. Wong. (2026). CIFE: Glossary-Grounded, Item-Level Evaluation of Cultural Fidelity in Literary Machine Translation. Submitted to ACL Rolling Review (October 2026).已投稿至 ACL Rolling Review(2026 年 10 月)。
Honors & Awards荣誉奖项
- Provincial Second Prize, Chinese Mathematics Competition for College Students.
- Outstanding Graduate Award and Honours Degree, Guangdong University of Foreign Studies.
- University Comprehensive Scholarship; “Merit Student” and “Outstanding Class Cadre” honors (multiple times).
- 全国大学生数学竞赛省级二等奖。
- 广东外语外贸大学优秀毕业生、荣誉学位。
- 大学综合奖学金、校级“三好学生”与“优秀班干部”等荣誉(多次)。
Relevant Coursework相关课程
Graduate: Applied Natural Language Processing, Deep Learning, Practical Machine Learning, Principles of Artificial Intelligence.
研究生: 应用自然语言处理、深度学习、实用机器学习、人工智能原理。
Undergraduate: Algorithm Design, C++ Programming, Natural Language Processing, Big Data Processing.
本科: 算法设计、C++ 程序设计、自然语言处理、大数据处理技术。
Languages语言能力
Chinese: Native · English: Fluent academic reading and writing (CET-4, CET-6)中文: 母语 · 英文: 学术读写流利(CET-4、CET-6)
Technical Skills专业技能
- Programming编程语言
- Python, C/C++
- ML & LLM Frameworks机器学习与大模型框架
- PyTorch, TensorFlow, Hugging Face Transformers, vLLM
- Evaluation & Analysis评测与分析
- LLM-as-a-Judge, Reward Models, Prompt/Few-shot Methods, Wilcoxon, Spearman, TF-IDF
- Developer Tools开发工具
- Linux, Git, LaTeX
References推荐人
Available upon request.可应要求提供。