| 2026 | ACL | MQM Re-Annotation: A Technique for Collaborative Evaluation of Machine Translation. | Parker Riley, Daniel Deutsch, Mara Finkelstein, Colten DiIanni, Juraj Juraska, Markus Freitag |
| 2025 | ACL | WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialects. | Daniel Deutsch, Eleftheria Briakou, Isaac Rayburn Caswell, Mara Finkelstein, Rebecca Galor, Juraj Juraska, Geza Kovacs, Alison Lui, Ricardo Rei, Jason Riesa, Shruti Rijhwani, Parker Riley, Elizabeth Salesky, Firas Trabelsi, Stephanie Winkler, Biao Zhang, Markus Freitag |
| 2025 | ACL | Enhancing Human Evaluation in Machine Translation with Comparative Judgement. | Yixiao Song, Parker Riley, Daniel Deutsch, Markus Freitag |
| 2025 | EMNLP | Don't Sweat the Small Stuff: Segment-Level Meta-Evaluation Based on Pairwise Difference Correlation. | Colten DiIanni, Daniel Deutsch |
| 2025 | EMNLP | SSA-COMET: Do LLMs Outperform Learned Metrics in Evaluating MT for Under-Resourced African Languages? | Senyu Li, Jiayi Wang, Felermino D. M. A. Ali, Colin Cherry, Daniel Deutsch, Eleftheria Briakou, Rui Sousa-Silva, Henrique Lopes Cardoso, Pontus Stenetorp, David Ifeoluwa Adelani |
| 2025 | ICML | From Jack of All Trades to Master of One: Specializing LLM-based Autoraters to a Test Set. | Mara Finkelstein, Daniel Deutsch, Parker Riley, Juraj Juraska, Geza Kovacs, Markus Freitag |
| 2025 | ICML | Overestimation in LLM Evaluation: A Controlled Large-Scale Study on Data Contamination's Impact on Machine Translation. | Muhammed Yusuf Kocyigit, Eleftheria Briakou, Daniel Deutsch, Jiaming Luo, Colin Cherry, Markus Freitag |
| 2024 | NAACL | On the Role of Summary Content Units in Text Summarization Evaluation. | Marcel Nawrath, Agnieszka Nowak, Tristan Ratz, Danilo C. Walenta, Juri Opitz, Leonardo F. R. Ribeiro, Joo Sedoc, Daniel Deutsch, Simon Mille, Yixin Liu, Sebastian Gehrmann, Lining Zhang, Saad Mahamood, Miruna Clinciu, Khyathi Raghavi Chandu, Yufang Hou |
| 2024 | NAACL | Finding Replicable Human Evaluations via Stable Ranking Probability. | Parker Riley, Daniel Deutsch, George F. Foster, Viresh Ratnakar, Ali Dabirmoghaddam, Markus Freitag |
| 2024 | NAACL | LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback. | Wenda Xu, Daniel Deutsch, Mara Finkelstein, Juraj Juraska, Biao Zhang, Zhongtao Liu, William Yang Wang, Lei Li, Markus Freitag |
| 2023 | ACL | A Needle in a Haystack: An Analysis of High-Agreement Workers on MTurk for Summarization. | Lining Zhang, Simon Mille, Yufang Hou, Daniel Deutsch, Elizabeth Clark, Yixin Liu, Saad Mahamood, Sebastian Gehrmann, Miruna Clinciu, Khyathi Raghavi Chandu, Joo Sedoc |
| 2023 | EACL | Incorporating Question Answering-Based Signals into Abstractive Summarization via Salient Span Selection. | Daniel Deutsch, Dan Roth |
| 2023 | EMNLP | Ties Matter: Meta-Evaluating Modern Metrics with Pairwise Accuracy and Tie Calibration. | Daniel Deutsch, George F. Foster, Markus Freitag |
| 2022 | ACL | Benchmarking Answer Verification Methods for Question Answering-Based Summarization Evaluation Metrics. | Daniel Deutsch, Dan Roth |
| 2022 | EMNLP | On the Limitations of Reference-Free Evaluations of Generated Text. | Daniel Deutsch, Rotem Dror, Dan Roth |
| 2022 | NAACL | Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metrics. | Daniel Deutsch, Rotem Dror, Dan Roth |
| 2021 | CoNLL | Understanding the Extent to which Content Quality Metrics Measure the Information Quality of Summaries. | Daniel Deutsch, Dan Roth |
| 2020 | COLING | Is Killed More Significant than Fled? A Contextual Model for Salient Event Detection. | Disha Jindal, Daniel Deutsch, Dan Roth |
| 2019 | CoNLL | A General-Purpose Algorithm for Constrained Sequential Inference. | Daniel Deutsch, Shyam Upadhyay, Dan Roth |
| 2019 | EMNLP | Summary Cloze: A New Task for Content Selection in Topic-Focused Summarization. | Daniel Deutsch, Dan Roth |
| 2018 | ACL | A Distributional and Orthographic Aggregation Model for English Derivational Morphology. | Daniel Deutsch, John Hewitt, Dan Roth |