| 2026 | ACL | Improving the Distributional Alignment of LLMs using Supervision. | Gauri Kambhatla, Sanjana Gautam, Angela Zhang, Alexander Liu, Ravi Srinivasan, Junyi Jessy Li, Matthew Lease |
| 2025 | ICTIR | Preaching to the ChoIR: Lessons IR Should Share with AI. | Gianluca Demartini, Claudia Hauff, Matthew Lease, Stefano Mizzaro, Kevin Roitero, Mark Sanderson, Falk Scholer, Chirag Shah, Damiano Spina, Paul Thomas, Arjen P. de Vries, Guido Zuccon |
| 2024 | AAAI | A General Model for Aggregating Annotations AcrossSimple, Complex, and Multi-object Annotation Tasks (Abstract Reprint). | Alexander Braylan, Madalyn Marabella, Omar Alonso, Matthew Lease |
| 2024 | EMNLP | Promoting Constructive Deliberation: Reframing for Receptiveness. | Gauri Kambhatla, Matthew Lease, Ashwin Rajadesingan |
| 2024 | IUI | Snapper: Accelerating Bounding Box Annotation in Object Detection Tasks with Find-and-Snap Tooling. | Alex C. Williams, Min Bai, Jonathan Buck, Tristan J. McKinney, Amy Rechkemmer, Koushik Kalyanaraman, Matthew Lease, Patrick Haffner, Xiong Zhou, Li Erran Li |
| 2024 | NAACL | Benchmark Transparency: Measuring the Impact of Data on Evaluation. | Venelin Kovatchev, Matthew Lease |
| 2023 | ECIR | New Metrics to Encourage Innovation and Diversity in Information Retrieval Approaches. | Mehmet Deniz Trkmen, Matthew Lease, Mcahid Kutlu |
| 2023 | WWW | Same Same, But Different: Conditional Multi-Task Learning for Demographic-Specific Toxicity Detection. | Soumyajit Gupta, Sooyong Lee, Maria De-Arteaga, Matthew Lease |
| 2022 | ACL | ProtoTEx: Explaining Model Decisions with Prototype Tensors. | Anubrata Das, Chitrank Gupta, Venelin Kovatchev, Matthew Lease, Junyi Jessy Li |
| 2022 | WWW | Measuring Annotator Agreement Generally across Complex Structured, Multi-object, and Free-text Annotation Tasks. | Alexander Braylan, Omar Alonso, Matthew Lease |
| 2022 | UAI | Learning a neural Pareto manifold extractor with constraints. | Soumyajit Gupta, Gurpreet Singh, Raghu Bollapragada, Matthew Lease |
| 2021 | AIES | Towards Unbiased and Accurate Deferral to Multiple Experts. | Vijay Keswani, Matthew Lease, Krishnaram Kenthapadi |
| 2021 | CHI | The Psychological Well-Being of Content Moderators: The Emotional Labor of Commercial Moderation and Avenues for Improving Support. | Miriah Steiger, Timir J. Bharucha, Sukrit Venkatagiri, Martin J. Riedl, Matthew Lease |
| 2021 | CSCW | Investigating and Mitigating Biases in Crowdsourced Data. | Danula Hettiachchi, Mark Sanderson, Jorge Gonalves, Simo Hosio, Gabriella Kazai, Matthew Lease, Mike Schaekermann, Emine Yilmaz |
| 2021 | IJCAI | Human-AI Collaboration with Bandit Feedback. | Ruijiang Gao, Maytal Saar-Tsechansky, Maria De-Arteaga, Ligong Han, Min Kyung Lee, Matthew Lease |
| 2021 | ICTIR | Pareto Solutions vs Dataset Optima: Concepts and Methods for Optimizing Competing Objectives with Constraints in Retrieval. | Soumyajit Gupta, Gurpreet Singh, Anubrata Das, Matthew Lease |
| 2021 | KDD | Aggregating Complex Annotations via Merging and Matching. | Alexander Braylan, Matthew Lease |
| 2020 | HCOMP | Fast, Accurate, and Healthier: Interactive Blurring Helps Moderators Reduce Exposure to Harmful Content. | Anubrata Das, Brandon Dang, Matthew Lease |
| 2020 | ICTIR | Efficient Test Collection Construction via Active Learning. | Md. Mustafizur Rahman, Mcahid Kutlu, Tamer Elsayed, Matthew Lease |
| 2020 | WWW | Modeling and Aggregation of Complex Annotations via Annotation Distances. | Alexander Braylan, Matthew Lease |
| 2019 | ECIR | Correlation, Prediction and Ranking of Evaluation Metrics in Information Retrieval. | Soumyajit Gupta, Mcahid Kutlu, Vivek Khetan, Matthew Lease |
| 2019 | IUI | Explainable modeling of annotations in crowdsourcing. | An T. Nguyen, Matthew Lease, Byron C. Wallace |
| 2019 | WWW | Constructing Test Collections using Multi-armed Bandits and Active Learning. | Md. Mustafizur Rahman, Mcahid Kutlu, Matthew Lease |
| 2018 | AAAI | An Interpretable Joint Graphical Model for Fact-Checking From Crowds. | An T. Nguyen, Aditya Kharosekar, Matthew Lease, Byron C. Wallace |
| 2018 | CIKM | When Rank Order Isn't Enough: New Statistical-Significance-Aware Correlation Measures. | Mcahid Kutlu, Tamer Elsayed, Maram Hasanain, Matthew Lease |
| 2018 | HCOMP | Your Behavior Signals Your Reliability: Modeling Crowd Behavioral Traces to Ensure Quality Relevance Annotations. | Tanya Goyal, Tyler McDonnell, Mcahid Kutlu, Tamer Elsayed, Matthew Lease |
| 2018 | SIGIR | Crowd vs. Expert: What Can Relevance Judgment Rationales Teach Us About Assessor Disagreement? | Mcahid Kutlu, Tyler McDonnell, Yassmine Barkallah, Tamer Elsayed, Matthew Lease |
| 2018 | UIST | Believe it or not: Designing a Human-AI Partnership for Mixed-Initiative Fact-Checking. | An T. Nguyen, Aditya Kharosekar, Saumyaa Krishnan, Siddhesh Krishnan, Elizabeth Tate, Byron C. Wallace, Matthew Lease |
| 2017 | AAAI | Active Discriminative Text Representation Learning. | Ye Zhang, Matthew Lease, Byron C. Wallace |
| 2017 | ACL | Aggregating and Predicting Sequence Labels from Crowd Annotations. | An Thanh Nguyen, Byron C. Wallace, Junyi Jessy Li, Ani Nenkova, Matthew Lease |
| 2017 | ACL | Exploiting Domain Knowledge via Grouped Weight Sharing with Application to Text Categorization. | Ye Zhang, Matthew Lease, Byron C. Wallace |
| 2017 | IJCAI | The Many Benefits of Annotator Rationales for Relevance Judgments. | Tyler McDonnell, Mcahid Kutlu, Tamer Elsayed, Matthew Lease |
| 2016 | HCOMP | Why Is That Relevant? Collecting Annotator Rationales for Relevance Judgments. | Tyler McDonnell, Matthew Lease, Mcahid Kutlu, Tamer Elsayed |
| 2016 | HCOMP | Probabilistic Modeling for Crowdsourcing Partially-Subjective Ratings. | An Thanh Nguyen, Matthew Halpern, Byron C. Wallace, Matthew Lease |
| 2016 | SIGIR | ArabicWeb16: A New Crawl for Today's Arabic Web. | Reem Suwaileh, Mcahid Kutlu, Nihal Fathima, Tamer Elsayed, Matthew Lease |
| 2016 | UAI | A Correlated Worker Model for Grouped, Imbalanced and Multitask Data. | An T. Nguyen, Byron C. Wallace, Matthew Lease |
| 2015 | ECIR | A Discriminative Approach to Predicting Assessor Accuracy. | Hyun Joon Jung, Matthew Lease |
| 2015 | HCOMP | Modeling Temporal Crowd Work Quality with Limited Supervision. | Hyun Joon Jung, Matthew Lease |
| 2015 | HCOMP | Combining Crowd and Expert Labels Using Decision Theoretic Active Learning. | An Thanh Nguyen, Byron C. Wallace, Matthew Lease |
| 2014 | HCOMP | Predicting Next Label Quality: A Time-Series Model of Crowdwork. | Hyun Joon Jung, Yubin Park, Matthew Lease |
| 2014 | SIGIR | Multidimensional relevance modeling via psychometrics and crowdsourcing. | Yinglong Zhang, Jin Zhang, Matthew Lease, Jacek Gwizdka |
| 2013 | HCOMP | SQUARE: A Benchmark for Research on Computing Crowd Consensus. | Aashish Sheshadri, Matthew Lease |
| 2012 | HCOMP | Improving Quality of Crowdsourced Labels via Probabilistic Matrix Factorization. | Hyun Joon Jung, Matthew Lease |
| 2012 | SIGIR | Inferring missing relevance judgments from crowd workers via probabilistic matrix factorization. | Hyun Joon Jung, Matthew Lease |
| 2012 | SIGIR | Crowdsourcing for search evaluation and social-algorithmic search. | Matthew Lease, Omar Alonso |
| 2012 | SIGIR | Parallelizing ListNet training using spark. | Shilpa Shukla, Matthew Lease, Ambuj Tewari |
| 2011 | AAAI | Improving Consensus Accuracy via Z-Score and Weighted Voting. | Hyun Joon Jung, Matthew Lease |
| 2011 | AAAI | On Quality Control and Machine Learning in Crowdsourcing. | Matthew Lease |
| 2011 | CIKM | Supervised language modeling for temporal resolution of texts. | Abhimanu Kumar, Matthew Lease, Jason Baldridge |
| 2011 | SIGIR | Crowdsourcing for information retrieval: principles, methods, and applications. | Omar Alonso, Matthew Lease |
| 2011 | SIGIR | Learning to rank from a noisy crowd. | Abhimanu Kumar, Matthew Lease |
| 2011 | SIGIR | Active learning to maximize accuracy vs. effort in interactive information retrieval. | Aibo Tian, Matthew Lease |
| 2011 | WSDM | Crowdsourcing 101: putting the WSDM of crowds to work for you. | Omar Alonso, Matthew Lease |
| 2011 | WSDM | Crowdsourcing for search and data mining. | Vitor R. Carvalho, Matthew Lease, Emine Yilmaz |
| 2010 | NAACL | Crowdsourcing Document Relevance Assessment with Mechanical Turk. | Catherine Grady, Matthew Lease |
| 2009 | ECIR | Regression Rank: Learning to Meet the Opportunity of Descriptive Queries. | Matthew Lease, James Allan, W. Bruce Croft |
| 2009 | SIGIR | An improved markov random field model for supporting verbose queries. | Matthew Lease |
| 2007 | CIKM | Natural language processing for information retrieval: the time is ripe (again). | Matthew Lease |
| 2006 | AAAI | A Look at Parsing and Its Applications. | Matthew Lease, Eugene Charniak, Mark Johnson, David McClosky |
| 2006 | ACL | PCFGs with Syntactic and Prosodic Indicators of Speech Repairs. | John Hale, Izhak Shafran, Lisa Yung, Bonnie J. Dorr, Mary P. Harper, Anna Krasnyanskaya, Matthew Lease, Yang Liu, Brian Roark, Matthew G. Snover, Robin Stewart |
| 2006 | ICASSP | Reranking for Sentence Boundary Detection in Conversational Speech. | Brian Roark, Yang Liu, Mary P. Harper, Robin Stewart, Matthew Lease, Matthew G. Snover, Izhak Shafran, Bonnie J. Dorr, John Hale, Anna Krasnyanskaya, Lisa Yung |
| 2006 | LREC | Linguistic Resources for Speech Parsing. | Ann Bies, Stephanie M. Strassel, Haejoong Lee, Kazuaki Maeda, Seth Kulick, Yang Liu, Mary P. Harper, Matthew Lease |
| 2006 | LREC | SParseval: Evaluation Metrics for Parsing Speech. | Brian Roark, Mary P. Harper, Eugene Charniak, Bonnie J. Dorr, Mark Johnson, Jeremy G. Kahn, Yang Liu, Mari Ostendorf, John Hale, Anna Krasnyanskaya, Matthew Lease, Izhak Shafran, Matthew G. Snover, Robin Stewart, Lisa Yung |
| 2006 | NAACL | Early Deletion of Fillers In Processing Conversational Speech. | Matthew Lease, Mark Johnson |
| 2005 | ICASSP | Parsing and its applications for conversational speech. | Matthew Lease, Eugene Charniak, Mark Johnson |
| 2005 | IJCNLP | Parsing Biomedical Literature. | Matthew Lease, Eugene Charniak |
| 2005 | NAACL | Effective Use of Prosody in Parsing Conversational Speech. | Jeremy G. Kahn, Matthew Lease, Eugene Charniak, Mark Johnson, Mari Ostendorf |
| 1999 | AMIA | Anatomical information in radiation treatment planning. | Ira J. Kalet, Jonn Wu, Matthew Lease, Mary M. Austin-Seymour, James F. Brinkley, Cornelius Rosse |