NSF IIS Program
Project # 1065250

Development and Evaluation of Search Technology
for Discovery of Evidence in Civil Litigation


  • Jyothi K. Vinjumur and Douglas W. Oard, Finding the privileged Few: Supporting Privilege Review for E-Discovery, in Annual Meeting of the Association for Information Science and Technology, 4 pages, St. Louis, MO, 2015.
  • Jiaul Paik, A Probabilistic Model for Information Retrieval Based on Maximum Value Distribution, Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, Santiago, Chile, 2015.
  • Ning Gao and Douglas W. Oard, A Head-Weighted Gap-Sensitive Correlation Coefficient, Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, Santiago, Chile, 2015.
  • Jyothi Vinjumur, Evaluating Expertise and Sample Bias Effects for Privilege Classification in E-Discovery, International Conference on Artificial Intelligence and Law, San Diego, CA, 2015.
  • Jyothi K. Vinjumur, Methodology for Constructing Test Collections using Collaborative Annotation, International Conference on Artificial Intelligence and Law Doctoral Consortium, San Diego, CA, 2015.
  • Rashmi Sankepally, An Information Retrieval Test Collection for English SMS Conversations, Masters Thesis, College of Information Studies, University of Maryland, College Park, MD 2015.
  • William Webber and Douglas W. Oard, Metrics in Predictive Coding, submitted to Perspectives on Predictive Coding and Other Advanced Search Technologies for the E-Discovery Practitioner (working title), American Bar Association, to appear in 2015.
  • Ning Gao, Douglas W. Oard and Mark Dredze, A Test Collection for Email Entity Linking". NIPS Workshop on Automated Knowledge Base Construction, Montreal, Canada, 2014.
  • Jiaul H. Paik and Douglas W. Oard, "A Fixed-Point Method for Weighting Terms in Verbose Informational Queries," ACM International Conference on Information and Knowledge Management, Shanghai, China, 2014.
  • Heejung Byun and David A. Kirsch "Organizational Timing Norms: Evidence from Email Time-to-Responses," in Workshop on the Micro Foundations of Social Networks and the Implications for Strategy and Entrepreneurship Research, Copenhagen Business School, Denmark, 2014.
  • Jyothi K. Vinjumur, Douglas W. Oard and Jiaul H. Paik, "Assessing the Reliability and Reusability of an E-Discovery Privilege Test Collection," in 37th Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval, 2014.
  • Rajiv Jain, Douglas W. Oard and David Doermann, "Scalable Ranked Retrieval Using Document Images," in 21st SPIE Document Recognition and Retrieval Conference, 2014. (Best Student Paper award)
  • Ning Gao, William Webber and Douglas W. Oard, "Reducing Reliance on Relevance Judgments for System Comparison by Using Expectation- Maximization," in Proceedings of the of the 36th European Conference on Information Retrieval, 2014.
  • Mossaab Bagdouri, William Webber, David D. Lewis and Douglas W. Oard, Towards Minimizing the Annotation Cost of Certified Text Classification, CIKM 2013.
  • Falk Scholer, Diane Kelly, Hanseul S. Lee, Wan-ching Wu, and William Webber, "The Effect of Threshold Priming and Need for Cognition on Relevance Assessment", SIGIR 2013.
  • William Webber, Mossaab Bagdouri, David D. Lewis, and Douglas W. Oard, "Sequential Testing in Classifier Evaluation Yields Biased Estimates of Effectiveness", SIGIR 2013.
  • Praveen Chandar, William Webber, and Ben Carterette, "Document Features Predicting Assessor Disagreement", SIGIR 2013.
  • William Webber and Jeremy Pickens, "Assessor Disagreement and Text Classifier Accuracy", SIGIR 2013.
  • Xianzhi Du, Wael Abd-Almageed, and David Doermann, Large-scale Signature Matching using Multi-Stage Hashing, ICDAR 2013.
  • Jayant Kumar and David Doermann, Unsupervised Classification of Structurally Similar Document Images, ICDAR 2013.
  • Douglas W. Oard and William Webber, Information Retrieval for E-Discovery, Foundations and Trends in Information Retrieval, Foundations and Trends in Information Retrieval, Volume 7, Issues 2--3 (June, 2013), pages 99--237 (authors' copy).
  • William Webber, Approximate Recall Confidence Intervals, ACM Transactions on Information Systems, Volume 31, Issue 1 (January, 2013), pages 2:1--33 (arXiv preprint; code and data").
  • Bench-Capon, T, Araskiewicz, M., Ashley, K., Atkinson, K., Bex, F., Borges, F., Bourcier, D., Bourgine, P., Conrad, J. G., Francesconi, E., Gordon, T. F., Governatori, G., Leidner, J. L., Lewis, D. D., Loui, R. P., McCarty, J. L., Prakken, H., Schilder, F., Schweighofer, E., Thompson, P., Tyrell, A., Verheij, B, Walson, D. N., Wyner, A. Z. A History of AI and Law in 50 papers: 25 Years of the International Conference on AI and Law. Artificial Intelligence and Law, vol. 20, no. 3, pp. 215-319, September 2012.
  • Maura R. Grossman and Gordon V. Cormack, Efficient E-Discovery, ABA Journal, April 2012, page 31.
  • William Webber, Praveen Chandar, and Ben Carterette, Alternative Assessor Disagreement and Retrieval Depth, CIKM 2012.
  • Jayant Kumar, Peng Ye, and David Doermann, Learning Document Structure for Retrieval and Classification, ICPR 2012.
  • William Webber, Bryan Toth, and Marjorie Desamito, Effect of Written Instructions on Assessor Agreement, SIGIR 2012.
  • Rajiv Jain and David S. Doermann, Logo Retrieval in Document Images, IAPR 2012.
  • Jayant Kumar, Jaishanker Pillai, and David Doermann, http://lampsrv02.umiacs.umd.edu/pubs/Papers/jayantkumar-11c/jayantkumar-11c.pdfDocument Image Classification and Labeling using Multiple Instance Learning, ICDAR 2011.
  • Maura R. Grossman and Gordon V. Cormack, Inconsistent Assessment of Responsiveness in E-Discovery: Difference of Opinion or Human Error?, DESI IV, Pittsburgh, USA, June 2011.
  • William Webber, Re-examining the Effectiveness of Manual Review, SIGIR Information Retrieval for E-Discovery Workshop, Beijing, China, July 2011.
  • Maura R. Grossman and Gordon V. Cormack, Technology-Assisted Review in E-Discovery can be More Effecive and More Efficient than Exhaustive Manual Review, Richmond J. of Law and Technology, XVII(3), 2011.
  • Gordon V. Cormack, Maura R. Grossman, Bruce Hedin, and Douglas W. Oard, Overview of the TREC 2010 Legal Track, NIST Special Publication SP 500-294, 2011.
  • William Webber, Douglas W. Oard, Falk Scholer, and Bruce Hedin, Assessor Error in Stratified Evaluation, CIKM, Toronto, Canada, October 2010.
   Project Pages:

Page created: January 30, 2012
Page last updated: May 14, 2012

This material is based upon work supported by the National Science Foundation under Grant No. 1065250. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.