NSF IIS Program
Project # 1065250

Development and Evaluation of Search Technology
for Discovery of Evidence in Civil Litigation


News

     
  • Prof. Doug Oard is presenting a poster on "Support for Interactive Identification of Mentioned Entities in Conversational Speech" at the ACM SIGIR Conference in Tokyo, Japan in August, 2017.
  • Ph.D. student Ning Gao's journal article on "Person Entity Linking in Email with NIL Detection," written with Profs. Doug Oard and Mark Deredze, is now online at the Journal of the Association for Information Science and Technology.
  • David Graus, who had visited our group in 2014, received his Ph.D. from the University of Amsterdam with a dissertaion on "Entities of Interest." Professor Oard served as an external examiner for his dissertation.
  • Ph.D. Student Mossaab Bagdouri successfully defended his dissertation on "Cross-Platform Question Answering in Social Networking Services" in May, 2017. Dr. Dave Lewis, who had worked with Mossaab on e-dsicivery reserch earlier in his parogram, participated as an external examiner.
  • Prof. Doug Oard presented a paper on "When is it Rational to Review for Privilege?" at the DESI Workshop at ICAIL in London, OK in June, 2017.
  • Profs. Doug Oard, Bill Hamilton and Daisy Wang hosted a Confab at the University of Florida in Gainesville for a dozen faculty with interests in langauge technology from around the state of Florida to discus potential collaboration on E-Discovery and other research topics.
  • Ph.D. Student Jyothi Vinjumur is visiting Professor Matt Lease's research Lab at the University of Texas at Austin in Spring, 2017.
  • Prof. Doug Oard is visiting the University of Florida on sabbatical in the 2017-2018 academic year, working with Prof. Daisy Wang in Computer Science and Prof. Bill Hamilton in the Law School on E-Discovery.
  • The National Science Foundation awarded our team a grant to develop new techniques for Search Among Sensitive Content that inclues both protection of privileged content in e-discovery and protection of sensitive content in other applications. The new grant starts in September, 2016.
  • Prof. Doug Oard presented a poster on "Pearson Rank: A Head-Weighted Gap-Sensitive Score-Based Correlation Coefficient" at the SIGIR Conference in Pisa, Italy in July, 2016.
  • Prof. Doug Oard presented an invited talk on "Search Among Sensitive Content" at the SIGIR Workshop on Privacy-Preserving Information Retrieval in Pisa, Italy in July, 2016.
  • Ph.D. student Jyothi Vinjumur is working on E-Discovery during her summer 2016 internship at H5 in San Francisco.
  • Prof. Doug Oard presented a talk on "Search Among Secrets" at ISKO Singapore in May, 2016.
  • Dr. Dave Lewis presented a talk on "E-Discovery: Gateway Drug to Data Science" at the 4th International Workshop on Digital Forensics Curriculum Standards in Urbana, IL in May, 2016.
  • Ph.D. student Jyothi Vinjumur successfully defended her dissertation proposal on "Retrieval of Privileged Emails by Optimizing the Cost of E-Discovery" in May 2016.
  • Dr. Dave Lewis presented a talk on "Information Retrieval and Machine Learning for E-Discovery" at the Watson, Esq.: Will Your Next Lawyer be a Machine? conference at Vanderbilt University in Nashville in April, 2016.
  • Ph.D. student Jyothi Vinjumur presented a full paper on "An AID for Avoiding Inadvertent Disclosure: Supporting Interactive Review for Privilege in E-Discovery" at the first ACM SIGIR Conference on Human Interaction in Information Retrieval (CHIIR) in Chapel Hill, NC in March, 2016.
  • Dr. Dave Lewis presented a talk on "Avoiding the TAR Pits: A Practitioners Roadmap to Technology Assisted Review and Predictive Coding" at LegalTech in New York City in February, 2016.
  • Dr. Dave Lewis gave a talk on "Process Design for Technology-Assisted Review at the Chicago Law and Technology Meetup in January, 2016.
  • Prof. Doug Oard presented a keynote address on "Information Abolition" at the Australian Document Computing Symposium near Sydney, Australia in December, 2015.
  • Prof. Doug Oard presented an invited talk on "Making Lawyers Happy" at the Forum for Information Retrieval Evaluation in Gandhinagar, India in December, 2015.
  • Prof. Doug Oard gave a talk on E-Discovery at the University of Illinois Urbana-Champaign in November, 2016.
  • Ph.D. student Jyothi Vinjumur presented a poster on "Finding the Privileged Few: Supporting Privilege Review for E-Discovery" at the ASIST 2015 Annual Meeting in St Louis in November, 2015.
  • Prof. Doug Oard presented a keynote address on "Beyond Information Retrieval: When and How Not to Find Things" at CLEF in Toulouse, France in September, 2015.
  • Assoc. Prof. David Kirsch and Ph.D. Student Heejung Byun presented their research on email timing norms at the Academy of Management conference in Vancouver, Canada in August, 2015.
  • Project members presented two papers at SIGIR 2015 in Santiago Chile in August 2015: (1) a full paper by postdoc Jia Paik on "A Probabilistic Model for Information Retrieval Based on Maximum Value Distribution," and (2) a short paper by Ph.D. student Ning Gao and Prof. Doug Oard on "A Head-Weighted Gap-Sensitive Correlation Coefficient."
  • Prof. Doug Oard presented a talk on e-discovery at the Universidade Federal de Minas Gerais (UFMG) in Belo Horizante, Brazil in August, 2015.
  • Dr. Dave Lewis presented a talk on "Natural Language Processing in Electronic Discovery: Status and Potential" as the Web of Law Workshop at NYU in New York City in July, 2015.
  • Undergrad Computer Science student Bryan Toth has returned from his Spring 2015 semester abroad at the University of Melbourne in Australia.
  • Three of us (Prof. Doug Oard, Dr. Dave Lewis and advisory board member Jason Baron) helped to organize the DESI VI Workshop at the International Conference on AI and Law in San Diego in June, 2015.
  • Ph.D. student Jyothi Vinjumur presented a full paper on "Evaluating Expertise and Sample Bias Effects for Privilege Classification in E-Discovery" at the International Conference on AI and Law in San Diego in June, 2015.
  • Assoc. Prof. David Kirsch made a panel presentation on our work with the Enron, Avocado and Brobeck email collections at the Archiving Email Symposium at the Library of Congress in June, 2015.
  • Ph.D. student Jyothi Vinjumur won the best doctoral consortium presentation at the Doctoral Consortium at the International Conference on AI and Law in San Diego in June, 2015!
  • Ph.D. student Ning Gao worked on email analysis during her internship at Microsoft Research in Redmond, WA in Summer, 2015.
  • Two project members (Prof. Doug Oard and Ph.D. student Jyothi Vinjumur) participated in the DIMACS workshop on Social and Collaborative Information Seeking at Rutgers University in May, 2015.
  • Masters student Rashmi Sankepally successfully defended her Masters of Information Management thesis on "An Information Retrieval Test Collection for English SMS Conversations" in April 2015.
  • Prof. Doug Oard gave a talk on E-Discovery at Louisiana State University in April, 2015.
  • The University of Maryland had planned to screen the documentary Decade of Discovery on April 7, 2015, followed by a panel discussion including the film maker, two faculty who have taught Maryland's e-discovery course, and one graduate of that course who is now working in the industry. A regional power failure resulted in rescheduling the event until Fall, 2015, however!
  • Masters student Anant Bhatt is visiting from the Indian Institute of Technology Kharagpur for a month in February 2015 to work with us on search technology for email archives.
  • The Linguistic Data Consortium released the Avocado Research Email Collection in February, 2015.
  • Prof. Doug Oard gave a talk on e-discovery at the University of Florida in January, 2015.
  • Ph.D. student Ning Gao presented a paper on "A Test Collection for Email Entity Linking" at the NIPS Advanced Knowledge Base Construction workshop in Montreal in December 2014.
  • Dr. Dave Lewis is back at Maryland for a couple of months in Fall, 2014.
  • Dr. Jia Paik presented a full paper on "A Fixed-Point Method for Weighting Terms in Verbose Informational Queries" at CIKM in Shanghai, China in November, 2014.
  • Prof. Doug Oard and Dr. Dave Lewis presented invited talks on review for privilege in e-discovery at the July 2014 SIGIR workshop on Privacy-Preserving Information Retrieval in Australia.
  • Ph.D. student Jyothi Vinjumur presented a short paper on "Assessing the Reusability of an E-Discovery Privilege Test Collection" at SIGIR in Australia in July, 2014.
  • University of Amsterdam Ph.D. student David Graus, who visited us earlier this year, presented the first published paper with the new Avocado collection as a short paper at SIGIR 2014 entitled "Recipient Recommendation in Enterprises using Communication Graphs and Email Content."
  • Dr. Dave Lewis participated in a panel on commercializing IR research at the SIGIR 2014 Symposium on IR in Practice in Melbourne, Australia in July, 2014.
  • Assoc. Prof. David Kirsch presented a talk on the first work with our new Avocado collection at a workshop at the Copenhagen Business School in Denmark in June, 2014.
  • Ph.D. student Jyothi Vinjumur is visiting the University of Amsterdam for a month in June, 2014 to work with the E-Discovery project there.
  • Prof. Doug Oard attended the world premiere of the documentary film "Decade of Discovery" at the Hoboken Film Festival on May 31, 2014, including a post-showing discussion with the audience on the technology/law intersection
  • Ph.D. student Ning Gao is presenting her work on evaluation based on very sparse relevance judgments at the European Conference on Information Retrieval in April, 2014.
  • David Graus, a Ph.D. student working on E-Discovery at the University of Amsterdam, is visiting for a month in February, 2014.
  • Jason Baron, Esq., is teaching a course on E-Discovery at the University of Maryland iSchool in the Spring semester, 2014.
  • Dr. David Lewis is teaching a course on Information Retrieval for E-Discovery at the Georgetown University Computer Science Department in the Fall semester, 2013.
  • Ph.D. student Mossaab Bagdouri presented a short paper on "Towards Minimizing the Annotation Cost of Certified Text Classification" at CIKM 2013 in October in San Francisco.
  • Dr. David Lewis is spending the second half of 2013 in residence at the University of Maryland.
  • Ph.D. student Mossaab Bagdouri presented a full paper on "Sequential Testing in Classifier Evaluation Yields Biased Estimates of Effectiveness" at SIGIR 2013 in Dublin, Ireland.
  • Prof. Doug Oard, Dr. Dave Lewis, and Jason Baron, Esq. helped to organize the June 2013 DESI V workshop in Rome, Italy.
  • Dr. Maura Grossman is teaching a course in Electronic Discovery at the Columbia Law School in the Spring semester, 2012.
  • Prof. Doug Oard and Jason Baron, Esq., are leading a graduate course on e-discovery at the College of Information Studies of the University of Maryland, College Park, in the Spring semester, 2012.
  • Dr. William Webber gave a talk at the University of North Carolina, Chapel Hill, entitled Do detailed instructions improve assessor agreement?, on March 23, 2012.
  • Dr. Maura Grossman participated on a panel entitled "Defending the Use of Analytical Software in Civil Discovery", with Hon. Elizabeth D. Laporte, Conor R. Crowley, Laura Kibble, and Howard J. C. Nichols, at the Georgetown University Law Center's Eighth Annual Advanced eDiscovery Institute in Crystal City, VA, on November 18th, 2011.
  • Prof. Doug Oard, Dr. David D. Lewis, and Dr. Maura Grossman helped organize the meetings of the TREC Legal Track at the 20th Annual Text REtrieval Conference (TREC 2011), on November 14th to 17th, 2011.
  • Dr. David D. Lewis appeared on the panel Predictive Coding: A More Efficient and Less Expensive Search – But will the Courts Agree? , on October 27, 2011, as part of the Federal Bar Association, Western Pennsylvania Chapter, E-Discovery Series.
  • Dr. David D. Lewis appeared on the panels "Improving Searches; Recall & Precision, TREC Study, Iterative Search Term Analysis" and "Machine Learning and Automated Review" on October 11, 2011 at the Chicago Association for Litigation Support annual CALSMposium .
  • Dr. William Webber presented a talk entitled "Are lawyers redundant? Automation and reliability in e-discovery", examining evidence for the comparative reliability of technology-assisted and manual review, at the University of Delaware, Newark, DE, USA, on September 25th, 2011.
  • Prof. Doug Oard and Dr. William Webber are running an e-discovery reading group in the fall semester of 2011 at the University of Maryland.
  • Prof. Doug Oard gave a talk at Emory University, Atlanta, on September 20, 2011, entitled "Evaluating E-Discovery Search".
  • Dr. William Webber presented his paper Re-examining the Effectiveness of Manual Review at SIRE 2011 (see below).
  • Dr. David D. Lewis presented the talk "Finite Population Annotation" and appeared on the panel "Future Directions" at SIRE 2011 (see below).
  • Prof. Doug Oard organized the SIGIR 2011 Information Retrieval for E-Discovery (SIRE) Workshop in Beijing, China, on July 28th, 2011. Participants came from academia, industry, and the legal profession. Research and position papers were presented, and panel discussions conducted.
  • Prof. Doug Oard gave a talk at Microsoft Research Asia on "Evaluating E-Discovery Search: The TREC Legal Tack" on July 19, 2011.
  • Dr. David D. Lewis presented the lecture Machine Learning for Discovery in Legal Cases (Slides) on June 24, 2011, at the Machine Learning Summer School 2011, Purdue University, West Lafayette, IN.
  • Dr. Maura R. Grossman presented her paper Inconsistent Assessment of Responsiveness in E-Discovery: Difference of Opinion or Human Error?, joint work with Professor Gordon V. Cormack of the University of Waterloo, Canada, at DESI IV (see below).
  • Dr. David D. Lewis presented a keynote talk Data, Knowledge, and Discovery in Legal Cases at the Artificial Intelligence and Law Conference at the University of Pittsburgh, Pittsburgh, PA, USA, on June 7th, 2011.
  • Prof. Doug Oard organized the ICAIL 2011 Workshop on Setting Standards for Searching Electronically Stored Information in Discovery Proceedings (DESI IV), at the University of Pittsburgh, Pittsburgh, PA, USA, on June 6th, 2011. Research and position papers were presented, and panel discussions here held.
   Project Pages:

Page created: September 27, 2011
Page last updated: May 14, 2012

This material is based upon work supported by the National Science Foundation under Grant No. 1065250. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.