Your single source for new lessons on legal technology, e-discovery, and the people innovating behind the scenes.

Jay Leib Interviews Shimmy Messing on Computer-assisted Review

Jay Leib

In this installment of his interview series, Jay Leib—resident computer-assisted review expert at kCura—interviews Shimmy Messing of Responsive Data Solutions about the Best in Service partner’s extensive experience using Relativity Assisted Review. Shimmy is Responsive Data’s director of technology and co-founded the company more than six years ago.

Jay: Tell us a little bit about yourself and Responsive Data.

Shimmy: I started Responsive Data with three partners in 2007. Coming from past experience in litigation support, our goal was to build an organization that’s client-focused and proactive. We help our clients with both paper and electronic discovery, printing, trial prep, and everything in between—the whole discovery lifecycle. I’m our director of technology, managing our internal infrastructure and our client-facing software. We have a Relativity team, an IT team, and few others that fall under my wing.

Tell us about your early experiences with Assisted Review.

We started using Assisted Review back in 2011. We dove right in, as we were confident in the technology behind Relativity Analytics and Assisted Review. One of our earlier projects was more than 7 million records, and we learned a lot about optimizing our infrastructure, building a better index, understanding which types of data are best for an Assisted Review project, and more.

One key takeaway from those early projects was recognizing that we needed to be more selective about documents that have minimal text. The four corners rule—which emphasizes coding a document based only on the text within the four corners of a page, as opposed to its family members or custodians—helped guide that decision-making process a lot, and helped teach us how to recognize the content richness of a document.

What have you observed about Assisted Review’s accuracy compared to human review?

For our first few cases, to get our team familiar with the process and how it works, we spent two or three days training the system while simultaneously unleashing human review on the first chunk of documents. Then, we compared the results of the responsiveness and overturn rates between our human reviewers and Assisted Review. For example, after we got about 300,000 documents through human review, all overturned documents were given to a human expert for QC. Time and again, the experts agreed more often with Assisted Review than they did with the human reviewers.

We’ve done that on a few more cases, and we still offer that comparison for clients who aren’t as comfortable with computer-assisted review. We’re consistently seeing similar results.

Has your team used Assisted Review for any particularly interesting or unique projects?

Second requests can come from various government agencies as a result of an in-progress merger. These agencies come in and request that some specified information about both companies be produced within 30 or 45 or 60 days, to ensure everything’s being done by the book and there will be no anticompetitive consequences as a result of a merger. It’s in the merging companies’ best interests to comply with the scope and timeline, because it means the transaction can be completed sooner. But these requests yield millions of documents, sometimes with hundreds of custodians and terabytes of data.

Those are excellent cases for Assisted Review, because the tight turnaround and large data sets are what the technology is made to tackle. We can spend a relatively small amount of time training, and then unleash the power of Assisted Review for efficient and accurate productions.

We’ve gotten positive feedback from our clients in these cases. In fact, we recently finished three second requests for different practice groups within the same firm—which really speaks for itself in terms of client adoption.

What are your team’s and clients’ feelings about the workflow?

One nice thing about Assisted Review is that, to a degree, whether you’re dealing with 6 million or 600,000 documents, the process is similar and the steps you take to get the job done are consistent. That has made it easier for us to wrap our arms around the process, break it down into an effective workflow, and adapt that workflow for the idiosyncrasies of every case.

Our clients love being able to target key documents and focus human review on these most important documents early in a case. In some cases, we’ve been able to save our clients up to 40 percent of their projected costs. That’s pretty huge, because a lot of these cases involve very large figures.

What’s your best advice for folks who are new to Assisted Review?

Trust it, but be aware of it. The technology is there and it’s proven, but that doesn’t mean you shouldn’t know what it’s doing every step of the way. We advise clients to talk to opposing counsel about using computer-assisted review. It’s not taboo and it’s well-received—as long as we’re careful. Like any technology, it relies on people using it effectively. We encourage our clients to learn it, because the experience is better when you understand it.

Additionally, don’t underestimate the power of training the team that’s going to be coding your seed sets. We’ve found that, the more you emphasize the unique considerations that go with coding for Assisted Review, the more it pays off in the end. Education is big, even if we have a team that says they’ve used it before. The more we invest in training our team, the more they bring back the fantastic questions that fuel the process and help them think it through.

Posted by advice@kCura on June 4, 2013.


The latest insights, trends, and spotlights — directly to your inbox.

The Relativity Blog covers the latest in legal tech and compliance, professional development topics, and spotlights on the many bright minds in our space. Subscribe today to learn something new, stay ahead of emerging tech, and up-level your career.

Interested in being one of our authors? Learn more about how to contribute to The Relativity Blog.