Each year, the New Zealand legal technology community comes together at Lawfest. Lawyers, e-discovery professionals, and technologists converge in Auckland to learn about the latest innovations, adopt technology solutions to legal challenges, and connect with other professionals.
This year icourts, a RelativityOne Silver Partner, attended Lawfest and showcased the latest innovations in AI, active learning, and RelativityOne.
We sat down with icourts Executive Director Rebecca Grant to learn about Lawfest and gain some insight into the legal technology sector in New Zealand more broadly.
David: You recently exhibited at Lawfest in collaboration with Relativity; how was it?
Rebecca: It was well attended and there was a great buzz in the air. After the pandemic, I think people appreciate the energy that comes from face-to-face connection. COVID restrictions limited the number of international representatives that could attend over the last few years. It was great to have so many interested and interesting people from all over the world sharing ideas—there were plenty of good conversations to be had.
How would you describe the legal technology sector in New Zealand more broadly?
Technology has been used for the last two decades by the New Zealand legal sector, but it has taken off in a big way in the last five years. Starting in 2017, icourts implemented Relativity into three large government regulators, exposing many other legal practitioners to the benefits of applying technology to their practice. It has led to a lot more interest and more sophistication in the legal technology industry. As with most parts of the world, the talent market is tight, but many people are motivated to increase their skill set because they see a strong future in the industry and a growing need for skilled people.
You presented a Tech Talk—“AI in Legal Practice: Benefit or Risk?”—during the conference. What was the response to the topic?
The room was overflowing with people, which is a good indication of the level of interest in the application of AI in the legal sector in New Zealand. As with elsewhere in the world, there has been a data explosion and a rapidly evolving privacy landscape, so people are keen to learn more. The session I presented addressed the inherent risks of applying AI to a document review, such as errors and embedded bias, alongside the range of benefits, such as time and money savings. I also presented several case studies involving one of RelativityOne's artificial intelligence features: active learning. There was a lot of positive feedback from the audience when I addressed the capacity for this type of technology to be applied to small record sets. People were keen to know more and gain new ideas about practice management.
Can you describe a couple of the case studies you presented?
I enjoy presenting active learning case studies because we've been using it since it was released in 2018. It is a proven performer, consistently identifying relevant content more quickly than human reviewers could ever hope to do so. I think it is important for people to understand active learning is not entirely unassisted machine learning; it still needs input from users and monitoring from administrators. A big advantage of active learning is the enormous variety of scenarios in which it can deliver benefits. We have applied active learning to large matters with dozens of reviewers and the smallest of cases with a single reviewer and everything in between.
For example, we deployed active learning in a small matter with a single subject matter expert reviewer. With consistent and accurate input from the reviewer, the model settled and reached the point where the system "understands" what a Relevant document looks like and what a Not Relevant document looks like—in this case after the subject matter expert had reviewed only 10 percent of the document set. After performing an elusion test to QC the model, our client saw no need to review the "discard pile" as they were confident that there was little to no relevant material remaining.
We have also worked with teams who had a requirement to review the entire document set and used active learning to serve up the relevant documents first, so they could start working on the case strategy before completing the review.
When you compare active learning to a traditional review, the benefits are stark. We recently helped a team that had been conducting a standard linear review focusing on a relevant time period. After reviewing 20,000 documents, they found that 70 percent of the documents they looked at were not relevant. Continuing with this traditional review method would be incredibly inefficient. At this point, they had reviewed less than 10 percent of the total document set and it would take them a year to finalize their review. So, we fed their existing coding decisions into an active learning project and the remaining 350,000 documents were quickly categorized and ranked by the system. The legal team could now focus their efforts on reviewing material that was relevant to the matter.
The time and cost savings provided by active learning are amplified when considering a matter with a very large data set. We recently assisted a client with over five terabytes of data and a large team of basic reviewers. As with most large volumes of data, the document set contained large volumes of irrelevant material. The team at icourts worked with the senior lawyers on the matter to perform a series of elusion tests to determine the accuracy of outcome validation. The goal of the elusion test is to provide an estimate of how many low-ranked documents are highly relevant documents that you would not review if you stopped at that point. Our client was able to refer to the elusion rate or statistical estimate of eluded documents; once it fell below the team's accepted risk threshold, they decided not to review further. This saved months of review time on a matter.
The key takeaway of the Tech Talk was that active learning delivers benefits to any data set (thousands of documents or millions), with any size of review team (small or large), and all levels of experience (subject matter experts or first-pass review paralegals).
What do you think the future holds for legal technology and AI in New Zealand?
Our mission at icourts is to help people get to the most responsive set of information they can as quickly as they can. We spend time researching, testing, advising on, and implementing any technology strategies that support that mission. The level of interest in automation and the application of AI and automated decision-making technology at Lawfest was exciting. It is clear from the attendees that there is a need for increased efficiency, and AI is well-placed to help solve many problems. I think AI and legal technology more broadly are only set to go from strength to strength in New Zealand, growing not just broadly with wider adoption, but also deeply, with accelerating levels of sophistication in the application. We will continue to spread the word and showcase the benefits and different feature sets that can help solve everyday problems.
Recommended for you
