With all the discussion about the promise and pitfalls of GenAI, there have been precious few specifics of how and where to use GenAI in the EDRM cycle. This is understandable mostly because there has yet to be a mainstream implementation of GenAI into eDiscovery software platforms. However, in 2024 this will change.

In December, Law.com published a commentary by Sidley Austin entitled Replacing Attorney Review? Sidley's Experimental Assessment of GPT-4’s Performance in Document Review.” Using an old, closed case, Sidley collaborated with Relativity “to evaluate how standard GPT-4 would perform in coding documents for responsiveness.” I would suggest you read the commentary in full, but it is fair to say the authors conclude that while GenAI – GPT-4 in this case – shows promise, it “is still considerably behind [technology assisted review]” citing issues with both accuracy when confidence was not high as well as speed.

As a point of comparison, our Blog post in November, Generative AI: Top Five Questions posits that GenAI will “hav[e] the biggest impact on processing, culling, and analysis” and, ultimately, “tak[e] over the functions of [early case assessment].” We do note the large, downstream effects on review, but see human review as a validation control or quality check.

For litigants and legal professionals, this brings up an interesting question: where is the highest and best use of GenAI tools in the review process?

Placement Within the Workflow

As Maslow said, “when all you have is a hammer, every problem becomes a nail.” So, when asking the question, it is important to acknowledge that review platform vendors will see GenAI’s highest and best use as part of the review process while processing vendors will see it as part of the processing and ECA portion of the EDRM.

As a full spectrum software provider, Venio sees this not as “either or,” but “how much and when.”

Where GenAI is placed in the eDiscovery workflow has significant implications on efficiency and accuracy. Further, GenAI can be tuned to different roles, so that multiple GenAI agents can work at different points in the process with highly specialized tasks.

In our view, a GenAI agent be embedded early in the process during ECA and document culling. Based on the actual document production request, GenAI can quickly scan documents and emails to eliminate irrelevant items, extract key information, and identify privileged or confidential content. This culling process speeds up further analysis, reduces downstream review volume, and improves confidence levels.

Further, a GenAI agent should fully handle first pass review. In the Sidley study, GPT-4 hit the benchmark of 85% when highly confident in the responsiveness or non-responsiveness of the document and, if recent history is a guide, we see that improving into the 95% range. By handling the initial document triage steps, the GenAI agent improves confidence levels and enables human reviewers to focus their efforts on more complex analysis and quality control checks.

Integration Challenges

While promising major productivity gains and with easy to use tools, you might be tempted to just try and integrate a GenAI tool onto your existing eDiscovery tech stack. However, simply bolting on GenAI into legacy eDiscovery installations presents some technological and process challenges including:

• Normalizing and preparing unstructured ESI for machine learning model consumption

• Extracting and organizing text and metadata from numerous native file types

• Mapping identified entities, relationships and facts onto existing database schemas

• Configuring supervised learning cycles to continuously improve accuracy

• Adding security controls to protect client data and confidences

For these reasons, it benefits organizations to leverage GenAI's capabilities through a fully integrated, end-to-end eDiscovery platform rather than attempting to attach machine learning modules onto legacy software.

The Right Technology Partnerships

From both workflow design and technical perspectives, how and where GenAI integration happens requires thoughtful deliberation. But identifying the right software platform is equally important. Key elements to consider include:

• Easy and secure access to organizations’ data stores for thorough document analysis – larger data sets improve the model.

• Collaborative review with advanced analytics, machine learning, and concept search to accelerate document analysis – do not forget that other AI tools have not lost their value.

• Intelligent process automation and templatization to remove manual tasks and enforce governance policies – AI is best approached in a layered fashion with less expensive tools doing the basic work.

• Flexibility in GenAI tool adoption, whether ChatGPT, Claude, Bard, Gemini, or any other tool – it is far from clear which tool will become the standard; in fact, assuming a standard at this point in Ai maturity would be mistake.

• Seamless integration to enable use across all technical skill sets – you should not need a PhD in AI prompting.

As with any new enterprise technology, you will be hearing many conflicting claims about what the technology can or cannot do. It is important to recognize the “cognitive bias” that is a natural part of the vendors’ sales approach and demo different platforms to understand what is right for your needs.

Many maturing technologies require appropriate time for market maturation. Remember HD DVD versus Blu-Ray or VHS versus Betamax? Stay resilient and open to the growth of these technologies. For example, if Chat GPT 3.5 had a roughly equivalent IQ of 83 (FastCompany) and three-months later Chat GPT 4 had an equivalent IQ of 155 (Scientific American), then where will this technology be in one year? Continuing this exponential intellect growth would make a tool like this roughly 10X the IQ of Einstein. Riding the improve intelligence of the emerging tools will reveal where you can get the most leverage and value from GenAI.

The Game Changer for eDiscovery

Of the many AI applications emerging across the legal sector, few match the cost and efficiency implications of GenAI on eDiscovery. This advanced natural language model has the potential to automate many of the most expensive and time-intensive document review tasks. But as outlined above, realizing that full value depends greatly on where and how GenAI gets integrated into the existing people, process and supporting systems. Take a thoughtful approach, seek experienced advisors, and GenAI may prove to be the game changer legal teams have been waiting for.

About Venio Systems

Venio Systems provides leading edge eDiscovery software tailored to meet the needs of corporate legal departments, law firms, and legal service providers. Founded in 2008, Venio combines deep expertise in eDiscovery and information governance with a passion for continuous technology innovation. Venio’s flagship product, VenioOne, delivers powerful eDiscovery capabilities including data identification, legal hold management, collection, processing, early case assessment, analysis, review, and production. VenioOne is available in on-prem or cloud configurations, allowing it to scale matters of any size for any client. For more information, please visit us at www.veniosystems.com