As the legal leader at a large corporation, you know the ever-increasing demands and costs of eDiscovery in litigation and regulatory matters. Technology like predictive coding has helped tame some of the burden, but the promise of advanced AI, like generative models, has so far been more hype than reality. However, some of our law firm clients have discovered that generative AI can actually deliver better, faster, cheaper results on real-world litigation tasks.

In a recent real-life case led by an Am Law 100 firm, a generative AI solution was applied to a set of 10 million documents that had undergone predictive coding for a second request matter. The data set included 300,000 potentially privileged documents fully logged for metrics analysis. In drafting privilege log lines, the generative AI achieved a 95% accuracy rate, significantly higher than the 75% rate for human attorney drafted log lines. Additionally, when leveraged for live contract review, the AI delivered 87% accuracy, reducing reviewer burden. This quantifiable performance on real legal work is far more meaningful than hypotheticals on AI’s potential.

As you evaluate outside counsel on your litigation roster, press outside counsel to show metrics on how AI tools can improve accuracy and speed. Here are four key questions you should be asking to determine who can best leverage AI to control costs and risk:

What’s the impact on review speed and cost?

Generative AI isn’t just more accurate, it’s faster and cheaper. In the privilege log example, reviewers leveraging AI descriptions could conduct privilege review 20% faster. That directly translates to lower fees through less review hours. Combined with the reduced need for quality control, there are real savings for clients. Insist that outside counsel demonstrate exactly how any AI they apply will reduce the time, effort and spending required for document review and analysis. If they use AI effectively, you should see those costs come down.

How will you implement and validate AI responsibly?

As promising as AI is, even advanced systems can exhibit unwanted bias or be applied inappropriately. It is essential that your law firms follow best practices like the Generative Artificial Intelligence Transparency, Fairness, Accountability, and Safety Standard with any AI tools. Specifically, ask how they will evaluate tools for issues of bias and fairness and validate performance across different demographic groups. Also, confirm there are appropriate guardrails on accessing sensitive client data. You need responsible AI governance—not reckless experimentation with your information.

What is your AI success rate in prior matters?

Any law firm can talk about plans to eventually adopt AI, but what you need to know is their actual track record of results. Ask specifically how many times they have employed generative AI or other advanced technologies in clients’ matters over the past 12 months. What were the quantified accuracy, cost and time improvements in those real-world uses? A 90% success rate across multiple matters should give you confidence — anything less calls their proficiency into question.

How will you keep our data secure?

You cannot outsource responsibility for protecting sensitive company information. Require outside counsel to explain in detail how client data will remain secure when using cloud-based AI. Elements like encryption in transit and at rest, access controls and monitoring should be baseline practices. Moreover, confirm AI models are trained and refined only on hypothetical data — not on privileged and confidential client documents. Data security with AI requires stringent diligence. 

The bottom line is do not let your outside counsel rely on AI as a vague aspiration. Require evidence-based answers on performance, speed, cost and responsible implementation. Adding benchmarks for past performance, security safeguards and ethical accountability sets a higher bar for truly proven, responsible AI capabilities versus speculative promises. The firms that can meet this high standard should lead your AI-powered litigation strategy. Settling for anything less is leaving savings and better outcomes on the table. The numbers don’t lie.