Based on estimates within the industry, there were 250 million generative AI (GenAI) prompts in January 2024. This number tripled to 750 million by July 2024 and is on pace to hit 1 billion monthly prompts by the end of the year.
Those are only the prompts. A 15- or 20-word prompt can generate 10 or 100 times more content. Simply put, the scale is massive. The legal technology industry hasn’t seen anything like this, and it has major risk implications for eDiscovery, litigation support and regulatory compliance professionals.
Casepoint hosted a webinar about GenAI and enterprise data challenges for more than a dozen industry experts. Oliver Silva, Vice President of Product Operations at Casepoint and David Cowen, President of The Cowen Group, co-led the conversation.
Collecting prompt data: Current or future priority?
Attendees gave a wide variety of responses regarding whether their organizations are collecting prompt data or whether they’re seeing it happen around them.
“Nobody in our realm is collecting or considering prompt eDiscovery content at this point. However, I am beating the drum that this is going to happen. Lots of things are going to start popping out of the world now, and we're going to start having, for instance, an HR system that terminates an employee based upon some analysis by a generative AI module. I can see that happening immediately. Getting people to start to think about that here is fun, but it's also kind of a challenge because it's not where they're living right now.”
- eDiscovery and Litigation Technology Manager at a government law office
Other experts chimed in on the question.
- A managing director for a top-tier law firm said they are “absolutely” seeing companies preserving and collecting prompt data.
- A compliance enablement manager for a multinational oil and gas company said they are “avoiding saving the prompt,” but the stance will evolve moving forward.
- A chief legal and compliance officer for a global fleet services company was called on to give a general counsel's point of view. The officer expressed how the plan is to launch GenAI, but “I haven’t even thought about some of the ideas that are coming out of this discussion, so I’m a little bit nervous.”
The professional wasn’t alone in their nervousness. Why? Due to the size and scope of the problems GenAI prompt data poses.
Describing the new “Chaos” organizations face
The discussion veered back to the sheer scale of data that organizations have to contend with from GenAI.
“The volume of data generated by prompts and responses can get into the gigabytes or terabytes,” Silva said. “It’s a massive data and value problem, and the industry is at the beginning of it all.
“Once upon a time, eDiscovery was chaos,” Cowen added. “Then it became organized chaos. Then it became, ‘OK, we’ve got some ad hoc solutions we’re cobbling together.’ Then we operationalized. Now we're back into a state where it's just complete chaos.”
So, if GenAI prompt data warrants a new era of chaos, so to speak, what does that mean for organizations? What issues do they need to work out?
“We're talking about IT policy considerations, information governance, data security — like how does this data work, and how does it get transferred over to a company or a partner like Casepoint?” Silva said.
“From a legal operations perspective, what does this mean for us keeping this data? Right now, we can try to ignore it, but we need to start planning for it. This will hit all of our plates in one way or another in the very, very near future. The thing is, we've solved problems like this before. This is a little different. We just have to apply our previous learnings about what we did to this new problem.”
How to respond: Advice from a veteran Legal Ops professional
A lot of participants expressed how they were still in the early stages of approaching GenAI prompt data.
An eDiscovery veteran advised leaders to focus on the needs and priorities of the entire organization, not just the legal team. By understanding the broader business context, including roadmaps and risk tolerance, legal operations can effectively support other teams and allocate resources for adopting and integrating AI and solutions that align with organizational goals.
“And I encourage you [GC/Legal Leadership] to continue in these kinds of conversations, to open your eyes to what those challenges may be and to take up the ‘Casepoints’ of the world on their offers to have an educational session, a demo. Don't just do it yourself. Bring others along with you, because the more of you across your team that know what's road-mapped and available, the better position you're going to be in to help build the buy-in because there's less buy-in for convincing them they need it. It's showing them what's on the horizon and what can be done for them.”
Should organizations collect generative AI (GenAI) data? Based on the perspectives gained from this discussion, all signs are pointing to “yes”. As the adoption of GenAI tools and models grow, the risk factors increase for eDiscovery, litigation support and regulatory compliance.
