The problem
Our customer wanted the ability to ask questions against their contracts in real-time. Today they outsource this work – which comes at a financial cost and creates a long process between question and answer. They asked us to put hila on their contracts, to ask questions on the fly.
Specifically, our client wanted:
Current situation: Overcoming the "agreement trap"
Drafting and negotiating agreements is done in a bulky and cumbersome way, similar to how it was done decades ago. Often, drafts are made in Word documents, sent over email for negotiations, and then stored (and forgotten about) in some kind of repository.
Should a lawyer need to draft a new document or make an amendment, they today, rely on institutional knowledge, or an army of junior attorneys to review contracts, extract the key components, evaluate them for risk, and provide a general “first pass” before they jump into the details. This happens, and potentially this information is stored somewhere, though likely when renegotiation time comes, it is forgotten about. At an enterprise, the person who negotiated the last deal could have left and taken all that knowledge with them.
And, it's not to say that the information in the system is even correct. In the case of our client, they had missing or incorrect meta data in their Ariba system. Many of the contracts weren’t even digitized.
When it comes to one or two documents, a lawyer with knowledge of the contract set can easily review them. However, when there are thousands of documents, all with a complex interdependence, the scale becomes extremely challenging and time consuming. All told, DocuSign puts the size of this problem at $2 trillion.
hila solves this by making metadata extraction from all contracts and natural-language querying easy.
The solution and methodology
First, we used a combination of agents to extract the key information. This process used several agents, such as one to review the contracts, another to organize the information and a third to provide legal analysis.
Next, we used a similar, but different process, to review the contracts for risk. Using the rubric provided to us, the same that they gave the junior lawyers who they had review the contracts before, we could use a conjunction of agents to review their contracts, apply a risk label to them and provide the associated clauses that caused the model to apply that risk.
Once that information was broken out, we put it all into a simple database, which we then could apply our text2sql technology to (here is a full blog on how our txt2sql process works reliably), and enable users to ask natural questions. These questions are listed below. This combines with our unstructured querying, which provides answers directly from the contract itself.
Example questions included:
hila, then, provides a comprehensive solution, with a multi-agent approach for metadata extraction and risk evaluation; a processing system to take that extraction and put it into a structured, simplified database; text2sql to pull out the information from a structured database; and anti-hallucination components to query against the contracts themselves. This is all under a single, easy user experience.
Our system ensures that the results are accurate andhallucination free. For more information on our anti-hallucination processes, please see our blog on the topic, here.
In testing, we found that our agentic approach reduced the cost of the process by 4X compared to other models and improved the accuracy oft he responses substantially compared to GPT4.
The results
The scalable solution can now sit on thousands of their contracts without any loss in fidelity. This has enabled our client to use hila Enterprise instead of outsourcing their initial pass, which saves time and money.
In the future, this kind of system can vastly speed up review of tens of thousands of documents, crucial during moments of time timelines, such as during a massive M&A deal.
Contact us to see how we fit into your landscape.