Today, hila Monitoring is now a standard component for Conversational Finance and a core feature of the hila Platform. With hila Monitoring, administrators can set up key policies that monitor the cost, quality and speed of the LLM models and their outputs.
For example, should the number of users in a system remain the same, but the cost increase, the reason could be an increased number of tokens sent to the model. Put differently, users could be asking far longer or more complex questions.
There are other items that impact cost, of course, such as the type of LLM used (GPT3.5 costs approximately 10X less than GPT4, for example), number of queries, the number of users and the model’s efficiency (the length of the answer compared with the length of the question).
In addition to these various aspects of an LLM, hila Monitoring tracks the time that it takes for an LLM to respond, the relevancy of an answer to the original question, and the quality of the SQL generated.
Should there appear a major change in any of the metrics tracked, hila Monitoring provides compelling visualizations that provide context around the shift.
Administrators also receive alerts if costs exceed preset boundaries, the quality drops below a set threshold, or the latency rise above average. This builds on the value of our hila Conversational Finance application, which works on financial data and demonstrates a significant value for enterprises to rapidly retrieve visualizations and insights from their data.
Should someone want to get started quickly, Conversational Finance is set up for business users to begin to derive value from generative AI.
This newest feature adds to an overall suite of components in the hila Platform that enable generative AI to be useful for an enterprise. There are substantial technical aspects to hila Monitoring as well for data scientists, including root-cause analysis, segmentation, policy management, outlier analysis, and explainability. All of these contribute to the mission of Vianai to build responsible AI for enterprises.
The other components in the hila Platform include txt2sql, unstructured querying, anti-hallucination, custom and fine-tuned LLMs, RAG, vectorizing documents, agentic village components for additional extractions, and more.