The hype v. the reality of generative AI in the enterprise

Companies are investing in generative AI. A recent poll from Gartner showed that 55 percent of organizations are in piloting or production mode. This same poll pointed out that 78 percent of respondents believe that the benefits of generative AI outweigh the risks - 10 percent greater than a poll earlier this year.  

It's clear that since ChatGPT went viral, the world has changed to this new normal. As the Gartner data shows, even enterprises have woken to the potential in generative AI. We've seen several conversations to this extent as well, recently announcing a partnership with KPMG¹ and working with many large Fortune 500 companies.  

Yet it's also clear that many enterprises have begun to make investments into generative AI with a plan to "get started" with a large foundational model, without having a full understanding of the investments they must make to have the AI be functional.  

There are scenarios where different models play a part in a comprehensive generative AI solution. These models can be bucketed into the following types:  

‍

• Large public models - such as OpenAI, Anthropic, Google or others, can offer immense performance, and often have a reduced cost. But they are limited, as they need to be pointed at the right data for many high-value use cases, and because of their nature, the enterprise's data must leave its landscape. They also have to have many services around the model itself to provide true business value, including pointing at the right data, chaining together the right prompts, and providing factual information. A large public model by itself is not usually the best solution for an enterprise.  
• Open-source models - Should a company have a dedicated data science team, they can often provide the resources to use an open source model. There are many available, such as Llama-2, Mistral 7B, the Falcon family or the Palm family, and each can be used for a specific task. The performance of these models, especially for specific use cases when fine-tuned either as a private model on a company's proprietary data or on public sources to improve an outcome, is often comparable to any large language model for specific use cases. These offer additional security - with an open-source model fitting behind a firewall, but they also carry additional upfront cost to run; GPU costs can be in the thousands per month, depending on the number of concurrent users.  
• Private models - This is an open-source model that is fine-tuned on a company's proprietary data.  This process can use LoRA techniques to improve efficiency, which also enables the model to be swapped with future updates in open source. This enables an enterprise to have the best possible generative AI models, specifically trained for its use cases on its data. The fine-tuning process, though, often requires fairly robust GPUs, and it, like the open-source use case, has GPU resource requirements for inferences as well.  

‍

There are pros and cons for every type of model. Enterprises must weigh the costs, performance, business requirements, privacy and security of the data with what the best model type (public, open source or private) is for them. These requirements that exist in the business world, but often not in the consumer world, act as guideposts for any engagement. For example, LLaMa 2 70B performs well on unstructured data question and answering, as does the much smaller model Mistral 7B. Claude 2, with its very large context windows, can work especially well for cross-document analysis. And models as small as RoBERTa can be used for highly specialized tasks, such as selecting tables and column names in structured data Q&A use cases.  

Several models can be used across many use cases, such as building charts from natural language questions from a system of record, improve loan terms, or can help teams to cross-compare contracts with their policies. This is just the tip of the iceberg, and there are several others that we've heard about from our customers or are emerging in our conversations. For many companies, leveraging consulting and services firms to help identify the best models to fit their business requirements, together with our Conversational AI capabilities and underlying Zero Hallucination(TM) techniques, can accelerate their ability to get started with generative AI.  

We're excited to see companies investing so heavily in AI, and helping them make AI a reality with hila Enterprise.  

To learn more about how to get off to a fast, reliable start with generative AI in your enterprise, get in touch here. 

‍