What OpenAI’s adoption of retrieval augmented generation (RAG) means for financial services

OpenAI announced some major changes this week, including the new OpenAI Retrieval tool, which enables ChatGPT to incorporate search engine results (from Bing) — and enhance its output using external data, such as proprietary or domain-specific documents provided by the user.

Industry observers have noted that these new capabilities mean OpenAI has adopted retrieval augmented generation (RAG), an AI technique first developed by Meta that combines the power of retrieval-based models that can access real-time information and industry-specific datasets with generative models that are able to produce natural language responses. 

OpenAI’s move to RAG makes a lot of sense. 

The Large Language Models (LLMs) used by generative AI tools like ChatGPT can’t access domain-specific knowledge or up-to-date information, nor they can’t provide sources for the information they generate. And as we all now know, they tend to fabricate responses — a lot!

These flaws have made generative AI tools unsuitable for a variety of enterprise use cases.

Auquan built its Intelligence Engine on RAG because our financial services customers have knowledge-intensive use cases, such as ESG intelligence, KYC, and company pre-screening. All of these require processing large volumes of unstructured data pulled from domain-specific sources, such as company-produced statements, biodiversity and deforestation data, regulatory and legal documents, supplier information, and local media coverage.

And Auquan’s customers have very high expectations when it comes to the accuracy, credibility, and trustworthiness of the insights and analysis they rely upon, and having RAG under Auquan’s hood makes it possible for us to meet and exceed them.

Based on our experience building our product using RAG, we expect the quality of ChatGPT to improve considerably. But those exploring generative AI for enterprise use cases should recognize that it’s still a general purpose AI tool. And a very useful one at that!

There are some big differences between using a general purpose RAG-based generative AI tool like ChatGPT for enterprise use cases and a domain-specific RAG-based solution like Auquan including: 

• The source of domain-specific data. With a general purpose RAG-based generative AI tool, the default is to use a wide open data source, like from a search engine like Bing or it’s on the user to curate the niche datasets and documents for the tool to use in producing responses. This means the responses may include context from generic, irrelevant and sometimes non-credible sources. With a purpose-built domain-specific RAG-based generative AI solution like Auquan, the system automatically curates the underlying data for you to be relevant and credible — with the option to bring your own proprietary data as well. 
  
  
• Context specific retrieval. Different users will have different needs and contexts that dictate the kind of output they need from the same underlying data. A marketing user might need insights into product updates and customer reviews, while an equity research analyst might need to understand future revenue indicators, and and a compliance analyst might need real-time intelligence on legal developments and regulatory fines. The system needs to understand how different data points are more or less relevant to each user. OpenAI’s Retrieval model can potentially optimize which retrieval technique to use, as long as you augment their system with knowledge from outside of their models — but retrieval can only be optimized for document type, not use case. 
  
  
• LLM fine tuning. LLMs require fine-tuning to understand domain specific language and produce responses in the appropriate style and terminology of the user. For instance, reporting language is different for a private equity analyst pre-screening a private company for potential investment than it is for an ESG analyst at an asset management firm producing a report on sustainability performance for a regulatory compliance report.

We knew we made the right call by building on RAG, and the success our customers have achieved with Auquan, combined with our ability to innovate fast, is all of the evidence we need! But it’s nice to see companies like OpenAI make the same decision, and it’s further validation that this is where the industry is moving.