In a previous article “Taking a low code approach to developing your own Microsoft Copilot” I wrote about how Copilot uses Large Language Model (LLM) post-training to add domain-specific content or knowledge. In the following article, I will cover how Retrieval Augmented Generation (RAG) can be used to 
Large language models (LLMs) have revolutionised the way we interact with machines. Their ability to understand and generate human-like text has opened doors for applications in writing assistance, code generation, and even creative storytelling. However, LLMs are limited by the data they are trained on. This can lead to factual inaccuracies, biases reflecting the training data, and an inability to handle information not included in that data.
Here's where Retrieval Augmented Generation (RAG) steps in. RAG is an innovative technique that empowers LLMs by granting them access to external knowledge sources during the generation process. Imagine an LLM as a talented author, but one confined to the content of a community or small library for reference purposes. RAG acts as a librarian in a national or academic library, providing the author with relevant and up-to-date information on demand, significantly enhancing the quality and accuracy of their output.
This article delves into the workings of RAG and how Microsoft Copilot Studio leverages it to unlock the full potential of LLMs, particularly in accessing post-training information and private data.
The Power of Retrieval:
A core component of RAG is the retriever. This intelligent system acts as a bridge between the LLM and the external knowledge sources. It takes the user's query or prompt and scours the designated databases or documents to identify the most relevant information. This retrieved information is then carefully crafted into a concise and informative prompt that guides the LLM in its generation process.
- You send a natural language prompt to Copilot. Here you are essentially giving Copilot some form of instruction in natural language.
- Copilot accesses the various configured data sources e.g. Microsoft Dataverse for an uploaded document or the Bing search API for a public website for pre-processing. Copilot is gathering and preparing information from the various data sources before using it to complete the instruction you gave it.
- Copilot may modify your natural language prompt before sending it to the Large Language Model (LLM). This helps to ensure clarity and technical accuracy and focuses the LLM on the most relevant aspects of your instruction.
- Copilot receives the LLM response, which could be code, text generation, or other information relevant to your instruction.
- Copilot accesses the various configured data sources for post-processing and this step ensures accuracy and provides the most relevant information based on the instruction.
- Copilot then presents you with the outcome, which could be generated code or informative text.
Benefits of Retrieval Augmented Generation:
- Enhanced Factual Accuracy: LLMs are susceptible to biases and factual errors present in their training data. RAG mitigates this by providing access to trustworthy and up-to-date information, ensuring the generated text is factually sound.
- Domain Expertise: By incorporating domain-specific information retrieved from relevant documents or public websites, RAG allows LLMs to become experts in specific fields. This can be particularly valuable in areas like healthcare, finance, or public sector industry verticals.
- Access to Private Data: RAG empowers LLMs to leverage private data sources that were not included in their initial training data. This is crucial for organisations that have sensitive information that needs to be considered during tasks like document generation or code completion.
Microsoft Copilot Studio: A Powerful RAG Implementation
Microsoft Copilot Studio stands out as one of the leading platforms that utilises RAG technology. Here's how Copilot Studio integrates RAG to empower its LLM capabilities:
- Customisable Knowledge Sources: Copilot Studio allows users to define the external knowledge sources that the RAG system will access. This could include public websites, SharePoint sources, or even user-uploaded files. This level of customisation ensures the LLM is drawing upon the most relevant information for the specific task at hand.
- Generative Answers Feature: This feature leverages RAG to provide comprehensive answers directly within Copilot Studio. Users can pose questions about topics related to their designated knowledge sources, and the LLM, guided by the retrieved information, will deliver informative and accurate responses.
- Security and Privacy: When dealing with private data, security is paramount. Copilot Studio ensures that access to sensitive information is strictly controlled and adheres to the highest security standards.
The Future of Retrieval Augmented Generation:
RAG represents a significant leap forward in LLM technology. By enabling access to real-world information and private data, RAG paves the way for LLMs to become even more versatile, accurate, and secure. As research in RAG progresses, we can expect even more innovative applications to emerge, pushing the boundaries of human-machine collaboration and transforming the way we interact with Generative AI.
Conclusion:
In conclusion, Retrieval Augmented Generation presents an exciting future for large language models. By providing access to external knowledge sources, RAG empowers LLMs to become more accurate, domain-specific, and capable of handling private data. Microsoft Copilot Studio is an example of integrating the power of RAG, offering a platform that unlocks the full potential of LLMs for various tasks, and may offer potential benefits for reduced computational power, by better enabling targeted information retrieval. As RAG technology continues to evolve, we can expect to see even more groundbreaking applications emerge in the months and years to come.
For more information on the many ways we can help you, https://www.hpe.com/uk/en/services/pointnext.html
Patrick Lownds
Hewlett Packard Enterprise
