Imagine if you could converse with your own business data, leveraging the human-like text responses of a Large Language Model (LLM), and have access to context-aware information to make well-informed decisions. We’re excited to introduce a central capability at SAP that grounds LLM responses in business data and provides users with relevant, reliable, and responsible output tailored to their specific needs. By feeding relevant and up-to-date information to an LLM, we reduce model hallucinations – all without the need to finetune the model.
SAP leverages retrieval-augmented generation (RAG) to deliver a central and domain-agnostic service that is easy to use and scales across all business functions. This service also fact-checks LLM-generated statements by tracing information back to its sources, enabling users to act with confidence based on verified data.
In this blog, let’s dive into two capabilities:
Document grounding, a RAG feature for Joule, SAP’s AI copilot The Grounding capability in the orchestration module of Generative AI hub to build custom AI applications
Document grounding: Enhancing the Q&A feature of Joule
The Document grounding capability allows customers to use their own unstructured or semi-structured documents to leverage the Q&A feature of Joule. By connecting their own repositories, such as SharePoint, customers can gain fast and reliable insights from their business documents, saving time spent on finding, opening, and reading documents.
Real-world use-case: SAP SuccessFactors is a perfect example of how Document grounding can revolutionize employee self-service capabilities via Joule. Let’s say an employee needs to find information on their company’s parental leave policy. Instead of sifting through lengthy documents, the employee simply asks Joule: “What is the maximum entitlement for parental leave?” Joule, powered by the Document grounding service, retrieves the relevant information from their company’s HR policy documents (previously indexed) and provides them with a concise, accurate answer.
View of the Q&A feature in Joule, enhanced with Document Grounding
Business Impact: This capability offers an intuitive user experience, improves self-service efficiency, and reduces the total cost of ownership (TCO) with 24×7 HR support. It’s not just an AI feature; it’s a whole new way of empowering teams with knowledge at their fingertips. It can lead to potential efficiency improvements of 70% reduction in employee time needed to investigate HR policies & 81% reduction in time spent by HR on ticket handling, showcasing the tangible impact of this capability.
But the benefits extend beyond just HR. Imagine a sales executive quickly accessing the latest product specifications, or a finance leader getting real-time insights into company financial policies. By making information more accessible and actionable, Document grounding has the potential to transform decision-making processes across the entire organization.
What’s under the hood?
SAP’s Document grounding service leverages RAG to add vital context to LLM responses through a multi-step indexing & retrieval process:
Indexing Pipeline: This component ingests data from diverse sources, processes it, and splits it into manageable chunks. It then generates embeddings of this data and stores them in the SAP HANA Vector Database. Vector Database: The SAP HANA Vector Engine queries these vector embeddings. Retrieval: For every user query, vector embeddings are generated using the LLM model and a search in the HANA Vector Database finds relevant text chunks.
Find more technical and commercial information about Document Grounding in our dedicated Help Portal page.
A Grounding capability via generative AI hub for your customized AI applications
While Document grounding is meant to cover Joule scenarios, SAP is also addressing the needs of developer persona who would want to benefit from the RAG capability within their customized AI applications.
By the end of 2024, the document grounding capability will be fully integrated in the orchestration module of the generative AI hub, alongside capabilities like prompt templating, data masking, and content filter. The mechanism exposed sooner in this article will be applicable via the consumption of the Orchestration service or direct API consumption (indexing and retrieval APIs).
Architecture of the Grounding capability inside Generative AI hub
What’s coming next?
Among many other features, our roadmap through 2025 includesmulti-language support, contextualization, extended list of supported document types and formats as well as documents repositories.
Also, to enhance the quality of indexing and retrieval, SAP is working on backstage technical layers like Automatic RAG (choosing the best chunking strategy automatically), advanced retrieval techniques, Knowledge Graph integration which comes preloaded with a comprehensive representation of SAP’s business entities (including ABAP tables, CDS views, APIs, and key data models), and will play a critical role in grounding AI models and enabling them to deliver more reliable, context-aware insights.
Any questions? Feel free to contact Grounding product management.
Email: sebastien.beghelli@sap.com
Interested in learning more? Refer to the links below
TechEd 2024 session about Document Grounding and the Grounding capability inside GenAI Hub[SAP BTP Onboarding Series]Joule – Getting Started with Document Grounding for Your SAP SFSF.
Document grounding Discovery center page
Imagine if you could converse with your own business data, leveraging the human-like text responses of a Large Language Model (LLM), and have access to context-aware information to make well-informed decisions. We’re excited to introduce a central capability at SAP that grounds LLM responses in business data and provides users with relevant, reliable, and responsible output tailored to their specific needs. By feeding relevant and up-to-date information to an LLM, we reduce model hallucinations – all without the need to finetune the model. SAP leverages retrieval-augmented generation (RAG) to deliver a central and domain-agnostic service that is easy to use and scales across all business functions. This service also fact-checks LLM-generated statements by tracing information back to its sources, enabling users to act with confidence based on verified data. In this blog, let’s dive into two capabilities: Document grounding, a RAG feature for Joule, SAP’s AI copilot The Grounding capability in the orchestration module of Generative AI hub to build custom AI applications Document grounding: Enhancing the Q&A feature of Joule The Document grounding capability allows customers to use their own unstructured or semi-structured documents to leverage the Q&A feature of Joule. By connecting their own repositories, such as SharePoint, customers can gain fast and reliable insights from their business documents, saving time spent on finding, opening, and reading documents. Real-world use-case: SAP SuccessFactors is a perfect example of how Document grounding can revolutionize employee self-service capabilities via Joule. Let’s say an employee needs to find information on their company’s parental leave policy. Instead of sifting through lengthy documents, the employee simply asks Joule: “What is the maximum entitlement for parental leave?” Joule, powered by the Document grounding service, retrieves the relevant information from their company’s HR policy documents (previously indexed) and provides them with a concise, accurate answer.View of the Q&A feature in Joule, enhanced with Document Grounding Business Impact: This capability offers an intuitive user experience, improves self-service efficiency, and reduces the total cost of ownership (TCO) with 24×7 HR support. It’s not just an AI feature; it’s a whole new way of empowering teams with knowledge at their fingertips. It can lead to potential efficiency improvements of 70% reduction in employee time needed to investigate HR policies & 81% reduction in time spent by HR on ticket handling, showcasing the tangible impact of this capability. But the benefits extend beyond just HR. Imagine a sales executive quickly accessing the latest product specifications, or a finance leader getting real-time insights into company financial policies. By making information more accessible and actionable, Document grounding has the potential to transform decision-making processes across the entire organization. What’s under the hood? SAP’s Document grounding service leverages RAG to add vital context to LLM responses through a multi-step indexing & retrieval process: Indexing Pipeline: This component ingests data from diverse sources, processes it, and splits it into manageable chunks. It then generates embeddings of this data and stores them in the SAP HANA Vector Database. Vector Database: The SAP HANA Vector Engine queries these vector embeddings. Retrieval: For every user query, vector embeddings are generated using the LLM model and a search in the HANA Vector Database finds relevant text chunks. Find more technical and commercial information about Document Grounding in our dedicated Help Portal page. A Grounding capability via generative AI hub for your customized AI applications While Document grounding is meant to cover Joule scenarios, SAP is also addressing the needs of developer persona who would want to benefit from the RAG capability within their customized AI applications. By the end of 2024, the document grounding capability will be fully integrated in the orchestration module of the generative AI hub, alongside capabilities like prompt templating, data masking, and content filter. The mechanism exposed sooner in this article will be applicable via the consumption of the Orchestration service or direct API consumption (indexing and retrieval APIs). Architecture of the Grounding capability inside Generative AI hubWhat’s coming next? Among many other features, our roadmap through 2025 includesmulti-language support, contextualization, extended list of supported document types and formats as well as documents repositories. Also, to enhance the quality of indexing and retrieval, SAP is working on backstage technical layers like Automatic RAG (choosing the best chunking strategy automatically), advanced retrieval techniques, Knowledge Graph integration which comes preloaded with a comprehensive representation of SAP’s business entities (including ABAP tables, CDS views, APIs, and key data models), and will play a critical role in grounding AI models and enabling them to deliver more reliable, context-aware insights. Any questions? Feel free to contact Grounding product management. Email: sebastien.beghelli@sap.com Interested in learning more? Refer to the links below TechEd 2024 session about Document Grounding and the Grounding capability inside GenAI Hub[SAP BTP Onboarding Series]Joule – Getting Started with Document Grounding for Your SAP SFSF. Document grounding Discovery center page Read More Technology Blogs by SAP articles
#SAP
#SAPTechnologyblog
