This blog is part of the series 8 Baby Steps to Learn Joule. In the previous blog #1 – Introduction to Joule, we learnt what Joule is and how customers can use it.
In this post, we will learn how Joule works.
Let’s begin!
Step 1 – User Enters the Prompt
Users start with their queries in the natively integrated Joule client embedded in SAP cloud solutions, such as S/4HANA, SuccessFactors etc.
Step 2 – The request is processed and orchestrated by Joule
1. First, Joule analyses if the prompt is relevant for Joule’s Scenario Catalog.
What is Joule’s Scenario Catalog?
Joule’s Scenario Catalog is a comprehensive repository that contains metadata of all available scenarios, functions, and skills across SAP cloud applications. When a user interacts with Joule, their input is first analysed against this catalog to determine if it aligns with any predefined scenarios or functionalities. This process ensures that Joule can provide accurate and contextually relevant responses based on the capabilities defined within the SAP ecosystem.
2. Further, Joule conducts an informational filtering based on the Knowledge Catalog. This process is based on RAG and lets the LLM generate an answer which is contextually relevant and precise.
Knowledge Catalog is a repository that contains SAP’s proprietary knowledge and customer owned knowledge via Document Grounding.
3. Joule is aware of the user’s context and chat history.
This includes information about which SAP cloud application the respective user is using and which additional SAP application the customer has licensed and activated with Joule.
Also, Joule is aware of the user’s role(s) and permissions. This means a user cannot access information or adjust business processes they couldn’t do directly in the SAP cloud application without using Joule.
Step 3 – Joule sends an enhanced query to LLM
Joule takes all of this information and provides a much richer query to the LLM.
These LLMs are also available directly via SAP’s Generative AI Hub, which is part of the SAP BTP.
Which LLM is used by Joule?
Joule follows SAP‘s general multi-partner strategy for generative AI. Joule may employ a variety of foundational models to deliver the best performance for each use case. This ensures that customers get benefitted from the most relevant LLM.
Joule leverages only those LLMs which are available for SAP customer and partners within Generative AI Hub in AI Foundation in SAP BTP.
SAP has under strict contractual agreements with LLM providers which prohibits them from training any of their AI-models on any SAP’s customer data.
Step 4 – LLM’s response back to Joule
Next, the LLM provides a grounded response back to the Joule service.
This is:
Either the response based on its training data and knowledge which could be sent directly to the useror there is a need to leverage Joule functions which includes the Joule scenarios of SAP Cloud Applications.
In simple words, if the user’s query requires action within an SAP system or information from SAP system, the LLM identifies the need to call Joule functions, which connect to SAP Cloud Applications (e.g., S/4HANA, SuccessFactors).
Step 5 – Joule Calls the SAP Backend System
If required, Joule leverages Joule functions to call an SAP backend system.
1. Joule sends the request to the relevant SAP backend system.
For this, the respective information is retrieved via Odata calls OR the LLM for dialog management calls another LLM which was used by an SAP Cloud Application to develop a specific scenario (e.g. interview question generation).
2. Joule gets the required information, processes it.
3. The response from SAP is also filtered and validated to ensure it meets enterprise security, data privacy, and responsible AI guidelines. This step ensures that only secure, accurate, and appropriate responses are sent back to the user.
Step 6 – Final Response to User
The user receives a response, knowing that the conversation and output has been processed all with enterprise security, data privacy, and responsible AI considered.
This is in nutshell – How Joule works!
𝐃𝐫𝐨𝐩 𝐚 “𝐘𝐞𝐬” 𝐢𝐧 𝐭𝐡𝐞 𝐜𝐨𝐦𝐦𝐞𝐧𝐭𝐬 𝐢𝐟 𝐲𝐨𝐮 𝐮𝐧𝐝𝐞𝐫𝐬𝐭𝐚𝐧𝐝 𝐜𝐥𝐞𝐚𝐫𝐥𝐲 
Stay tuned for Blog #3 – Joule Architecture. Meanwhile, you may read the blog Joule, SAP’s Generative AI Copilot
This blog is part of the series 8 Baby Steps to Learn Joule. In the previous blog #1 – Introduction to Joule, we learnt what Joule is and how customers can use it. In this post, we will learn how Joule works.Let’s begin! Step 1 – User Enters the PromptUsers start with their queries in the natively integrated Joule client embedded in SAP cloud solutions, such as S/4HANA, SuccessFactors etc. Step 2 – The request is processed and orchestrated by Joule1. First, Joule analyses if the prompt is relevant for Joule’s Scenario Catalog.What is Joule’s Scenario Catalog?Joule’s Scenario Catalog is a comprehensive repository that contains metadata of all available scenarios, functions, and skills across SAP cloud applications. When a user interacts with Joule, their input is first analysed against this catalog to determine if it aligns with any predefined scenarios or functionalities. This process ensures that Joule can provide accurate and contextually relevant responses based on the capabilities defined within the SAP ecosystem.2. Further, Joule conducts an informational filtering based on the Knowledge Catalog. This process is based on RAG and lets the LLM generate an answer which is contextually relevant and precise. Knowledge Catalog is a repository that contains SAP’s proprietary knowledge and customer owned knowledge via Document Grounding.3. Joule is aware of the user’s context and chat history.This includes information about which SAP cloud application the respective user is using and which additional SAP application the customer has licensed and activated with Joule.Also, Joule is aware of the user’s role(s) and permissions. This means a user cannot access information or adjust business processes they couldn’t do directly in the SAP cloud application without using Joule. Step 3 – Joule sends an enhanced query to LLMJoule takes all of this information and provides a much richer query to the LLM.These LLMs are also available directly via SAP’s Generative AI Hub, which is part of the SAP BTP. Which LLM is used by Joule?Joule follows SAP‘s general multi-partner strategy for generative AI. Joule may employ a variety of foundational models to deliver the best performance for each use case. This ensures that customers get benefitted from the most relevant LLM.Joule leverages only those LLMs which are available for SAP customer and partners within Generative AI Hub in AI Foundation in SAP BTP.SAP has under strict contractual agreements with LLM providers which prohibits them from training any of their AI-models on any SAP’s customer data. Step 4 – LLM’s response back to JouleNext, the LLM provides a grounded response back to the Joule service.This is:Either the response based on its training data and knowledge which could be sent directly to the useror there is a need to leverage Joule functions which includes the Joule scenarios of SAP Cloud Applications.In simple words, if the user’s query requires action within an SAP system or information from SAP system, the LLM identifies the need to call Joule functions, which connect to SAP Cloud Applications (e.g., S/4HANA, SuccessFactors). Step 5 – Joule Calls the SAP Backend SystemIf required, Joule leverages Joule functions to call an SAP backend system.1. Joule sends the request to the relevant SAP backend system.For this, the respective information is retrieved via Odata calls OR the LLM for dialog management calls another LLM which was used by an SAP Cloud Application to develop a specific scenario (e.g. interview question generation).2. Joule gets the required information, processes it.3. The response from SAP is also filtered and validated to ensure it meets enterprise security, data privacy, and responsible AI guidelines. This step ensures that only secure, accurate, and appropriate responses are sent back to the user. Step 6 – Final Response to UserThe user receives a response, knowing that the conversation and output has been processed all with enterprise security, data privacy, and responsible AI considered. This is in nutshell – How Joule works! 𝐃𝐫𝐨𝐩 𝐚 “𝐘𝐞𝐬” 𝐢𝐧 𝐭𝐡𝐞 𝐜𝐨𝐦𝐦𝐞𝐧𝐭𝐬 𝐢𝐟 𝐲𝐨𝐮 𝐮𝐧𝐝𝐞𝐫𝐬𝐭𝐚𝐧𝐝 𝐜𝐥𝐞𝐚𝐫𝐥𝐲 Stay tuned for Blog #3 – Joule Architecture. Meanwhile, you may read the blog Joule, SAP’s Generative AI Copilot Read More Technology Blogs by SAP articles
#SAP
#SAPTechnologyblog
