Overview:
This guide provides comprehensive instructions for setting up and configuring document grounding capabilities in Joule Studio. Document grounding enables your Joule agent to search through documents, retrieve relevant information, and provide contextually accurate responses based on your organization’s specific content.
By following this guide, you will:
Set up the required SAP BTP infrastructure (Object Store and SAP AI Core)Create and configure a document grounding resource groupBuild a Joule agent with document grounding capabilitiesApply best practices for prompting agents that use document grounding
Prerequisite:
Before you begin, ensure you have the following:
Requirement
Description
URL’s
SAP BTP Global Account
Active global account with administrator access
SAP BTP Subaccount
Dedicated subaccount for Joule Studio and AI services
AI Core Entitlement
‘aicore` service entitlement assigned to your subaccount
Object Store Service
S3-compatible object store service access
SAP Build Process Automation
Joule Studio Capability and Grounding services
Technical Knowledge:
Basic understanding of SAP BTP destinationsFamiliarity with REST API conceptsUnderstanding of OAuth2 authenticationBasic knowledge of document embedding and vector search concepts
Architecture Overview:
Components:
Joule Agent: Your AI agent that interacts with users and utilizes document groundingDocument Grounding Tool: Enables semantic search across embedded document chunksSAP AI Core: Provides embedding models and vector database capabilities for document groundingObject Store (S3): Stores your documents and their metadataSAP BTP Build Process Automation: is a comprehensive low-code/no-code platform enables Joule studioSAP BTP: The underlying platform hosting all services
Setup Document Grounding tool in Joule Studio:
Step 1: Configure SAP BTP Destinations
1.1 Configure AI Core Destination:
a. In your subaccount, navigate to Connectivity → Destinations
b. Click New Destination
c. Configure with the following settings:
Field
Value
Notes
Name
ai-core
Use a descriptive name
Type
HTTP
–
URL
`<serviceurls>`
From service key
Proxy Type
Internet
–
Authentication
‘OAuth2ClientCredentials’–
Client ID
‘<clientid>’
From service key
Client Secret
‘<clientsecret>’From service key
Token Service URL
`<url>/oauth/token`
From service key Token Service URL Type‘Dedicated’–
Add the following additional property to the destination in order to appear it in Joule Studio:
`sap-joule-studio-document-grounding-ai-core` `true`
!!! tip “Verify Destination”
After creating the destination, use the Check Connection button to verify it’s configured correctly.
1.2 Create Object Store (S3) Destination:
a. In your subaccount, navigate to Connectivity → Destinations
b. Click New Destination
c. Configure with the following settings:
Field
Value
Notes
Name
‘object-store-s3’
Use a descriptive name
Type
HTTP
–
URL
‘https://<host>’
Your S3 host URL
Proxy Type
Internet
–
Authentication
‘OAuth2ClientCredentials’–
Client ID
‘<access_key_id>’
Your S3 access key
Client Secret
‘<secret_access_key>’Your S3 secret
Token Service URL
‘https://<host>’
Same as URL
Token Service URL Type‘Dedicated’–
Add the following additional property to the destination in order to appear it in Joule Studio:
Key
Value
Description
sap-joule-studio-document-grounding-object-store
true
Identifies destination for document grounding
region
‘<your-region>’
S3 region (e.g., `us-east-1`)
bucket
`<your-bucket>`
S3 bucket name
username
`<username>`
Username if required
type‘S3’
Specifies S3 as the object store type
!!! warning “Destination Name Consistency”
Ensure the destination names are identical, if SAP Build and Joule are in separate subaccount.
Step 2: Create Document Grounding Resource Group in Joule Studio
Now that the SAP AI Core infrastructure is ready, configure document grounding in Joule Studio.
2.1 Access Document Grounding in Control Tower
a. Open SAP Build Lobby
b. Navigate to Control Tower
c. Select Document Grounding
2.2 Add a Resource Group
a. Click Add to create a new resource group
b. In the Add Resource Group dialog:
Step i: Name
– Enter a descriptive name (e.g., `Product Documentation`)
– Click Next
Step ii: Select AI Core Destination
– Destination Type: `HTTP`
– Authentication Type: `OAuth2ClientCredentials`
– Select your AI Core destination: `ai-core`
– Click Next
Step iii: Select Object Store Destination
– Destination Type: `HTTP`
– Authentication Type: `OAuth2ClientCredentials`
– Select your object store destination: `object-store-s3`
– Click Next
c. Click Add to create the resource group
3.3 Monitor Resource Group Status
After creation, the resource group will go through a grounding process:
StatusDescriptionAction RequiredPendingGrounding process is runningWait for completionCompletedReady to use Can be added to agentsErrorSomething went wrongCheck logs and retry
!!! info “Grounding Process”
The grounding process creates the necessary connections and validates that the AI Core resource group and collections are accessible. This may take a few minutes.
3.4 Manage Resource Groups
Once created, you can perform the following actions on resource groups:
Edit: Modify the resource group configurationRefresh: Re-run the grounding process (useful if you’ve added new documents)Delete: Remove the resource group
!!! note “Refresh vs New Documents”
The Refresh action refers to re-running the grounding process in Joule Studio, not refreshing the AI Core resources. To add new documents, add them via AI Core API (Step 2.4), then refresh the resource group in Joule Studio.
Step 3: Create a Joule Agent
Now create a Joule agent that will use document grounding capabilities.
3.1 Create the Agent
a. In your Joule Studio project, click Create → Joule Agent
b. Name: Enter a descriptive name (e.g., `Product Support Agent`)
c. Description: Provide a meaningful description:
“`
This agent assists customers with product-related questions by searching through
product documentation, manuals, and technical guides to provide accurate,
contextual answers.
“`
d. Click Create
3.2 Configure Agent Expertise
The Expertise field defines the agent’s role and focus area. Use role prompting to make the agent more specialized.
Example:
“`
You are a knowledgeable product support specialist with expertise in our enterprise
software products. You provide accurate, helpful answers based on official product
documentation.
“`
Best Practices:
Be short and clear (2-3 sentences)Define a specific role or personaAvoid being too generalFocus on the agent’s domain expertise
3.3 Configure Agent Instructions
The Instructions field defines what tasks the agent should perform. Leave empty for conversational agents, or provide specific instructions for structured, repetitive tasks.
Example for Document Search Agent:
“`
When a user asks a question:
1. Use the document search tool to find relevant information from the documentation
2. Read the retrieved content carefully
3. Provide a clear, accurate answer based on the documentation
4. If information is not found in the documents, clearly state that
5. Include relevant context and examples from the documentation when helpful
“`
!!! warning “Conversational vs Structured”
– Conversational agents: Leave Instructions empty to allow adaptive responses
– Structured workflows: Provide detailed, step-by-step instructions
3.4 Add Additional Context
Use the Additional Context field to provide guidance on specific situations. All additional context should begin with `##`.
Example:
“`markdown
##Tool Usage Guidelines
When using the document search tool, prefer specific search queries over broad ones.
For technical questions, prioritize searching for exact error codes or feature names.
## Response Formatting
When providing installation instructions, always format steps as numbered lists.
When referencing specific product versions, always include the version number.
## Handling Missing Information
If the documentation does not contain the answer, use the Final Answer tool to inform
the user and suggest alternative resources such as contacting support directly.
“`
Prompting Guide: https://pages.github.tools.sap/ICN-AI-Agents/baf-documentation/documentation/general/prompting/
3.5 Configure Model Settings
a. LLM Provider: Choose your LLM provider (e.g., OpenAI, Azure OpenAI)
b. Base Model: Select the model for writing responses (e.g., `gpt-4o`)
c. Advanced Model: Select the model for planning and tool selection (e.g., `gpt-4o`)
d. Backup LLM: (Optional) Enable a backup LLM for resilience
!!! info “Model Postfixes”
– `-minimal`: Nearly no reasoning
– `-low`: Basic reasoning
– `-auto`: Model decides reasoning depth
3.6 Configure Agent Execution Steps
a. Maximum Number of Thinking Steps: Set to `10-15` for document search agents
Higher values allow more complex searches and reasoningLower values are faster but may miss complex queries
b. Pre-processing Steps: Enable (recommended)
Includes direct instructions, task decomposition, and planningHelps break down complex questions
c. Post-processing Steps: Enable (recommended)
Includes refinement and answer bundling Provides cleaner, more coherent responses
3.7 Configure Agent Output
a. Default Output Format:
Text: Plain text (simple responses)Markdown: Formatted text (recommended for document grounding)JSON: Structured dataData Type: Structured data with scheme
b. Allow Joule to interpret the output: Enable for conversational agents
Joule may rephrase or summarize responsesDisable for technical accuracy requirements
!!! success “Recommended Settings for Document Grounding”
– Output Format: Markdown (allows formatted responses with lists, headings, code blocks)
– Interpret Output: Enabled (for more natural conversations)
Step 4: Add Document Grounding Tool to Agent
Now integrate the document grounding tool with your Joule agent.
4.1 Add the Documents Tool
a. In your Joule agent configuration, scroll to the Tools section
b. Click Add Tool → Documents
4.2 Configure the Documents Tool
In the Add Tool dialog:
a. Name: Enter a descriptive name for the tool
Example:
“`
Product Documentation Search
“`
b. Description: Provide a clear description of what the tool searches
Example:
“`
Searches through product documentation, user manuals, installation guides, and
technical specifications to find relevant information about our enterprise products.
Use this tool when answering questions about product features, installation,
configuration, troubleshooting, or best practices.
“`
!!! tip “Description Best Practices”
– Be specific about what content is available
– Mention key topic areas covered
– Include when the tool should be used
– Use clear, actionable language
c. Resource Group: Select the resource group you created in Step 3
Only resource groups with status Completed are available. Example: `Product Documentation`
d. Click Add
5.3 Verify Tool Addition
After adding the tool, you should see it listed in the Tools section of your agent configuration:
Tool Name: `Product Documentation Search`Tool Type: `Documents`Resource Group: `Product Documentation`
You can edit or remove the tool at any time by clicking the corresponding icons.
Debugging & Issues:
Use the Trace Feature
Use Joule Studio’s trace feature to debug and optimize:
a. Open the agent chat interface
b. Enable Trace or Debug Mode
c. Ask a question
d. Review:
Which tools were called
What search queries were used
What documents were retrieved
How the agent reasoned through the response
Issue: Agent Doesn’t Use Document Search Tool
Symptoms: Agent provides answers without searching documents
Possible Causes & Solutions:
a. Vague Tool Description: Make tool description more specific and actionable
“`
Bad: “Search documents”
Good: “Searches product documentation including installation guides, user manuals,
and API references. Use this tool when answering product-related questions.”
“`
b. Missing Instructions: Add explicit instructions to use the tool
“`
“When answering questions, use the ‘Product Documentation Search’ tool to find
relevant information before responding.”
“`
c. Too Many Tools: If agent has many tools, prioritize document search in instructions
For more troubleshooting refer this link: https://pages.github.tools.sap/ICN-AI-Agents/baf-documentation/documentation/ui-walkthrough/troubleshoot/
Reference Document: https://help.sap.com/docs/Joule_Studio/45f9d2b8914b4f0ba731570ff9a85313/4444cd1ce4cd471bbe127ea2e4735b40.html?locale=en-US
Overview:This guide provides comprehensive instructions for setting up and configuring document grounding capabilities in Joule Studio. Document grounding enables your Joule agent to search through documents, retrieve relevant information, and provide contextually accurate responses based on your organization’s specific content.By following this guide, you will:Set up the required SAP BTP infrastructure (Object Store and SAP AI Core)Create and configure a document grounding resource groupBuild a Joule agent with document grounding capabilitiesApply best practices for prompting agents that use document groundingPrerequisite:Before you begin, ensure you have the following:RequirementDescriptionURL’sSAP BTP Global AccountActive global account with administrator accessSAP BTP Global AccountSAP BTP SubaccountDedicated subaccount for Joule Studio and AI servicesSAP BTP SubaccountAI Core Entitlement’aicore` service entitlement assigned to your subaccountSAP AI CoreObject Store ServiceS3-compatible object store service accessObject Store on AWSSAP Build Process AutomationJoule Studio Capability and Grounding servicesSAP Build Process Automation Technical Knowledge:Basic understanding of SAP BTP destinationsFamiliarity with REST API conceptsUnderstanding of OAuth2 authenticationBasic knowledge of document embedding and vector search conceptsArchitecture Overview:Components:Joule Agent: Your AI agent that interacts with users and utilizes document groundingDocument Grounding Tool: Enables semantic search across embedded document chunksSAP AI Core: Provides embedding models and vector database capabilities for document groundingObject Store (S3): Stores your documents and their metadataSAP BTP Build Process Automation: is a comprehensive low-code/no-code platform enables Joule studioSAP BTP: The underlying platform hosting all servicesSetup Document Grounding tool in Joule Studio:Step 1: Configure SAP BTP Destinations1.1 Configure AI Core Destination:a. In your subaccount, navigate to Connectivity → Destinationsb. Click New Destinationc. Configure with the following settings: FieldValueNotesNameai-coreUse a descriptive nameTypeHTTP-URL`<serviceurls>` From service key Proxy TypeInternet-Authentication’OAuth2ClientCredentials’-Client ID'<clientid>’From service key Client Secret'<clientsecret>’From service key Token Service URL`<url>/oauth/token`From service key Token Service URL Type’Dedicated’-Add the following additional property to the destination in order to appear it in Joule Studio: `sap-joule-studio-document-grounding-ai-core` `true`!!! tip “Verify Destination” After creating the destination, use the Check Connection button to verify it’s configured correctly.1.2 Create Object Store (S3) Destination:a. In your subaccount, navigate to Connectivity → Destinationsb. Click New Destinationc. Configure with the following settings: FieldValueNotesName’object-store-s3’Use a descriptive nameTypeHTTP-URL’https://<host>’Your S3 host URL Proxy TypeInternet-Authentication’OAuth2ClientCredentials’-Client ID'<access_key_id>’Your S3 access keyClient Secret'<secret_access_key>’Your S3 secret Token Service URL’https://<host>’Same as URL Token Service URL Type’Dedicated’-Add the following additional property to the destination in order to appear it in Joule Studio:KeyValueDescriptionsap-joule-studio-document-grounding-object-storetrueIdentifies destination for document grounding region'<your-region>’S3 region (e.g., `us-east-1`)bucket`<your-bucket>` S3 bucket nameusername`<username>` Username if required type’S3’Specifies S3 as the object store type!!! warning “Destination Name Consistency” Ensure the destination names are identical, if SAP Build and Joule are in separate subaccount.Step 2: Create Document Grounding Resource Group in Joule StudioNow that the SAP AI Core infrastructure is ready, configure document grounding in Joule Studio.2.1 Access Document Grounding in Control Towera. Open SAP Build Lobbyb. Navigate to Control Towerc. Select Document Grounding2.2 Add a Resource Groupa. Click Add to create a new resource groupb. In the Add Resource Group dialog: Step i: Name- Enter a descriptive name (e.g., `Product Documentation`)- Click Next Step ii: Select AI Core Destination- Destination Type: `HTTP`- Authentication Type: `OAuth2ClientCredentials`- Select your AI Core destination: `ai-core`- Click Next Step iii: Select Object Store Destination- Destination Type: `HTTP`- Authentication Type: `OAuth2ClientCredentials`- Select your object store destination: `object-store-s3`- Click Nextc. Click Add to create the resource group3.3 Monitor Resource Group StatusAfter creation, the resource group will go through a grounding process: StatusDescriptionAction RequiredPendingGrounding process is runningWait for completionCompletedReady to use Can be added to agentsErrorSomething went wrongCheck logs and retry!!! info “Grounding Process”The grounding process creates the necessary connections and validates that the AI Core resource group and collections are accessible. This may take a few minutes.3.4 Manage Resource GroupsOnce created, you can perform the following actions on resource groups:Edit: Modify the resource group configurationRefresh: Re-run the grounding process (useful if you’ve added new documents)Delete: Remove the resource group!!! note “Refresh vs New Documents” The Refresh action refers to re-running the grounding process in Joule Studio, not refreshing the AI Core resources. To add new documents, add them via AI Core API (Step 2.4), then refresh the resource group in Joule Studio.Step 3: Create a Joule AgentNow create a Joule agent that will use document grounding capabilities.3.1 Create the Agenta. In your Joule Studio project, click Create → Joule Agentb. Name: Enter a descriptive name (e.g., `Product Support Agent`)c. Description: Provide a meaningful description: “`This agent assists customers with product-related questions by searching throughproduct documentation, manuals, and technical guides to provide accurate,contextual answers. “`d. Click Create3.2 Configure Agent ExpertiseThe Expertise field defines the agent’s role and focus area. Use role prompting to make the agent more specialized.Example:“`You are a knowledgeable product support specialist with expertise in our enterprisesoftware products. You provide accurate, helpful answers based on official productdocumentation.“`Best Practices:Be short and clear (2-3 sentences)Define a specific role or personaAvoid being too generalFocus on the agent’s domain expertise3.3 Configure Agent InstructionsThe Instructions field defines what tasks the agent should perform. Leave empty for conversational agents, or provide specific instructions for structured, repetitive tasks.Example for Document Search Agent:“`When a user asks a question:1. Use the document search tool to find relevant information from the documentation2. Read the retrieved content carefully3. Provide a clear, accurate answer based on the documentation4. If information is not found in the documents, clearly state that5. Include relevant context and examples from the documentation when helpful“`!!! warning “Conversational vs Structured”- Conversational agents: Leave Instructions empty to allow adaptive responses- Structured workflows: Provide detailed, step-by-step instructions3.4 Add Additional ContextUse the Additional Context field to provide guidance on specific situations. All additional context should begin with `##`.Example:“`markdown##Tool Usage GuidelinesWhen using the document search tool, prefer specific search queries over broad ones.For technical questions, prioritize searching for exact error codes or feature names.## Response FormattingWhen providing installation instructions, always format steps as numbered lists.When referencing specific product versions, always include the version number.## Handling Missing InformationIf the documentation does not contain the answer, use the Final Answer tool to informthe user and suggest alternative resources such as contacting support directly.“`Prompting Guide: https://pages.github.tools.sap/ICN-AI-Agents/baf-documentation/documentation/general/prompting/3.5 Configure Model Settingsa. LLM Provider: Choose your LLM provider (e.g., OpenAI, Azure OpenAI)b. Base Model: Select the model for writing responses (e.g., `gpt-4o`)c. Advanced Model: Select the model for planning and tool selection (e.g., `gpt-4o`)d. Backup LLM: (Optional) Enable a backup LLM for resilience!!! info “Model Postfixes”- `-minimal`: Nearly no reasoning- `-low`: Basic reasoning- `-auto`: Model decides reasoning depth3.6 Configure Agent Execution Stepsa. Maximum Number of Thinking Steps: Set to `10-15` for document search agentsHigher values allow more complex searches and reasoningLower values are faster but may miss complex queriesb. Pre-processing Steps: Enable (recommended)Includes direct instructions, task decomposition, and planningHelps break down complex questionsc. Post-processing Steps: Enable (recommended) Includes refinement and answer bundling Provides cleaner, more coherent responses3.7 Configure Agent Outputa. Default Output Format: Text: Plain text (simple responses)Markdown: Formatted text (recommended for document grounding)JSON: Structured dataData Type: Structured data with schemeb. Allow Joule to interpret the output: Enable for conversational agentsJoule may rephrase or summarize responsesDisable for technical accuracy requirements!!! success “Recommended Settings for Document Grounding”- Output Format: Markdown (allows formatted responses with lists, headings, code blocks)- Interpret Output: Enabled (for more natural conversations)Step 4: Add Document Grounding Tool to AgentNow integrate the document grounding tool with your Joule agent.4.1 Add the Documents Toola. In your Joule agent configuration, scroll to the Tools sectionb. Click Add Tool → Documents4.2 Configure the Documents ToolIn the Add Tool dialog:a. Name: Enter a descriptive name for the toolExample:“`Product Documentation Search“`b. Description: Provide a clear description of what the tool searchesExample:“`Searches through product documentation, user manuals, installation guides, andtechnical specifications to find relevant information about our enterprise products.Use this tool when answering questions about product features, installation,configuration, troubleshooting, or best practices.“`!!! tip “Description Best Practices”- Be specific about what content is available- Mention key topic areas covered- Include when the tool should be used- Use clear, actionable languagec. Resource Group: Select the resource group you created in Step 3Only resource groups with status Completed are available. Example: `Product Documentation`d. Click Add5.3 Verify Tool AdditionAfter adding the tool, you should see it listed in the Tools section of your agent configuration:Tool Name: `Product Documentation Search`Tool Type: `Documents`Resource Group: `Product Documentation`You can edit or remove the tool at any time by clicking the corresponding icons.Debugging & Issues:Use the Trace FeatureUse Joule Studio’s trace feature to debug and optimize:a. Open the agent chat interfaceb. Enable Trace or Debug Modec. Ask a questiond. Review: Which tools were called What search queries were used What documents were retrieved How the agent reasoned through the response Issue: Agent Doesn’t Use Document Search ToolSymptoms: Agent provides answers without searching documentsPossible Causes & Solutions:a. Vague Tool Description: Make tool description more specific and actionable“`Bad: “Search documents”Good: “Searches product documentation including installation guides, user manuals,and API references. Use this tool when answering product-related questions.” “`b. Missing Instructions: Add explicit instructions to use the tool“`”When answering questions, use the ‘Product Documentation Search’ tool to findrelevant information before responding.” “`c. Too Many Tools: If agent has many tools, prioritize document search in instructionsFor more troubleshooting refer this link: https://pages.github.tools.sap/ICN-AI-Agents/baf-documentation/documentation/ui-walkthrough/troubleshoot/Reference Document: https://help.sap.com/docs/Joule_Studio/45f9d2b8914b4f0ba731570ff9a85313/4444cd1ce4cd471bbe127ea2e4735b40.html?locale=en-US Read More Technology Blog Posts by SAP articles
#SAP
#SAPTechnologyblog
