Agentic AI Chatbots in the Enterprise Context

Enterprises today need more than a chatbot that simply provides guidelines or answers FAQs. In real business contexts, users expect AI systems that can actually take action — querying databases for insights, drafting or sending emails, and orchestrating multi-step workflows. This growing demand has led to the rise of agentic AI chatbots, which combine the familiarity of a conversational interface with the power to execute enterprise-scale tasks. While frameworks such as LlamaIndex, AutoGPT, and LangGraph enable orchestration, enterprises need robust deployment, seamless integration, and governance to run them at scale.

This blog series will walk through that journey step by step. We’ll start by building a custom agent backend, then deploy it on SAP AI Core, and finally make it directly consumable from Joule Studio — enabling business users to interact with the agent through a natural conversational interface inside Joule.

🚀 To make the process easier to follow, we’ve divided the content into three parts:

Part 1 — Building the Custom AI Agent Backend  (Flask + API Endpoints)Part 2 — Deploying the Agent on AI Core (Docker, ServingTemplate, Deployment, Endpoint test)Part 3 — Integrating the Agent into Joule Studio (Destination setup, Skill creation, testing)

By the end of this series, you will have a clear, reproducible path from

Custom AI agent code in GitHubScalable deployment on AI CoreBusiness-ready conversational interface through Joule Studio.

Part 1 — Building the Custom AI Agent Backend

In this first post, we focus on the backend logic of the custom agentic chatbot. This backend is implemented with Flask and exposes REST API endpoints that support both one-shot and multi-step interactions. While one-shot calls return only the final answer, the multi-step design allows the system to expose the agent’s reasoning process step by step, improving interpretability and transparency.

Custom AI Agent Chatbot Backend Capabilities 

For demonstration purposes, the backend is designed to handle common enterprise tasks such as:

Structured Data Access – Query SAP HANA Database to support reporting, analytics, and decision-making.Vector-based Retrieval – Search SAP HANA Vector DB to enable advanced RAG (Retrieval-Augmented Generation) use cases.Enterprise Communication – Manage enterprise emails (drafting, sending, or forwarding messages) through integration with existing systems.External Information Retrieval – Fetch context from real-world services (e.g., weather updates, web search results) to enrich responses.

API Endpoint Design for Agent Orchestration

In our implementation, we adopted LangGraph as the orchestration framework. Since the agent dynamically generates a plan first, and then builds a LangGraph structure based on that plan, it was natural to split the API into multiple steps:

/v2/plan → /v2/action-graph → /v2/start-execution → /v2/continue-execution

This is just one way to design it. With a different orchestration style, you might merge steps into a single /chat endpoint or a simpler two step flow. 

Endpoint Summary:

MethodPathSummaryRequest BodyGET/v2/healthHealth check-POST/v2/planStep 1. Generate planconversation_id, messagePOST/v2/action-graphStep 2. Create action graphconversation_idPOST/v2/start-executionStep 3. Start Executionconversation_idPOST/v2/continue-executionStep 4. Complete Execution conversation_idPOST/v2/chatOne-shot chatconversation_id, message

These endpoints reflect how the workflow is divided into steps in our design. To make it more concrete, here’s a minimal Flask example showing how each endpoint might look in practice.

Example Flask Code:

from flask import Flask, request, jsonify
import uuid

app = Flask(__name__)

# Health check
@app.route(‘/v2/health’, methods=[‘GET’])
def health_check():
return jsonify({“status”: “healthy”})

# Step 1: Plan generation
@app.route(‘/v2/plan’, methods=[‘POST’])
def generate_plan():
“””
Example route for generating a plan.
– Replace the stub logic with your own planner (LLM, LangGraph, rule-based, etc.).
– Store the plan into session or external state manager if needed.
“””
data = request.json or {}
message = data.get(‘message’,”)
session_id = data.get(‘conversation_id’) or f”session-{uuid.uuid4().hex[:8]}”

if not message:
return jsonify({‘error’: ‘Message is required’}), 400

# TODO: Call planning logic here

return jsonify({“conversation_id”: session_id,
“type”: “plan_generated”,
“content”: “<example plan object>”})

# Step 2: Action graph creation
@app.route(‘/v2/action-graph’, methods=[‘POST’])
def generate_action_graph():
“””
Example route for generating an action graph.
– Replace with your real plan → graph transformation.
– Retrieve the plan from session or agent manager before creating the graph.
“””
data = request.json or {}
session_id = data.get(‘conversation_id’)
if not session_id:
return jsonify({‘error’:’conversation_id is required’}), 400

# TODO: Get planning state > Implement graph creation logic here

return jsonify({“conversation_id”: session_id,
“type”: “action_graph_generated”,
“content”: “<example content you want to show user>”})

# Step 3: Start execution
@app.route(‘/v2/start-execution’, methods=[‘POST’])
def start_execution():
“””
Example route for starting execution.
– Replace with logic to execute the first node of the action graph.
“””
data = request.json or {}
session_id = data.get(‘conversation_id’)
if not session_id:
return jsonify({‘error’:’conversation_id is required’}), 400

# TODO: Get action graph > execute the first node of the graph

return jsonify({“conversation_id”: session_id,
“type”: “execution_started”,
“content”: “<example content you want to show user>”})

# Step 4: Continue execution
@app.route(‘/v2/continue-execution’, methods=[‘POST’])
def continue_execution():
“””
Example route for continuing execution.
– Replace with logic to execute remaining nodes and return the final response.
“””
data = request.json or {}
session_id = data.get(‘conversation_id’)
if not session_id:
return jsonify({‘error’:’conversation_id is required’}), 400

# TODO: Get action graph > execute remaining nodes and return final response

return jsonify({“conversation_id”: session_id,
“type”: “execution_complete”,
“content”: “<example content you want to show user>”})

# One-shot chat
@app.route(‘/v2/chat’, methods=[‘POST’])
def chat():
“””
Example route for one-shot chat.
– Replace with your pipeline that runs plan → graph → execution in one step.
“””
data = request.json or {}
message = data.get(‘message’, ”)
session_id = data.get(‘conversation_id’) # use conversation_id as session_id

# TODO: Process agent pipeline here all at once

return jsonify({“conversation_id”: session_id,
“type”: “execution_complete”,
“content”: “<example content you want to show user>”})

if __name__ == ‘__main__’:
app.run(host=”0.0.0.0″, port=5000)

⚠️Note: this is only an example. Replace the TODO sections with your own orchestration logic.

With this backend in place, we now have a clear foundation for orchesting multi-step agent workflows, from plan generation to action execution, all exposed via REST APIs.

👉In the next part, we’ll take this custom agent backend and deploy it on SAP AI Core: packaging the Flask service into a Docker container, creating a ServingTemplate, and exposing it as a production-ready endpoint. 

​ Agentic AI Chatbots in the Enterprise ContextEnterprises today need more than a chatbot that simply provides guidelines or answers FAQs. In real business contexts, users expect AI systems that can actually take action — querying databases for insights, drafting or sending emails, and orchestrating multi-step workflows. This growing demand has led to the rise of agentic AI chatbots, which combine the familiarity of a conversational interface with the power to execute enterprise-scale tasks. While frameworks such as LlamaIndex, AutoGPT, and LangGraph enable orchestration, enterprises need robust deployment, seamless integration, and governance to run them at scale.This blog series will walk through that journey step by step. We’ll start by building a custom agent backend, then deploy it on SAP AI Core, and finally make it directly consumable from Joule Studio — enabling business users to interact with the agent through a natural conversational interface inside Joule.🚀 To make the process easier to follow, we’ve divided the content into three parts:Part 1 — Building the Custom AI Agent Backend  (Flask + API Endpoints)Part 2 — Deploying the Agent on AI Core (Docker, ServingTemplate, Deployment, Endpoint test)Part 3 — Integrating the Agent into Joule Studio (Destination setup, Skill creation, testing)By the end of this series, you will have a clear, reproducible path fromCustom AI agent code in GitHubScalable deployment on AI CoreBusiness-ready conversational interface through Joule Studio. Part 1 — Building the Custom AI Agent BackendIn this first post, we focus on the backend logic of the custom agentic chatbot. This backend is implemented with Flask and exposes REST API endpoints that support both one-shot and multi-step interactions. While one-shot calls return only the final answer, the multi-step design allows the system to expose the agent’s reasoning process step by step, improving interpretability and transparency.Custom AI Agent Chatbot Backend Capabilities For demonstration purposes, the backend is designed to handle common enterprise tasks such as:Structured Data Access – Query SAP HANA Database to support reporting, analytics, and decision-making.Vector-based Retrieval – Search SAP HANA Vector DB to enable advanced RAG (Retrieval-Augmented Generation) use cases.Enterprise Communication – Manage enterprise emails (drafting, sending, or forwarding messages) through integration with existing systems.External Information Retrieval – Fetch context from real-world services (e.g., weather updates, web search results) to enrich responses.API Endpoint Design for Agent OrchestrationIn our implementation, we adopted LangGraph as the orchestration framework. Since the agent dynamically generates a plan first, and then builds a LangGraph structure based on that plan, it was natural to split the API into multiple steps:/v2/plan → /v2/action-graph → /v2/start-execution → /v2/continue-executionThis is just one way to design it. With a different orchestration style, you might merge steps into a single /chat endpoint or a simpler two step flow. Endpoint Summary:MethodPathSummaryRequest BodyGET/v2/healthHealth check-POST/v2/planStep 1. Generate planconversation_id, messagePOST/v2/action-graphStep 2. Create action graphconversation_idPOST/v2/start-executionStep 3. Start Executionconversation_idPOST/v2/continue-executionStep 4. Complete Execution conversation_idPOST/v2/chatOne-shot chatconversation_id, messageThese endpoints reflect how the workflow is divided into steps in our design. To make it more concrete, here’s a minimal Flask example showing how each endpoint might look in practice.Example Flask Code:from flask import Flask, request, jsonify
import uuid

app = Flask(__name__)

# Health check
@app.route(‘/v2/health’, methods=[‘GET’])
def health_check():
return jsonify({“status”: “healthy”})

# Step 1: Plan generation
@app.route(‘/v2/plan’, methods=[‘POST’])
def generate_plan():
“””
Example route for generating a plan.
– Replace the stub logic with your own planner (LLM, LangGraph, rule-based, etc.).
– Store the plan into session or external state manager if needed.
“””
data = request.json or {}
message = data.get(‘message’,”)
session_id = data.get(‘conversation_id’) or f”session-{uuid.uuid4().hex[:8]}”

if not message:
return jsonify({‘error’: ‘Message is required’}), 400

# TODO: Call planning logic here

return jsonify({“conversation_id”: session_id,
“type”: “plan_generated”,
“content”: “<example plan object>”})

# Step 2: Action graph creation
@app.route(‘/v2/action-graph’, methods=[‘POST’])
def generate_action_graph():
“””
Example route for generating an action graph.
– Replace with your real plan → graph transformation.
– Retrieve the plan from session or agent manager before creating the graph.
“””
data = request.json or {}
session_id = data.get(‘conversation_id’)
if not session_id:
return jsonify({‘error’:’conversation_id is required’}), 400

# TODO: Get planning state > Implement graph creation logic here

return jsonify({“conversation_id”: session_id,
“type”: “action_graph_generated”,
“content”: “<example content you want to show user>”})

# Step 3: Start execution
@app.route(‘/v2/start-execution’, methods=[‘POST’])
def start_execution():
“””
Example route for starting execution.
– Replace with logic to execute the first node of the action graph.
“””
data = request.json or {}
session_id = data.get(‘conversation_id’)
if not session_id:
return jsonify({‘error’:’conversation_id is required’}), 400

# TODO: Get action graph > execute the first node of the graph

return jsonify({“conversation_id”: session_id,
“type”: “execution_started”,
“content”: “<example content you want to show user>”})

# Step 4: Continue execution
@app.route(‘/v2/continue-execution’, methods=[‘POST’])
def continue_execution():
“””
Example route for continuing execution.
– Replace with logic to execute remaining nodes and return the final response.
“””
data = request.json or {}
session_id = data.get(‘conversation_id’)
if not session_id:
return jsonify({‘error’:’conversation_id is required’}), 400

# TODO: Get action graph > execute remaining nodes and return final response

return jsonify({“conversation_id”: session_id,
“type”: “execution_complete”,
“content”: “<example content you want to show user>”})

# One-shot chat
@app.route(‘/v2/chat’, methods=[‘POST’])
def chat():
“””
Example route for one-shot chat.
– Replace with your pipeline that runs plan → graph → execution in one step.
“””
data = request.json or {}
message = data.get(‘message’, ”)
session_id = data.get(‘conversation_id’) # use conversation_id as session_id

# TODO: Process agent pipeline here all at once

return jsonify({“conversation_id”: session_id,
“type”: “execution_complete”,
“content”: “<example content you want to show user>”})

if __name__ == ‘__main__’:
app.run(host=”0.0.0.0″, port=5000)⚠️Note: this is only an example. Replace the TODO sections with your own orchestration logic.With this backend in place, we now have a clear foundation for orchesting multi-step agent workflows, from plan generation to action execution, all exposed via REST APIs. 👉In the next part, we’ll take this custom agent backend and deploy it on SAP AI Core: packaging the Flask service into a Docker container, creating a ServingTemplate, and exposing it as a production-ready endpoint.    Read More Technology Blog Posts by SAP articles 

#SAP

#SAPTechnologyblog