Dear SAP Community,
In this article, we share our experience and lessons learned while developing an MVP of an agentic AI application running on SAP Business Technology Platform (SAP BTP).
Our goal was to build an agent that supports users when creating projects in SAP. Some of you may be familiar with transactions such as CJ20N (Project Builder), CJ11 (Create WBS Elements), or CN21 (Create Networks). An SAP project consists of multiple elements, including project definitions, WBS elements, networks, activities, and activity-related materials. Based on this, we built an agent designed to guide users through project creation with the support of LLMs powered through SAP AI Core.
The following architecture illustrates the approach we followed to build the MVP. While developing the Project Builder Agent, we also used the publicly available OData V4 services from SAP Business Accelerator Hub.
Start with the APIs, not with AI
Before starting to develop any ideas regarding LLMs, Agentic Frameworks or code related to the interaction with AI, it was and is (from our point of view) particularly important to first start dealing with the APIs you want to use within your project. Because we think no matter which tech stack and which tools you ise along the development of your use case, you will work with APIs / OData services within your tools. For that you need to exactly know how the APIs are working (e.g. which payloads are expected + accepted or structure of data). This took the longest time while developing the MVP and for us it was also the most critical part.
Since a project in SAP consists of several elements (mentioned above) we created a project via APIs step by step. We started by posting just the project base (shell) and then slowly added WBS Elements (root and child elements) to the project, then Networks and Activities to the given WBS elements. While playing around with the API we got to know the project structure in SAP and how exactly the API works, especially when the Business Object with which you deal consists of multiple sub dataobjects. This “testing” and “playing” with the APIs allowed us to build the payload structure which we used in a later step for our AI-based user interaction.
For us it was very helpful to do it that way. While developing MVPs before, we immediately started developing the AI part. Calling LLMs, Agents and implemented tools. Then we ran into problems since we hadn’t touched the API part and ran into several issues because we had not spent much time thinking about the most important part: the data source and structure of the tool the Agent uses.
That’s why we decided to do it the other way around this time, and it sped up development enormously. Also, we knew all the pitfalls (at certain inputs in the payload, SAP only accepts a certain length of a string, or some other detail related to the hierarchy of project elements). Once we got to know them, we immediately implemented rules that the Agent can follow and can post a successful payload to SAP without running into weird errors (which usually end in long debug sessions most of the time).
Fixed Payload Structure for Fast Prototyping
Also for an MVP it was crucial to start with a fixed payload structure. Meaning for our Project Builder Agent case: a particular number of WBS Elements, Activities and Networks. While doing this, the complexity was reduced drastically and allowed us fast prototyping. With that “fixed” payload we could focus on the AI-generated content (descriptions and content for project definitions, activities, networks and WBS elements) and the user interaction flow.
We also decided to use a more “static” approach for the MVP and have chosen LangGraph as an orchestrator. Here is a high-level overview of the LangGraph nodes:
Faced Problems During Development
Agent Ambiguity and Compound Intents & Loops: When I wanted to post a data set, I coded that node of the agent to react to something like “Yes, post.” But if the user also writes “please post” and adds “but also change the description to X”, it broke. The binary confirm/reject logic had no concept of compound intents. The result: either silent intent loss or robotic fallbacks (“Please confirm with ‘yes, post please’”).
Robotic Responses feel awkward: Users felt they were talking to a form, not an assistant. Especially while developing the Langgraph flow we needed to make sure to route generic questions (which are out of the decision range of a Langgraph based Agent) to route directly to the LLM.
(In the screenshots above some sampels of the problems we have faced. Long debuggung until getting the userflow somehow interactve 🤖😂)
Improvements & Next Steps
While creating Agents / Projects like this, we also came up with a huge improvement list which we want to implement in the future. Here are some key ideas:
Dynamic Payload Building: Let the AI / the Agent decide how many Project Elements the payload should haveProject Based Analytics: Since a project is often connected to other well-known SAP Elements like Cost Center or Materials it would be interesting to Analyze the material consumption of certain projects or the time and cost they need usuallyRebuild to an MCP server: Rebuild the deployed python service as MCP server to make the tools available to Joule, Claude or MS CoPilotMove from Langgraph to a truly agentic setup: Make use of known LLM SDKs to create one Agent per Task (Like creating, validating and posting the Payload) as well as an orchestrator
Aspects to consider
Even though our list of improvements is long, there are a few things which should be considered implementing AI Agent / embedded AI in Enterprise Applications:
Token Consumption: When shifting to a Multi Agent Approach (e.g. away from Langgraph) you need to consider that more Tokens will be consumed. Unless you set limited guardrails for your Agentic System a lot of tokens can be consumed for a very simple and clear intentLatency: Always think about the user experience and the time the LLM answer needs to come back to the application. Keep in mind to make use of Streaming APIs as well to improve user experience -> https://help.sap.com/doc/generative-ai-hub-sdk/CLOUD/en-US/_reference/streaming.html
Last but not least, we would be interested to know how you build Enterprise AI Applications (architecture wise) and how you deal with upcoming problems. Please let us know in the comments below.
Dear SAP Community,In this article, we share our experience and lessons learned while developing an MVP of an agentic AI application running on SAP Business Technology Platform (SAP BTP).Our goal was to build an agent that supports users when creating projects in SAP. Some of you may be familiar with transactions such as CJ20N (Project Builder), CJ11 (Create WBS Elements), or CN21 (Create Networks). An SAP project consists of multiple elements, including project definitions, WBS elements, networks, activities, and activity-related materials. Based on this, we built an agent designed to guide users through project creation with the support of LLMs powered through SAP AI Core.The following architecture illustrates the approach we followed to build the MVP. While developing the Project Builder Agent, we also used the publicly available OData V4 services from SAP Business Accelerator Hub. Start with the APIs, not with AIBefore starting to develop any ideas regarding LLMs, Agentic Frameworks or code related to the interaction with AI, it was and is (from our point of view) particularly important to first start dealing with the APIs you want to use within your project. Because we think no matter which tech stack and which tools you ise along the development of your use case, you will work with APIs / OData services within your tools. For that you need to exactly know how the APIs are working (e.g. which payloads are expected + accepted or structure of data). This took the longest time while developing the MVP and for us it was also the most critical part.Since a project in SAP consists of several elements (mentioned above) we created a project via APIs step by step. We started by posting just the project base (shell) and then slowly added WBS Elements (root and child elements) to the project, then Networks and Activities to the given WBS elements. While playing around with the API we got to know the project structure in SAP and how exactly the API works, especially when the Business Object with which you deal consists of multiple sub dataobjects. This “testing” and “playing” with the APIs allowed us to build the payload structure which we used in a later step for our AI-based user interaction.For us it was very helpful to do it that way. While developing MVPs before, we immediately started developing the AI part. Calling LLMs, Agents and implemented tools. Then we ran into problems since we hadn’t touched the API part and ran into several issues because we had not spent much time thinking about the most important part: the data source and structure of the tool the Agent uses.That’s why we decided to do it the other way around this time, and it sped up development enormously. Also, we knew all the pitfalls (at certain inputs in the payload, SAP only accepts a certain length of a string, or some other detail related to the hierarchy of project elements). Once we got to know them, we immediately implemented rules that the Agent can follow and can post a successful payload to SAP without running into weird errors (which usually end in long debug sessions most of the time).Fixed Payload Structure for Fast PrototypingAlso for an MVP it was crucial to start with a fixed payload structure. Meaning for our Project Builder Agent case: a particular number of WBS Elements, Activities and Networks. While doing this, the complexity was reduced drastically and allowed us fast prototyping. With that “fixed” payload we could focus on the AI-generated content (descriptions and content for project definitions, activities, networks and WBS elements) and the user interaction flow.We also decided to use a more “static” approach for the MVP and have chosen LangGraph as an orchestrator. Here is a high-level overview of the LangGraph nodes: Faced Problems During DevelopmentAgent Ambiguity and Compound Intents & Loops: When I wanted to post a data set, I coded that node of the agent to react to something like “Yes, post.” But if the user also writes “please post” and adds “but also change the description to X”, it broke. The binary confirm/reject logic had no concept of compound intents. The result: either silent intent loss or robotic fallbacks (“Please confirm with ‘yes, post please’”). Robotic Responses feel awkward: Users felt they were talking to a form, not an assistant. Especially while developing the Langgraph flow we needed to make sure to route generic questions (which are out of the decision range of a Langgraph based Agent) to route directly to the LLM. (In the screenshots above some sampels of the problems we have faced. Long debuggung until getting the userflow somehow interactve 🤖😂)Improvements & Next StepsWhile creating Agents / Projects like this, we also came up with a huge improvement list which we want to implement in the future. Here are some key ideas:Dynamic Payload Building: Let the AI / the Agent decide how many Project Elements the payload should haveProject Based Analytics: Since a project is often connected to other well-known SAP Elements like Cost Center or Materials it would be interesting to Analyze the material consumption of certain projects or the time and cost they need usuallyRebuild to an MCP server: Rebuild the deployed python service as MCP server to make the tools available to Joule, Claude or MS CoPilotMove from Langgraph to a truly agentic setup: Make use of known LLM SDKs to create one Agent per Task (Like creating, validating and posting the Payload) as well as an orchestratorAspects to considerEven though our list of improvements is long, there are a few things which should be considered implementing AI Agent / embedded AI in Enterprise Applications:Token Consumption: When shifting to a Multi Agent Approach (e.g. away from Langgraph) you need to consider that more Tokens will be consumed. Unless you set limited guardrails for your Agentic System a lot of tokens can be consumed for a very simple and clear intentLatency: Always think about the user experience and the time the LLM answer needs to come back to the application. Keep in mind to make use of Streaming APIs as well to improve user experience -> https://help.sap.com/doc/generative-ai-hub-sdk/CLOUD/en-US/_reference/streaming.htmlLast but not least, we would be interested to know how you build Enterprise AI Applications (architecture wise) and how you deal with upcoming problems. Please let us know in the comments below. Read More Technology Blog Posts by Members articles
#SAP
#SAPTechnologyblog
