Introduction:
I often come across integrations that have not been designed to exploit the Message bandwidth optimally.
Before I expand any further on this, it would be good to familiarize yourself with the following blog.
https://www.integration-excellence.com/message-traffic-licensing-with-sap-integration-suite/
Below is an extract from the above blog.
What is a message in Cloud Integration?
In Cloud Integration each OUTGOING message from an Integration Flow is countedE.g. if you have a splitter step within your iflow which sends 100 messages, you might see only 1 message in the monitor, but 100 are countedEach message is counted up to 250 KB payload size, so when you transfer a 2 MB message, 9 messages will be charged (2 * 1024 / 250 = 8,192 => rounded to 9)If you send each message into an archive (S3 bucket or whereever) it is also counted additionallyIf you have a splitter step with 100 messages and 50 KB each within the same iflow message transfer (1 message in monitor), they are summed up and then the total is checked against the 250 KB size limit, so 20 messages will be charged (100 * 50 / 250 = 20)Although only OUTGOING messages are counted, the payload from a reply of a request-reply step is also considered and chargedRetries are considered each time! Implement a circuit breaker for your iflows using JMS/Data Store by stopping the further processing after a configured amount of retriesInternal adapters, such as ProcessDirect and JMS are NOT considered as OUTGOING messagesTraffic via Edge Integration Cell is counted with a factor of 0.5 (50%) and there is only one runtime component (API Gateway and Cloud Integration Runtime is one technical runtime, thus a message is only counted once)
For now Let’s talk about three types of Integrations
1) SAP To SAP – Prepackaged / Standard Integrations
This category basically covers all pre packaged / out of the box integrations provided by SAP (configure and deploy). Outgoing Messages / Message Responses through these integrations are not chargeable and do not count towards the overall Bandwidth of your Cloud Integration Tenant.
2) SAP to SAP – Modified / Custom Integrations
If you have built a custom integration / iflow on SAP CI or you have modified a prepackaged integration flow then the messages flowing through these IFlows will contribute towards the bandwidth of your tenant. If you have built an extension of a standard integration using post exits or pre exits then messages flowing through these exits / extensions will also count towards the bandwidth of your tenant.
3) SAP to Non SAP Integrations – Prepackaged / Custom
All outgoing messages for SAP to Non-SAP prepackaged / custom integrations are charged and count towards your tenants message bandwidth.
Now that we have cleared the basics (And someone correct me in comments if I am wrong about these assertions) lets move on to the next section.
Optimal Exploitation of the Bandwidth:
In good old days when systems were on-premise and we owned the hardware and were responsible for the network we didn’t have to think twice about the number of messages flowing through the Integration platform as long as we were able to scale up the hardware and the network bandwidth if our overall outgoing increased.
Now that everything is on cloud and SAP Integration suite runtime is managed by SAP and the pricing is consumption based i.e. you pay for what you consume, we have to be really careful and optimal while designing integrations. For example consider these scenarios
Scenario 1: Custom Integration: A Message hits SAP CI and has 50 records, SAP CI needs to do 4 lookups in SAP FSM and then based on some conditions based on the result of these lookups one record at a time will be updated in FSM through SAP CI with some additional fields probably which again come from these lookups.
This roughly translates to 250 outgoing messages of around 2-3 kb each in size.
What could have been improved? The above scenario could have been restricted to use 200 messages by doing a bulk insert into FSM if the API permitted bulk upload instead of doing 50 individual inserts.
Or the scenario could have been restricted to just 5 messages if all the api calls to FSM could have been batch calls
Before you get all riled up let’s understand what is a Batch call nd what is a Bulk call?
Bulk operations typically involve processing a large number of records or operations in a single API call, often using a dedicated endpoint optimized for such large-scale actions (CREATE/UPDATE/POST). Batch operations, on the other hand, involve grouping multiple operations into a single request, which can be either a single API call or multiple calls within a larger context
https://help.sap.com/docs/SAP_FIELD_SERVICE_MANAGEMENT/fsm_bulk_api/bulk-api-overview.html
https://help.sap.com/docs/SAP_FIELD_SERVICE_MANAGEMENT/fsm_data_api/batch-request.html
Refer to above links to understand more.
Why is optimizing these integrations to use Message bandwidth optimally so important?
Depending on your license messages going out of SAP CI have a cost associated to them. The earlier you realize this and build you integration around this, the happier you would be. WHY?
Imagine discovering an integration is badly designed and is running in production and now needs to be redesigned to lets say reduce the over consumption by up-to 2-3 times. Great! you have a solution and you know how to optimize it but where is the ROI? Re-designing this will take some effort let’s say 6000$’s and business will have to pay for it but it might just incur an additional few hundred dollars every month if it is allowed to run as is. Ultimately it’s a loss for the business. Therefore design your integrations right the first time.
Pro Tip: Make it a practice to save the outgoing message size in kilo bytes in a custom header for each outgoing message, i.e. just before the receiver adapter, insert a script that calculates the message size in kilobytes and adds that to a custom header. this will help you identify the problematic integrations.
This brings us to the question, how to decide how optimal an integration is on a scale of 1 to 10 in terms of bandwidth.
In scenario 1 if the integration consumes 250 messages while it could have been designed to use just 5 messages, It would be a 0.2 on a scale of 1 to 100 , where 1 is the poorest and 100 is the most optimal state.
How did we reach this?
( (number of messages it should optimally consume) / (total number of messages it consumes now) ) * 100
in our case (5 / 250) * 100 = 2. Basically on the lower side.
Conclusion
With SaaS tools like SAP FSM, Concur, SAP SF, its fairly common to do multiple lookups, inserts before sending out the final boss update for a SAP To SF integration for instance. This can result in exponential consumption of messages therefore its imperative that the bandwidth is used optimally.
Introduction:I often come across integrations that have not been designed to exploit the Message bandwidth optimally.Before I expand any further on this, it would be good to familiarize yourself with the following blog.https://www.integration-excellence.com/message-traffic-licensing-with-sap-integration-suite/Below is an extract from the above blog.What is a message in Cloud Integration?In Cloud Integration each OUTGOING message from an Integration Flow is countedE.g. if you have a splitter step within your iflow which sends 100 messages, you might see only 1 message in the monitor, but 100 are countedEach message is counted up to 250 KB payload size, so when you transfer a 2 MB message, 9 messages will be charged (2 * 1024 / 250 = 8,192 => rounded to 9)If you send each message into an archive (S3 bucket or whereever) it is also counted additionallyIf you have a splitter step with 100 messages and 50 KB each within the same iflow message transfer (1 message in monitor), they are summed up and then the total is checked against the 250 KB size limit, so 20 messages will be charged (100 * 50 / 250 = 20)Although only OUTGOING messages are counted, the payload from a reply of a request-reply step is also considered and chargedRetries are considered each time! Implement a circuit breaker for your iflows using JMS/Data Store by stopping the further processing after a configured amount of retriesInternal adapters, such as ProcessDirect and JMS are NOT considered as OUTGOING messagesTraffic via Edge Integration Cell is counted with a factor of 0.5 (50%) and there is only one runtime component (API Gateway and Cloud Integration Runtime is one technical runtime, thus a message is only counted once)For now Let’s talk about three types of Integrations1) SAP To SAP – Prepackaged / Standard IntegrationsThis category basically covers all pre packaged / out of the box integrations provided by SAP (configure and deploy). Outgoing Messages / Message Responses through these integrations are not chargeable and do not count towards the overall Bandwidth of your Cloud Integration Tenant. 2) SAP to SAP – Modified / Custom IntegrationsIf you have built a custom integration / iflow on SAP CI or you have modified a prepackaged integration flow then the messages flowing through these IFlows will contribute towards the bandwidth of your tenant. If you have built an extension of a standard integration using post exits or pre exits then messages flowing through these exits / extensions will also count towards the bandwidth of your tenant. 3) SAP to Non SAP Integrations – Prepackaged / CustomAll outgoing messages for SAP to Non-SAP prepackaged / custom integrations are charged and count towards your tenants message bandwidth.Now that we have cleared the basics (And someone correct me in comments if I am wrong about these assertions) lets move on to the next section.Optimal Exploitation of the Bandwidth:In good old days when systems were on-premise and we owned the hardware and were responsible for the network we didn’t have to think twice about the number of messages flowing through the Integration platform as long as we were able to scale up the hardware and the network bandwidth if our overall outgoing increased.Now that everything is on cloud and SAP Integration suite runtime is managed by SAP and the pricing is consumption based i.e. you pay for what you consume, we have to be really careful and optimal while designing integrations. For example consider these scenariosScenario 1: Custom Integration: A Message hits SAP CI and has 50 records, SAP CI needs to do 4 lookups in SAP FSM and then based on some conditions based on the result of these lookups one record at a time will be updated in FSM through SAP CI with some additional fields probably which again come from these lookups.This roughly translates to 250 outgoing messages of around 2-3 kb each in size.What could have been improved? The above scenario could have been restricted to use 200 messages by doing a bulk insert into FSM if the API permitted bulk upload instead of doing 50 individual inserts.Or the scenario could have been restricted to just 5 messages if all the api calls to FSM could have been batch callsBefore you get all riled up let’s understand what is a Batch call nd what is a Bulk call?Bulk operations typically involve processing a large number of records or operations in a single API call, often using a dedicated endpoint optimized for such large-scale actions (CREATE/UPDATE/POST). Batch operations, on the other hand, involve grouping multiple operations into a single request, which can be either a single API call or multiple calls within a larger contexthttps://help.sap.com/docs/SAP_FIELD_SERVICE_MANAGEMENT/fsm_bulk_api/bulk-api-overview.htmlhttps://help.sap.com/docs/SAP_FIELD_SERVICE_MANAGEMENT/fsm_data_api/batch-request.htmlRefer to above links to understand more.Why is optimizing these integrations to use Message bandwidth optimally so important?Depending on your license messages going out of SAP CI have a cost associated to them. The earlier you realize this and build you integration around this, the happier you would be. WHY?Imagine discovering an integration is badly designed and is running in production and now needs to be redesigned to lets say reduce the over consumption by up-to 2-3 times. Great! you have a solution and you know how to optimize it but where is the ROI? Re-designing this will take some effort let’s say 6000$’s and business will have to pay for it but it might just incur an additional few hundred dollars every month if it is allowed to run as is. Ultimately it’s a loss for the business. Therefore design your integrations right the first time. Pro Tip: Make it a practice to save the outgoing message size in kilo bytes in a custom header for each outgoing message, i.e. just before the receiver adapter, insert a script that calculates the message size in kilobytes and adds that to a custom header. this will help you identify the problematic integrations. This brings us to the question, how to decide how optimal an integration is on a scale of 1 to 10 in terms of bandwidth.In scenario 1 if the integration consumes 250 messages while it could have been designed to use just 5 messages, It would be a 0.2 on a scale of 1 to 100 , where 1 is the poorest and 100 is the most optimal state. How did we reach this?( (number of messages it should optimally consume) / (total number of messages it consumes now) ) * 100in our case (5 / 250) * 100 = 2. Basically on the lower side.ConclusionWith SaaS tools like SAP FSM, Concur, SAP SF, its fairly common to do multiple lookups, inserts before sending out the final boss update for a SAP To SF integration for instance. This can result in exponential consumption of messages therefore its imperative that the bandwidth is used optimally. Read More Technology Blog Posts by Members articles
#SAP
#SAPTechnologyblog
