Introduction
SAP has released the Pipeline Concept for Cloud Integration as a robust solution for asynchronous message processing, bringing capabilities closer to what SAP PI/PO offered out of the box. It is an excellent resource — especially for complex ICO scenarios with multiple receivers, where it shines with receiver determination, interface splits, and sophisticated restart capabilities using Partner Directory and generic iFlows.
However, when your scenario has a single or small number of receivers, adopting the Pipeline Concept becomes overengineering. The pipeline concept reduces the number of JMS queues to four generic queues, plus four more for retry and dead letter handling — totalling eight queues. For a simple point-to-point interface, this translates into a proliferation of artifacts, configurations in Partner Directory, XSLT mappings for receiver determination, and generic iFlows that add operational complexity without real business value. SAP Community
For point-to-point and async request-reply where a sender communicates with a single receiver, JMS is the ideal choice when Integration Suite already provides the necessary mediation and orchestration. SAP Community
Choosing the right integration model
Before jumping into any retry implementation, it is worth understanding the broader landscape of options available in SAP Integration Suite for asynchronous scenarios:
Pipeline Concept — the right choice when you have complex ICO scenarios with multiple receivers, interface splits, and recipient list patterns. It brings PI/PO-style pipeline processing to Cloud Integration with centralized governance through Partner Directory. The trade-off is significant operational overhead: XSLT mappings, generic iFlows, Partner Directory configuration, and strict naming conventions that must be defined correctly from day one. Correcting a poorly designed Partner Profile setup after go-live is a painful and risky operation.
Custom JMS per iFlow — the approach described in this blog. Simple, direct, and fully controlled per scenario. Each interface owns its queue, its retry logic, and its DLQ. The trade-off here is a different kind of proliferation: JMS queues are not exclusive to a single iFlow by design, but in practice each iFlow should own its own queue to avoid cross-contamination between message flows. In a large landscape this can lead to a high number of queues. SAP Integration Suite has limits on the total number of JMS queues per tenant, so this model requires serious governance on queue naming conventions and ownership from the very beginning.
HTTP Adapter Retry — built-in retry directly in the HTTP adapter, limited to 3 attempts with no backoff configuration. Suitable for low-criticality synchronous scenarios where persistence is not required.
Data Store retry — an alternative when JMS quota is a concern. Messages are persisted in the tenant database and retried via scheduled iFlows. More flexible for manual reprocessing but adds database overhead and is not designed for high-throughput messaging.
When to use this approach
This custom JMS retry pattern is valid and recommended for scenarios with high criticality and a single or small number of receivers, where the overhead of the Pipeline Concept is not justified. However, it comes with clear prerequisites:
A strict naming convention for JMS queues must be defined before the first iFlow goes live. Example: <SENDER>_<RECEIVER>_<INTERFACE>_Q and <SENDER>_<RECEIVER>_<INTERFACE>_DLQ. Changing queue names after messages are in flight is disruptive.Queue ownership must be documented — one iFlow, one queue. Sharing queues between iFlows creates ordering and redelivery issues that are very difficult to debug.Tenant queue limits must be monitored. If your landscape grows, the number of queues can become a bottleneck and a migration to the Pipeline Concept may become necessary.
This blog presents a simpler, lightweight alternative — a custom JMS retry mechanism with one key differentiator: exception-based routing that decides whether a retry is actually worth attempting before wasting resources.
The Problem with Blind Retry
Most JMS retry implementations available in the community retry the message regardless of what caused the failure. This works fine for transient errors like network timeouts or server unavailability — but it wastes resources and delays DLQ routing when the error is non-retryable, such as a missing credential, a SQL syntax error, or an HTTP 401.
Custom retry mechanisms should handle transient errors such as network issues and HTTP 503, while avoiding retries for persistent errors such as mapping issues or authorization failures. SAP Community
For large message payloads in particular, unnecessary retries consume JMS storage, worker node memory, and processing time — all for an error that will never resolve itself without a configuration fix.
The Solution — iFlow Architecture
The solution uses two iFlows and two JMS queues:
IFlow 1 — Receives the message from the sender system and stores it in the JMS queue for asynchronous processing.
IFlow 2 — Reads from the JMS queue, executes the business logic, and in case of failure delegates to a Local Integration Process that evaluates the exception type and decides between retry or DLQ routing.
IFlow 3 — Reads from the DLQ as an Exclusive queue with retries set to 0, attempts one reprocess, and on failure triggers an alert via email or any monitoring tool — ending always with a normal End event to avoid infinite loops.
Iflow 1 representation:
Iflow 2 representation:
Iflow 2 – Local Process representation:
Iflow 3 representation:
SAP CPI Representation:
The exception subprocess in iFlow 2 calls a Local Integration Process that first evaluates the exception type via a Groovy script, then routes to either an Error End event — keeping the message in the queue for JMS-native retry — or directly to the DLQ when the error is classified as non-retryable.
Key points:
The Groovy — Exception Evaluator
The script below covers the most common exception types found in SAP Integration Suite scenarios. Each team should adapt the classification to their own environment — adding specific Oracle error codes, custom HTTP status patterns, or BAPI return types that are relevant to their landscape. The logic is intentionally straightforward: every exception either sets RetryExceptionValid = true and allows the JMS retry cycle to continue, or sets RetryExceptionValid = false combined with SAP_MarkMessageAsFailed = false to route the message directly to the DLQ without further retry attempts.
Covered out of the box:
Default behaviour: any unrecognised exception is treated as retryable, assuming it may be transient. Adjust this default if your landscape has known non-retryable patterns not listed above.
import com.sap.gateway.ip.core.customdev.util.Message
def Message processData(Message message) {
def props = message.getProperties()
def exception = props.get(“CamelExceptionCaught”)?.toString() ?: “”
// ── JDBC ──────────────────────────────────────────
if (exception.contains(“JDBCConnectionException”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“JDBCTransientException”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“JDBCDataSourceException”) &&
!exception.contains(“CredentialStore”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“CredentialStoreCredentialNotFoundException”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
if (exception.contains(“SQLSyntaxErrorException”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
if (exception.contains(“JDBCCreatorException”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
if (exception.contains(“ORA-00942”) || exception.contains(“ORA-00904”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
// ── HTTPS ─────────────────────────────────────────
if (exception.contains(“ConnectException”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“SocketTimeoutException”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“Connection refused”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“HTTP 500”) || exception.contains(“HTTP 502”) ||
exception.contains(“HTTP 503”) || exception.contains(“HTTP 504”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“HTTP 401”) || exception.contains(“HTTP 403”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
if (exception.contains(“HTTP 400”) || exception.contains(“HTTP 404”) ||
exception.contains(“HTTP 422”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
// ── SOAP ──────────────────────────────────────────
if (exception.contains(“SOAPFaultException”)) {
if (exception.contains(“env:Receiver”) || exception.contains(“:Server”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“env:Sender”) || exception.contains(“:Client”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
message.setProperty(“RetryExceptionValid”, “true”); return message
}
// ── OData ─────────────────────────────────────────
if (exception.contains(“ODataException”)) {
if (exception.contains(“500”) || exception.contains(“503”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“401”) || exception.contains(“403”) ||
exception.contains(“400”) || exception.contains(“404”) ||
exception.contains(“409”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
message.setProperty(“RetryExceptionValid”, “true”); return message
}
// ── RFC / BAPI ────────────────────────────────────
if (exception.contains(“RfcCommunicationException”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“RfcLogonException”)) {
if (exception.contains(“timeout”) || exception.contains(“Timeout”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message
}
if (exception.contains(“AbapException”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
if (exception.contains(“BAPI”) && exception.contains(“TYPE=E”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
if (exception.contains(“BAPI”) && exception.contains(“TYPE=W”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
// ── Default: retry ────────────────────────────────
message.setProperty(“RetryExceptionValid”, “true”)
return message
}
DLQ Strategy
The DLQ iFlow should be treated as a notification and manual review mechanism, not as another retry loop. Configure the JMS Sender of the DLQ queue with 0 retries. In the Exception Subprocess, send an alert email with the error details and end with a normal End event — this removes the message from the DLQ as completed, avoiding any infinite loop. Each team should implement the alert strategy that fits their operations model.
References
Introducing the new pipeline concept in Cloud IntegrationConfigure Asynchronous Messaging with Retry Using JMS AdapterRetry failed messages in CPI with automatic and manual reprocessingHandling Connectivity and Recoverable Errors in SAP CPI with JMS QueuesConfigure Dead Letter Handling in JMS AdapterJMS Mastery Series Part 3 — JMS in Production
Thank you.
Kind regards,
Viana.
IntroductionSAP has released the Pipeline Concept for Cloud Integration as a robust solution for asynchronous message processing, bringing capabilities closer to what SAP PI/PO offered out of the box. It is an excellent resource — especially for complex ICO scenarios with multiple receivers, where it shines with receiver determination, interface splits, and sophisticated restart capabilities using Partner Directory and generic iFlows.However, when your scenario has a single or small number of receivers, adopting the Pipeline Concept becomes overengineering. The pipeline concept reduces the number of JMS queues to four generic queues, plus four more for retry and dead letter handling — totalling eight queues. For a simple point-to-point interface, this translates into a proliferation of artifacts, configurations in Partner Directory, XSLT mappings for receiver determination, and generic iFlows that add operational complexity without real business value. SAP CommunityFor point-to-point and async request-reply where a sender communicates with a single receiver, JMS is the ideal choice when Integration Suite already provides the necessary mediation and orchestration. SAP CommunityChoosing the right integration modelBefore jumping into any retry implementation, it is worth understanding the broader landscape of options available in SAP Integration Suite for asynchronous scenarios:Pipeline Concept — the right choice when you have complex ICO scenarios with multiple receivers, interface splits, and recipient list patterns. It brings PI/PO-style pipeline processing to Cloud Integration with centralized governance through Partner Directory. The trade-off is significant operational overhead: XSLT mappings, generic iFlows, Partner Directory configuration, and strict naming conventions that must be defined correctly from day one. Correcting a poorly designed Partner Profile setup after go-live is a painful and risky operation.Custom JMS per iFlow — the approach described in this blog. Simple, direct, and fully controlled per scenario. Each interface owns its queue, its retry logic, and its DLQ. The trade-off here is a different kind of proliferation: JMS queues are not exclusive to a single iFlow by design, but in practice each iFlow should own its own queue to avoid cross-contamination between message flows. In a large landscape this can lead to a high number of queues. SAP Integration Suite has limits on the total number of JMS queues per tenant, so this model requires serious governance on queue naming conventions and ownership from the very beginning.HTTP Adapter Retry — built-in retry directly in the HTTP adapter, limited to 3 attempts with no backoff configuration. Suitable for low-criticality synchronous scenarios where persistence is not required.Data Store retry — an alternative when JMS quota is a concern. Messages are persisted in the tenant database and retried via scheduled iFlows. More flexible for manual reprocessing but adds database overhead and is not designed for high-throughput messaging.When to use this approachThis custom JMS retry pattern is valid and recommended for scenarios with high criticality and a single or small number of receivers, where the overhead of the Pipeline Concept is not justified. However, it comes with clear prerequisites:A strict naming convention for JMS queues must be defined before the first iFlow goes live. Example: <SENDER>_<RECEIVER>_<INTERFACE>_Q and <SENDER>_<RECEIVER>_<INTERFACE>_DLQ. Changing queue names after messages are in flight is disruptive.Queue ownership must be documented — one iFlow, one queue. Sharing queues between iFlows creates ordering and redelivery issues that are very difficult to debug.Tenant queue limits must be monitored. If your landscape grows, the number of queues can become a bottleneck and a migration to the Pipeline Concept may become necessary.This blog presents a simpler, lightweight alternative — a custom JMS retry mechanism with one key differentiator: exception-based routing that decides whether a retry is actually worth attempting before wasting resources.The Problem with Blind RetryMost JMS retry implementations available in the community retry the message regardless of what caused the failure. This works fine for transient errors like network timeouts or server unavailability — but it wastes resources and delays DLQ routing when the error is non-retryable, such as a missing credential, a SQL syntax error, or an HTTP 401.Custom retry mechanisms should handle transient errors such as network issues and HTTP 503, while avoiding retries for persistent errors such as mapping issues or authorization failures. SAP CommunityFor large message payloads in particular, unnecessary retries consume JMS storage, worker node memory, and processing time — all for an error that will never resolve itself without a configuration fix. The Solution — iFlow ArchitectureThe solution uses two iFlows and two JMS queues:IFlow 1 — Receives the message from the sender system and stores it in the JMS queue for asynchronous processing.IFlow 2 — Reads from the JMS queue, executes the business logic, and in case of failure delegates to a Local Integration Process that evaluates the exception type and decides between retry or DLQ routing.IFlow 3 — Reads from the DLQ as an Exclusive queue with retries set to 0, attempts one reprocess, and on failure triggers an alert via email or any monitoring tool — ending always with a normal End event to avoid infinite loops.Iflow 1 representation:Iflow 2 representation: Iflow 2 – Local Process representation:Iflow 3 representation:SAP CPI Representation: The exception subprocess in iFlow 2 calls a Local Integration Process that first evaluates the exception type via a Groovy script, then routes to either an Error End event — keeping the message in the queue for JMS-native retry — or directly to the DLQ when the error is classified as non-retryable. Key points:SAPJMSRetries header must be added to Allowed Header(s) in Runtime Configuration and saved as a property in the first Groovy script at the beginning of the main flow.SAP_MarkMessageAsFailed = false is mandatory for non-retryable errors — without it, the End normal event is ignored and the JMS adapter still retries.The DLQ iFlow should have retries set to 0. Each company should implement their own alerting strategy — email, ServiceNow ticket, monitoring dashboard — since the DLQ message has already been classified as non-recoverable.The Groovy — Exception EvaluatorThe script below covers the most common exception types found in SAP Integration Suite scenarios. Each team should adapt the classification to their own environment — adding specific Oracle error codes, custom HTTP status patterns, or BAPI return types that are relevant to their landscape. The logic is intentionally straightforward: every exception either sets RetryExceptionValid = true and allows the JMS retry cycle to continue, or sets RetryExceptionValid = false combined with SAP_MarkMessageAsFailed = false to route the message directly to the DLQ without further retry attempts.Covered out of the box:JDBC — connection failures, transient locks, pool exhaustion, credential store errors, SQL syntax errors, Oracle ORA codesHTTPS — connection refused, socket timeout, HTTP 4xx and 5xx status codesSOAP — fault classification by env:Receiver (retryable) vs env:Sender (non-retryable)OData — HTTP status-based classification, including 409 conflictRFC / BAPI — communication exceptions, logon timeout vs auth failure, ABAP exceptions, BAPI RETURN TYPE=E and TYPE=WDefault behaviour: any unrecognised exception is treated as retryable, assuming it may be transient. Adjust this default if your landscape has known non-retryable patterns not listed above.import com.sap.gateway.ip.core.customdev.util.Message
def Message processData(Message message) {
def props = message.getProperties()
def exception = props.get(“CamelExceptionCaught”)?.toString() ?: “”
// ── JDBC ──────────────────────────────────────────
if (exception.contains(“JDBCConnectionException”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“JDBCTransientException”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“JDBCDataSourceException”) &&
!exception.contains(“CredentialStore”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“CredentialStoreCredentialNotFoundException”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
if (exception.contains(“SQLSyntaxErrorException”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
if (exception.contains(“JDBCCreatorException”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
if (exception.contains(“ORA-00942”) || exception.contains(“ORA-00904”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
// ── HTTPS ─────────────────────────────────────────
if (exception.contains(“ConnectException”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“SocketTimeoutException”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“Connection refused”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“HTTP 500”) || exception.contains(“HTTP 502”) ||
exception.contains(“HTTP 503”) || exception.contains(“HTTP 504”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“HTTP 401”) || exception.contains(“HTTP 403”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
if (exception.contains(“HTTP 400”) || exception.contains(“HTTP 404”) ||
exception.contains(“HTTP 422”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
// ── SOAP ──────────────────────────────────────────
if (exception.contains(“SOAPFaultException”)) {
if (exception.contains(“env:Receiver”) || exception.contains(“:Server”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“env:Sender”) || exception.contains(“:Client”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
message.setProperty(“RetryExceptionValid”, “true”); return message
}
// ── OData ─────────────────────────────────────────
if (exception.contains(“ODataException”)) {
if (exception.contains(“500”) || exception.contains(“503”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“401”) || exception.contains(“403”) ||
exception.contains(“400”) || exception.contains(“404”) ||
exception.contains(“409”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
message.setProperty(“RetryExceptionValid”, “true”); return message
}
// ── RFC / BAPI ────────────────────────────────────
if (exception.contains(“RfcCommunicationException”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
if (exception.contains(“RfcLogonException”)) {
if (exception.contains(“timeout”) || exception.contains(“Timeout”)) { message.setProperty(“RetryExceptionValid”, “true”); return message }
message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message
}
if (exception.contains(“AbapException”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
if (exception.contains(“BAPI”) && exception.contains(“TYPE=E”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
if (exception.contains(“BAPI”) && exception.contains(“TYPE=W”)) { message.setProperty(“RetryExceptionValid”, “false”); message.setProperty(“SAP_MarkMessageAsFailed”, “false”); return message }
// ── Default: retry ────────────────────────────────
message.setProperty(“RetryExceptionValid”, “true”)
return message
}DLQ StrategyThe DLQ iFlow should be treated as a notification and manual review mechanism, not as another retry loop. Configure the JMS Sender of the DLQ queue with 0 retries. In the Exception Subprocess, send an alert email with the error details and end with a normal End event — this removes the message from the DLQ as completed, avoiding any infinite loop. Each team should implement the alert strategy that fits their operations model.ReferencesIntroducing the new pipeline concept in Cloud IntegrationConfigure Asynchronous Messaging with Retry Using JMS AdapterRetry failed messages in CPI with automatic and manual reprocessingHandling Connectivity and Recoverable Errors in SAP CPI with JMS QueuesConfigure Dead Letter Handling in JMS AdapterJMS Mastery Series Part 3 — JMS in ProductionThank you.Kind regards,Viana. Read More Technology Blog Posts by Members articles
#SAP
#SAPTechnologyblog
