If you chain the General Splitter in CPI iFlow, what will happen behind the scene

Hello CPI folks,

Recently I faced a support case that my customer is using a chained General Splitter (3 to be specific) design in their iFlow integration. This caused an unexpected result from the test.

I did some test with similar scenario to understand the behavior behind the scene. It appears that this was not documented anywhere before. Hence I would like to share the result from my test to explain how the General Splitter works in a chained situation.

The SAP standard document about General Splitter is here https://help.sap.com/docs/integration-suite/sap-integration-suite/define-general-splitter

The General Splitter splits a composite message comprising N messages into N individual messages, each containing 1 message with the enveloping elements of the composite message. The term enveloping elements refers to these elements including the split point. Note elements that follow the one that is indicated as split point in the original message (but on the same level), aren’t counted as enveloping elements. They won’t be part of the resulting messages.

As mentioned above, in most cases we will use General Splitter as a ‘ForEach’ iterator to loop from a large XML payload to extract each element and process it individually because your API may only able to deal with one element at a time. But if your API supports to process elements in batch, you can set the ‘grouping’ in your General Splitter so it can help to significantly reduce the overhead with many API calls to the backend.

Let’s take one example payload here that I used to test this scenario:

<?xml version=”1.0″ encoding=”UTF-8″?>
<root>
<parent>
<child>
<item id=”1″>1</item>
</child>
<child>
<item id=”2″>2</item>
</child>
<child>
<item id=”3″>3</item>
</child>
</parent>
<parent>
<child>
<item id=”4″>4</item>
</child>
<child>
<item id=”5″>5</item>
</child>
<child>
<item id=”6″>6</item>
</child>
</parent>
<parent>
<child>
<item id=”7″>7</item>
</child>
<child>
<item id=”8″>8</item>
</child>
<child>
<item id=”9″>9</item>
</child>
</parent>
</root>

I want to use two Splitters to firstly split at parent to do something, and then split the child under each parent to process it. It will look like below JavaScript code logic if you don’t set any grouping.

Parents.forEach( (Parent) => {
//<do something with parent>
Parent.Childen.forEach( (Child) => {
//<do something with child>
})
})

Just assuming that my API can process parents in batch. Then I could set the grouping to “2” so that two parents will be processed by API within single call. The logic will look like following. (NOTE: There’s no forEachTwo method available in JS)

Parents.forEachTwo( (TwoParents) => {
//<do something with two parents>
TwoParents.getChilden().forEach( (Child) => {
//<do something with child>
})
})

Here is the iFlow design for this scenario:

As usual, I use Timer upon deployment to trigger this flow on demand. And I use Content Modifier to set the payload into message Body. After each step I use a groovy script to log the message payload.

The setting for first parent Splitter is like following (Grouping = 2)

The setting for second child Splitter (Grouping = <blank>)

Ideally we shall see

1) 2 splits from parents (2 parents in first split + 1 parent in second split)
2) 6 child splits from first group parent (2*3), 3 child splits from the last parent (1*3).

So the groovy logger (after 2nd splitter) shall be called 9 times (we have 9 children in total) as we expected. But that is not the case.

From the message MPL log, we can see only 5 attachments with “Header (after 2nd split)” which means the groovy script logger for child has been called only 5 times in total. (Even though the number of parent logger with “Header (after 1st split)” attachment is correct: 2 times)

Why ? Because children also have been further splitted with grouping 2. The diagram is as following:

The general splitter works in a way like this:

1) If you precisely set the grouping to “X”, it will create a new header called “SapGroup” and fill the value “X” to it. (If this header already exists, it will be modified to “X”)
2) If you keep the grouping setting to blank, the Splitter will firstly detect if there’s any existing “SapGroup” value in the message header. If yes, it will take this value “X”. If this header does not exist, it will set the default value “1” so forEach logic applies. (But bear in mind this will not create “SapGroup” header)

I can clearly see the value of this header in my logger attachment.

As you can see, even I keep the child Splitter with blank in the grouping setting. It will still take the SapGroup = 2 value from message header which was created by the parent Splitter.

The resolution to address this issue is very simple, either you precisely set the grouping to 1. (Again, this will create SapGroup = 1 header so subsequent splitter with blank grouping will be affected). Or if you want to resume the default grouping behavior for all the subsequent splitter(s), you can use Content Modifier or Groovy script to remove this “SapGroup” header before calling another Splitter in the chain.

After all, you still need to do comprehensive test to make sure that Splitter chain in your iFlow design will work as what you expect.

Hope this article can help you to better understand the behavior of this powerful buildblock in iFlow design.

Cheers
Lionel

Hello CPI folks,Recently I faced a support case that my customer is using a chained General Splitter (3 to be specific) design in their iFlow integration. This caused an unexpected result from the test.I did some test with similar scenario to understand the behavior behind the scene. It appears that this was not documented anywhere before. Hence I would like to share the result from my test to explain how the General Splitter works in a chained situation.The SAP standard document about General Splitter is here https://help.sap.com/docs/integration-suite/sap-integration-suite/define-general-splitterThe General Splitter splits a composite message comprising N messages into N individual messages, each containing 1 message with the enveloping elements of the composite message. The term enveloping elements refers to these elements including the split point. Note elements that follow the one that is indicated as split point in the original message (but on the same level), aren’t counted as enveloping elements. They won’t be part of the resulting messages.As mentioned above, in most cases we will use General Splitter as a ‘ForEach’ iterator to loop from a large XML payload to extract each element and process it individually because your API may only able to deal with one element at a time. But if your API supports to process elements in batch, you can set the ‘grouping’ in your General Splitter so it can help to significantly reduce the overhead with many API calls to the backend.Let’s take one example payload here that I used to test this scenario:  <?xml version=”1.0″ encoding=”UTF-8″?>
<root>
<parent>
<child>
<item id=”1″>1</item>
</child>
<child>
<item id=”2″>2</item>
</child>
<child>
<item id=”3″>3</item>
</child>
</parent>
<parent>
<child>
<item id=”4″>4</item>
</child>
<child>
<item id=”5″>5</item>
</child>
<child>
<item id=”6″>6</item>
</child>
</parent>
<parent>
<child>
<item id=”7″>7</item>
</child>
<child>
<item id=”8″>8</item>
</child>
<child>
<item id=”9″>9</item>
</child>
</parent>
</root>  I want to use two Splitters to firstly split at parent to do something, and then split the child under each parent to process it. It will look like below JavaScript code logic if you don’t set any grouping.  Parents.forEach( (Parent) => {
//<do something with parent>
Parent.Childen.forEach( (Child) => {
//<do something with child>
})
})  Just assuming that my API can process parents in batch. Then I could set the grouping to “2” so that two parents will be processed by API within single call. The logic will look like following. (NOTE: There’s no forEachTwo method available in JS)  Parents.forEachTwo( (TwoParents) => {
//<do something with two parents>
TwoParents.getChilden().forEach( (Child) => {
//<do something with child>
})
})   Here is the iFlow design for this scenario:As usual, I use Timer upon deployment to trigger this flow on demand. And I use Content Modifier to set the payload into message Body. After each step I use a groovy script to log the message payload.The setting for first parent Splitter is like following (Grouping = 2)The setting for second child Splitter (Grouping = <blank>)Ideally we shall see1) 2 splits from parents (2 parents in first split + 1 parent in second split)2) 6 child splits from first group parent (2*3), 3 child splits from the last parent (1*3).So the groovy logger (after 2nd splitter) shall be called 9 times (we have 9 children in total) as we expected. But that is not the case.From the message MPL log, we can see only 5 attachments with “Header (after 2nd split)” which means the groovy script logger for child has been called only 5 times in total. (Even though the number of parent logger with “Header (after 1st split)” attachment is correct: 2 times)Why ? Because children also have been further splitted with grouping 2. The diagram is as following:The general splitter works in a way like this:1) If you precisely set the grouping to “X”, it will create a new header called “SapGroup” and fill the value “X” to it. (If this header already exists, it will be modified to “X”)2) If you keep the grouping setting to blank, the Splitter will firstly detect if there’s any existing “SapGroup” value in the message header. If yes, it will take this value “X”. If this header does not exist, it will set the default value “1” so forEach logic applies. (But bear in mind this will not create “SapGroup” header)I can clearly see the value of this header in my logger attachment.As you can see, even I keep the child Splitter with blank in the grouping setting. It will still take the SapGroup = 2 value from message header which was created by the parent Splitter.The resolution to address this issue is very simple, either you precisely set the grouping to 1. (Again, this will create SapGroup = 1 header so subsequent splitter with blank grouping will be affected). Or if you want to resume the default grouping behavior for all the subsequent splitter(s), you can use Content Modifier or Groovy script to remove this “SapGroup” header before calling another Splitter in the chain.After all, you still need to do comprehensive test to make sure that Splitter chain in your iFlow design will work as what you expect.Hope this article can help you to better understand the behavior of this powerful buildblock in iFlow design.CheersLionel   Read More Technology Blogs by SAP articles