Introduction:

In this blog post, I will show the common scenarios of Advanced Event Mesh. To get familiar with basic concepts, go through the below links as a prerequisite.

Advanced Event Mesh: Basic Concepts 

Advanced Event Mesh: Create your first event broker

Advanced Event Mesh: Queue 

Advanced event mesh is powered by Solace and throughout the blog post I will use a solace trial account. All the steps remain same if you are using AEM (Advanced Event Mesh).

In this blog post, the following topics will be covered.

Publish Message to Queue using AEM adapterConsume Message from Queue using AEM adapterDMQ Setup and Error HandlingDirect MessageDurable topic endpointUse of SelectorSend NACK to PublisherTime-to-Live Rest Delivery PointReplay LogParallel ConsumersRequest Reply Pattern

Scenario 1: Publish Message to Queue using AEM adapter

Step 1: 

Create a queue in AEM

Step 2:

Create an iFlow

Configure HTTP sender adapter

Configure AEM receiver adapter. Here, the password secure alias is of type Secure Parameter.

The queue name is populated based on the value of Camel header – CamelHttpPath

The message is set as DMQ eligible.

Step 3:

Send event message using API testing tool to the endpoint: https://<host>/http/generic/order

Step 4:

Validate if the message is pushed to the AEM queue

Scenario 2: Consume Message from Queue using AEM adapter

Step 1:

Create an iFlow to consumer message from the AEM queue. Set Allowed headers as * to trace the incoming headers.

Configure AEM sender channel

Step 2:

Check message trace in Cloud Integration and verfiy headers. As the publisher iFlow set the the header DMQ eligible as true, the ‘IsDMQEligible’ header is true.

Step 3:

Verify the consumer status attached to the AEM queue.  Here, it reflects ‘Messages Confimed Delivered’ as 1.

Scenario 3: DMQ Setup and Error Handling

To move a message to DEAD Message queue upon delivery failure, follow the steps below.

Step 1:

Configure AEM queue as below.

Also, the queue is subscribed to the topic expression order/>

Step 2:

Step 3:

POST message to the endpoint https://<host>/http/generic/order. No message will be pushed to the queue, because the topic subscription order/> expects at least one level after order.

Step 4:

​ Introduction:In this blog post, I will show the common scenarios of Advanced Event Mesh. To get familiar with basic concepts, go through the below links as a prerequisite.Advanced Event Mesh: Basic Concepts Advanced Event Mesh: Create your first event brokerAdvanced Event Mesh: Queue Advanced event mesh is powered by Solace and throughout the blog post I will use a solace trial account. All the steps remain same if you are using AEM (Advanced Event Mesh).In this blog post, the following topics will be covered.Publish Message to Queue using AEM adapterConsume Message from Queue using AEM adapterDMQ Setup and Error HandlingDirect MessageDurable topic endpointUse of SelectorSend NACK to PublisherTime-to-Live Rest Delivery PointReplay LogParallel ConsumersRequest Reply PatternScenario 1: Publish Message to Queue using AEM adapterStep 1: Create a queue in AEMStep 2:Create an iFlow Configure HTTP sender adapterConfigure AEM receiver adapter. Here, the password secure alias is of type Secure Parameter.The queue name is populated based on the value of Camel header – CamelHttpPathThe message is set as DMQ eligible.Step 3:Send event message using API testing tool to the endpoint: https://<host>/http/generic/order Step 4:Validate if the message is pushed to the AEM queueScenario 2: Consume Message from Queue using AEM adapterStep 1:Create an iFlow to consumer message from the AEM queue. Set Allowed headers as * to trace the incoming headers.Configure AEM sender channelStep 2:Check message trace in Cloud Integration and verfiy headers. As the publisher iFlow set the the header DMQ eligible as true, the ‘IsDMQEligible’ header is true.Step 3:Verify the consumer status attached to the AEM queue.  Here, it reflects ‘Messages Confimed Delivered’ as 1.Scenario 3: DMQ Setup and Error HandlingTo move a message to DEAD Message queue upon delivery failure, follow the steps below.Step 1:Configure AEM queue as below.Also, the queue is subscribed to the topic expression order/> Step 2: Change receiver channel configuration in order to push event message to a topic. The topic name is dynamically set using the header – CamelHttpPath . The sender channel will consume messages from the queue named ‘order’. The message sent from the publisher iFlow is a JSON message. In order to simulate failure, XML to JSON converter block is added . Notice the acknowledgement mode is ‘Automatic Immediate’ at Cloud Integration (Consumer).  Step 3:POST message to the endpoint https://<host>/http/generic/order. No message will be pushed to the queue, because the topic subscription order/> expects at least one level after order.  Step 4: Push message to the endpoint https://<host>/http/generic/order/create  But, observe the message is successfully deleted from the queue. It is because acknowledgment mode is ‘automatic immediate’.  Now, the retry happens in Cloud Integration 4 times and then the message is marked as failed.  The below shows the trace of Run 1. The header ‘DeliveryCount’ is set from AEM. The header ‘LocalProcessingAttempt’ refers to processing attempts at Cloud Integration.  At run 5,  LocalProcessingAttempt value will be 5. After run 5, the message moves to failed state, as the maximum message processing attempts is set as 5.Step 5:Now, change the acknowledge mode to ‘Automatic on Exchange Complete’.Trigger the event message from API testing tool. After trigger, the message is still in the queue, while retry is going on at Cloud Integration.Now, after 4 retries, in Cloud Integration, failed status is sent back to the AEM. During the 5th run, we can see the local processing attempt as 5.As we have provided, the re-delivery count as 3 in AEM, the AEM re-delivers it 3 times.Below shows 1st re-delivery i.e, 2nd delivery from AEM (Run 5 at Cloud Integration)3rd re-delivery from AEM (Run 5 at Cloud Integration) Then the message is moved to DMQ (Dead Message Queue). Please note, the DMQ queue must be created beforehand. DMQs are not automatically created during message failure. Scenario 4: Direct MessageFor direct message, if we use topic, no queue or topic endpoint configuration is required at AEM. If consumer is available during publishing of message, the message will be consumed, else the message will be lost.Step 1:Configure Publisher iFlow.Step 2:Configure Consumer iFlow.Step 3:Test and validate.Scenario 5: Durable topic endpointStep 1:Create a topic endpoint at AEM.Step 2:Configure the AEM receiver adapter which acts as a publisher.Configure the AEM sender adapter which acts as a subscriber.Durable Topic endpoint supports a single subscription. Once a consumer is attached, the topic name gets populated.Now, currently the topic endpoint has 2 messages queued.Step 3:Change the topic name at Cloud Integration side to ‘product/create’.After changing the topic name to ‘product/create’ and deploying the consumer iFlow, all existing messages are deleted automatically without delivering it to the consumer and the recent topic name gets reflected.Scenario 6: Use of SelectorStep 1:Configure Publisher iFlow as below. Set event type, priorty and status as user properties.Step 2:Populate the selector field at AEM sender adapter as below. Step 3:Test with payload field values as below:data/eventTypeorder.updateddata/statusdelayeddata/priorityhighCheck the trace of consumed message at Cloud Integration.Now, test with payload field values as below:data/eventTypeorder.updateddata/priorityhighThe message is not consumed from the queue as the selector does not match.Scenario 7: Send NACK to PublisherGeneral behavior is If topic subscription does not exist for delivery mode persisted(guaranteed) or direct, no error response is sent to the publisherIf queue does not exist for delivery mode persisted(guaranteed), error response is sent to the publisherIf the queue does not exist for delivery mode direct, no error response is sent to the publisher.The profile related to used client username (solace-cloud-client) is ‘default’ at AEM. Go to ‘default’ profile at AEM.After ‘Reject Messages to Sender On No Subscription Match Discard’ is enabled, if no matching topic subscription is found for persistent delivery mode, an error message is returned to the publisher.Now, if we stop the incoming calls to the queue,we will get the error message as below. Now, we change the property ‘Reject Messages to Sender on Discard’ on the queue to ‘Always’,we will get the error message as below,Scenario 8: TTL (Time to Live) Enable TTL feature of queue at AEM.If the message is not consumed, the message will be moved to DMQ after 1 min. Here, no TTL specific settings are configured in the receiver channel at Cloud Integration.If we don’t want to store it in the DMQ and discard the message after TTL, set DMQ eligible to false at publisher iFlow.After TTL, the stats will reflect the same at AEM.If TTL is set at receiver adapter, the minimum of the queue level TTL and the publisher level TTL takes effect. Scenario 9: RDP (REST Delivery Point)We have a common DMQ queue for all processes and the Rest Delivery points to Cloud Integration endpoint. Based on the endpoint, Cloud Integration forwards it for relevant processing of the DMQ messages.Business ObjectEndpointBusiness Partnerredeliver/businesspartner/<BPNumber>Orderredeliver/order/<OrderNumber>Step 1:Design the publisher iFlow as below.User Property named ‘identifier’ is added in AEM receiver adapter.Step 2:Create RDP at AEM.Create Rest Consumer with authentication type ‘OAuth2’ for Cloud IntegrationBind the AEM queue to the RDP endpoint.On the queue bindings settings, turn on the ‘Substitution Expressions’ for ‘Request Target Evaluation’. The below substitution expression is used for target endpoint genertion. The expression ${topic(1)} represents the 1st level of the topic./http/redeliver/${topic(1)}/${userPropertyAsString(“identifier”)}In case of failure at RDP, it retries forever unless we disable ‘Try Forever’ and set ‘Maximum Redelivery Count’ at DMQ.Step 3:The generated endpoint for order message is shown below:The generated endpoint for business partner message is shown below: Scenario 10: Replay LogIf replay log is enabled, the messages can be replayed from Broker without publisher intervention. To avoid saving the log for all queues, we can enable logging based on topic subscription.For example, the consumer ‘Cloud Integration’ forwards message to a target but the target system is down. All the messages from that point need to be replayed. For that, we need a start pointer and ‘ReplicationGroupMessageId’ can be used. As a pre-requisite, replication group message id for each message should be logged as a custom header at Cloud Integration.Now, replay all the messages after the above replication group message id. Please note that selective replay of messages or providing an end replication group message id is not possible. Therefore, idempotency must be implemented on the target side to avoid processing duplicate messages.Messages after Replication Group Message ID will be replayed.Similarly, we can create a new queue and attach it to a new consumer and replay the previous log related to its topic subscription from the beginning or from a particular date and time or based on replication group message id.Scenario 11: Parallel ConsumersNo. of consumers = (no. of worker node) * (no. of parallel consumer).If the tenant has 2 worker nodes and 1 parallel consumer, the consumer attached the queue will show up as below:If we increase the parallel consumer to 5, the consumer count becomes 10.Only 1 consumer is active, the remaining 9 consumers are in inactive state. This is because the queue type is ‘Exclusive’.Now, if we change the queue type to ‘Non-Exclusive’, all the consumers are active. E.g., if we want a particular object id (say Business Partner) to be processed by the same consumer, we can utilize the partition concept. Set the partition count as 5.Set the parallel consumers as 3As the tenant has 2 worker nodes, the total no. of consumer will show up as 6. Only 1 consumer is inactive. Remaining consumers are in active state as the number of partition count is 5.If we set ‘JMSXGroupID’ value, it will be used as a partition key. For example, for BP number ‘10001234’, the message was consumed by Flow ID 248 and for BP number ‘10001236’, it was consumed by Flow ID 281.All the updates related to BP ‘10001236’, are lined up for the same consumer.Scenario 12: Request Reply PatternThe publisher iFlow is designed as below:Configure the AEM receiver channel as below. This will create a non-durable queue at AEM when the message is sent from publisher iFlow.Consumer iFlow receives the headers populated from Publisher iFlow related to response propagation, such as ReplyToDestination, ReplyToEndpointType and CorrelationIdDesign Consumer iFlow as belowPlease note: The delay is used here only to simulate processing time instead of adding additional pallets. AEM receiver adapter is used to send back the response.After ‘request-reply’ step in the publisher iFlow message trace, we will see the response headers. Lets test the publisher iFlow.As we can see, the response message is received.Thank you for reading this blog post.    Read More Technology Blog Posts by Members articles 

#SAP

#SAPTechnologyblog