Scatter-Gather Router

The Scatter-Gather component is a routing event processor that processes a Mule event through different parallel processing routes that contain different event processors.

Each route receives a reference to the Mule event and executes a sequence of one or more event processors. Each of these routes uses a separate thread to execute the event processors, and the resulting Mule event can be either the same Mule event without modifications or a new Mule event with its own payload, attributes, and variables. The Scatter-Gather component then combines the Mule events returned by each processing route into a new Mule event that is passed to the next event processor only after every route completes successfully.
The routing message processor Scatter-Gather sends a request message to multiple targets concurrently. It collects the responses from all routes, and aggregates them into a single message.

Scatter Gather in MuleSoft

The ScatterGather component executes each route in parallel.
Each route receives a reference to the Mule event and executes a sequence of one or more event processors.
Each of these routes uses a separate thread to execute the event processors, and the resulting Mule event can be either the same Mule event without modifications or a new Mule event with its own payload, attributes, and variables.The Scatter-Gather component then combines the Mule events returned by each processing route into a new Mule event that is passed to the next event processor only after every route completes successfully
Scatter-Gather component to have at least two routes; otherwise, your Mule application throws an exception and does not start.
Use Try scope in each route of a ScatterGather component to handle any errors that might be generated by a route’s event processors

The following diagram details the behavior of the Scatter-Gather component:

Scatter Gather Component:

Scatter-Gather router receives a mule event and sends a reference/copy of mule event to each route simultaneously.
- In our case, Scatter-Gather sends mule events to route-1, route-2 and route-3 parallel.
All the routes start processing the mule event in parallel. After all routes finish the processing then returns a mule event.
- The event can be a same mule event or a new event created by the processors in the route.
After all the routes finish their processing, the Scatter-Gather router combines the all resulting Mule events from each route and create a new mule event then passes the mule event to the next processor.

Example

Drag and drop HttpListener and Configure

Global configuration

General configuration

Step 2: Add a Set Variable and set a variable and name it.

Scatter-Gather router config:

Route-1:

Add HTTP Requester node and refer to the global HTTP Requester configuration and then configure the General parameters.

Add Set Variable and override the variable the we set before the Scatter-Gather router.

Route-2:

Add HTTP Requester node and refer to the global HTTP Requester configuration and then config the General parameters.

Add Set Variable and override the variable the we set before the Scatter-Gather router.

The Scatter-Gather router should look like:

Step 4: Add Transform Message and simply transform the object to JSON model.

Step 5: At last add the logger and log the variable “name“.

That’s it. Now run the mule application and trigger a request to http://localhost:8089/scatterGather.

Scatter-Gather returns the payload as an Object. The sample output would look like:

{

“0”: {

“inboundAttachmentNames”: [],

“exceptionPayload”: null,

“inboundPropertyNames”: [],

“outboundAttachmentNames”: [],

“payload”: {

“userId”: 1,

“id”: 1,

“title”: “delectus aut autem”,

“completed”: false

“outboundPropertyNames”: [],

“attributes”: {

“headers”: {

“date”: “Sun, 21 Feb 2021 05:42:24 GMT”,

“content-type”: “application/json; charset=utf-8”,

“content-length”: “83”,

“connection”: “keep-alive”

“reasonPhrase”: “OK”,

“statusCode”: 200

}

“1”: {

“inboundAttachmentNames”: [],

“exceptionPayload”: null,

“inboundPropertyNames”: [],

“outboundAttachmentNames”: [],

“payload”: {

“userId”: 1,

“id”: 2,

“title”: “quis ut nam facilis et officia qui”,

“completed”: false

“outboundPropertyNames”: [],

“attributes”: {

“headers”: {

“date”: “Sun, 21 Feb 2021 05:42:24 GMT”,

“content-type”: “application/json; charset=utf-8”,

“content-length”: “99”

“reasonPhrase”: “OK”,

“statusCode”: 200

}

Note: We have removed few headers to keep the response simple.

The above response contains the attributes and the payload but most of the times the attributes are not required.

In such cases, to extract the only payload you can use flatten function to merge the both the responses.

To do that, replace the above dataweave script as shown below:

%dw 2.0

Output application/json

—

Flatten (payload..payload)

The response would look like after the above transformation:

Point to Rememeber:

Scatter-Gather returns the mule event as an Object.
Use flatten function to merge the both responses.

Variable Propagation in Scatter-Gather

Now let’s see what happens to variables when we override the variables inside Scatter-Gather router.

Every route starts with the same initial variable values. Modifications to a variable within a specific route do not affect other routes. So, if a variable is added or modified in one route, then, after aggregation, the value is defined by that route. If a variable is added or modified by more than one route, the value is added to a list of all the values defined for that variable within all the routes.

So, if a variable is added or modified in one route, then, after aggregation, the value is defined by that route.

Concurrency Control

By default the Scatter-Gather process the mule event in parallel but sometimes we may need to process the mule event sequentially.

To achieve the sequential process, we can use two methods: