Load balancer

This accelerator flow will distribute the work of a flow over multiple workers. It will do this by splitting the work and assigning objects to the workers. The workers will then process the objects and report back to the load balancer. The load balancer will then assign the next object to the worker that reported back first.

Some workers can process multiple objects at the same time, while others can only process one object at a time. This depends on the flow that is being distributed. You can configure the maximum number of objects that a worker can process at the same time in the workerThreshold setting.

You can set the number of available workers in the availableWorkers setting, by specifying the IDs of the workers. The load balancer will then assign the objects to the workers. The load balancer will keep track of the objects that are assigned to the workers and will not assign the same object to multiple workers.

The objectsToAssignQuery setting is used to specify the query that will be used to retrieve the objects that need to be processed. The query should return a list of objects that need to be processed. The load balancer will then assign the objects to the workers.

The Box.com (3. Traverse roots) and the Box.com (5. Download binaries) are examples of worker flows that are controlled by the load balancer.

Settings

This section describes the settings for the load balancer accelerator flow.

mongoConnection

The Mongo connection string including the database name to connect to.

workerThreshold

The maximum number of objects that a worker can process at the same time.

availableWorkers

The list of worker IDs that are available for processing the objects.

Example: ["worker1", "worker2"]

objectsToAssignQuery

The query that will be used to retrieve the objects that need to be processed.

mongoConnection

The Mongo connection string including the database name to connect to.