Elasticsearch Bulk

Component that can talk to Elasticsearch and utilizes bulk API endpoint. Requires Elasticsearch 8.4.x or higher.

Configuration

Bulk Size

Number specifying the number of documents in each bulk request.

Bulk Timeout

Timeout in milliseconds before triggering a request in case the bulk size is not reached.

Connection

Connection URL

Connection URL to the API endpoint.

Example: https://localhost:9200/

Authentication

Username

As required by the API.

Example: elastic

Password

As required by the API.

Example: changeMe

HTTP(s)

The path to the Elasticsearch certificate.

Example: Home/Documents/Elasticsearch/config/certs/http_ca.crt

Rate limit

Request limit

The max amount of requests during the interval.

Interval

The interval in milliseconds in which the requests happen. Should be a multiple of 250.

Max concurrent

The maximum concurrent executions.

Inputs | Outputs

Elasticsearch Bulk

Input

Output

Error

00 0 (ilyzdloe3)

Description:

Performs multiple operations in a single API call. This reduces overhead and can greatly increase indexing speed. As an input, this component takes any operations and combines them until reaching the bulk size or until the bulk timeout is reached and then triggers the request.

On successful execution, the output will be the response of Elasticsearch.

On error, the default error handling applies.

Example:

A message containing the operation and, depending on the type of operation, the data.

{
      "operation":{},
      "data":{}

}

Example: Most common types of operations:

Index operation

{
      "operation":{ "index" : { "_index" : "test", "_id" : "1" } },
      "data":{ "field1" : "value1" }

}

Delete operation

{
      "operation":{ "delete" : { "_index" : "test", "_id" : "2" } },
      "data":{}

}

Update operation

{
      "operation":{ "update" : {"_id" : "1", "_index" : "test"} },
      "data":{ "doc" : {"field2" : "value2"} }

}