Document Retrieve

The Document Retrieve component allows you to retrieve documents from the Content Store.

By default it will retrieve all the documents, additionally you can filter which documents are retrieved based on query and specify a sort order.

note

Aggregates and projects are not supported.

See how it works:
Document Retrieve Explainer

note

Also covered in the Content Store overview video:
Watch video

Configuration

Connection string

A MongoDB connection string.

Example: mongodb://<username>:<password>@localhost:27017/<databaseName>

Here <databaseName> is the database to retrieve the content from.

Use TLS

Whether or not to use TLS in case your mongoDB requires TLS.

Allow Invalid Certificates

Checking this will disable certificate validation.

warning

Specifying this option in a production environment makes your application insecure and potentially vulnerable to expired certificates and to foreign processes posing as valid client instances.

Certificate Authority File

One or more certificate authorities to trust when making a TLS connection.

Example: .\ca.pem

info

All configuration values listed below can be overridden using the incoming object.
If a configuration value expects an object (for example query, sort or projection), the value must be provided as stringified JSON in the incoming object.

Query

A Mongo query that can be passed in find().

Example: {"kind":"CONTAINER"}

note

The Query input field uses HandleBars. You can use Handlebars to create dynamic queries. For nested keys, you can use the so-called dot-notation. For more information, please read HandleBars

An empty query value or empty object {} will return all the data from the database.

Override key: query (stringified JSON)

Sort

Specify a sort order using the Mongo sort syntax which can be passed into Mongo sort().

Example: {"kind":1}

Override key: sort (stringified JSON)

Projection

Specify a projection using the Mongo projection syntax which can be passed into Mongo projection().

Example: {"kind":1}

Override key: projection (stringified JSON)

Limit

Specify a limit on the number of documents that you want to retrieve.

Override key: limit (number)

Skip

Specify the amount of documents that should be skipped before results are retrieved.

Override key: skip (number)

Enable allowDiskUse

By enabling allowDiskUse, MongoDB can process the sort operation even if it requires more than 100 megabytes of system memory. If this option is disabled and the operation required more than 100 megabytes of system memory, MongoDB will return an error: Executor error during find command :: caused by :: Sort exceeded memory limit.

Override key: allowDiskUse (boolean)

Enable case insensitive sort

By default Mongo sorts upper case characters before lower case characters. By enabling case-insensitive sorting, this behavior will be disabled.

note

This feature will decrease the performance of the query and should only be used when case insensitivity matters.

Override key: sortCaseInsensitive (boolean)

Enable parsing numberDecimal

By default Mongo will retrieve NumberDecimals (in this example 1.05) as { $numberDecimal: "1.05" }, By enabling parsing numberDecimal the returning value will be a float: 1.05 instead.

note

Rounded numbers will parse to a number that does not have its decimals anymore. For example { "testKey": { $numberDecimal: "1.0" } } will parse to { "testKey": 1 }

Override key: parseNumberDecimal (boolean)

Include related data

Only include related data that is flagged for migration

Only related data, that has migration.migrate set to true, will be retrieved. Otherwise all related data will be retrieved.

Override key: includeMarkDataOnly (boolean)

Include source/target versions

Enable to also retrieve other versions of the queried documents. The fields sourceVersions:[] and targetVersions:[] will be added to each document if either feature is enabled, respectively. When enabled, a projection, skip and limit option can be set to only retrieve a subset of all versions. Limit can be left 0 to impose no limit.

Override keys:

• includeSourceVersions (boolean)
• sourceVersionsProjection (stringified JSON)
• sourceVersionsSkip (number)
• sourceVersionsLimit (number)
• includeTargetVersions (boolean)
• targetVersionsProjection (stringified JSON)
• targetVersionsSkip (number)
• targetVersionsLimit (number)

Include source/target binaries

Enable to also retrieve the binaries associated with a document. The source.binaries and target.binaries arrays will be replaced with the retrieved binaries. When enabled, a projection, skip and limit option can be set to retrieve a subset of the arrays as seen in the document. Limit can be left 0 to impose no limit.

Override keys:

• includeSourceBinaries (boolean)
• sourceBinariesProjection (stringified JSON)
• sourceBinariesSkip (number)
• sourceBinariesLimit (number)
• includeTargetBinaries (boolean)
• includeTargetBinariesProjection (stringified JSON)
• targetBinariesSkip (number)
• targetBinariesLimit (number)

Include source/target ACLs

Enable to also retrieve the access control lists associated with a document. The source.acls and target.acls arrays will be replaced with the retrieved control lists. When enabled, a projection, skip and limit option can be set to retrieve a subset of the arrays as seen in the document. Limit can be left 0 to impose no limit.

Override keys:

• includeSourceACLs (boolean)
• sourceACLsProjection (stringified JSON)
• sourceACLsSkip (number)
• sourceACLsLimit (number)
• includeTargetACLs (boolean)
• targetACLsProjection (stringified JSON)
• targetACLsSkip (number)
• targetACLsLimit (number)

Include source/target audit logs

Enable to also retrieve the audit logs lists associated with a document. The source.auditLogs and target.auditLogs arrays will be replaced with the retrieved control lists. When enabled, a projection, skip and limit option can be set to retrieve a subset of the arrays as seen in the document. Limit can be left 0 to impose no limit.

Override keys:

• includeSourceAuditLogs (boolean)
• sourceAuditLogsProjection (stringified JSON)
• sourceAuditLogsSkip (number)
• sourceAuditLogsLimit (number)
• includeTargetAuditLogs (boolean)
• targetAuditLogsProjection (stringified JSON)
• targetAuditLogsSkip (number)
• targetAuditLogsLimit (number)

Rate limit

The rate limit settings are used to throttle the number of documents that are sent into the flow. The minimum interval is set at 10 milliseconds. The minimum batch size is 1 outgoing message per interval.

Batch size

Allows you to configure the batch size.

Override key: batchSize (number)

Interval

The interval in milliseconds in which the batches are sent.

Override key: interval (number)

Inputs | Outputs

Document Retrieve

Input

Output

Finished Output

Error

00 0 (ilywox0em)

The incoming data can be used to override settings, except for the connection and TLS options. If you don't need to override any settings or provide variable values when using handlebars in settings, the incoming data can be empty. For more information about the available override keys, see the configuration section.

Example:

//query in config:
{ "kind":"{{kind}}" }

//incoming data:
{
  _id: "135176441e008d9d0c5ba0cee61783ce",
  kind: "CONTAINER"
}