Document Retrieve

The Document Retrieve component allows you to retrieve documents from the Content Store.

By default it will retrieve all the documents, additionally you can filter which documents are retrieved based on query and specify a sort order.

note

Aggregates and projects are not supported.

Handlebars

This component lets you use Handlebars templates. More information about Handlebars can be found in this section

Configuration

Connection string

A MongoDB connection string.

Example: mongodb://<username>:<password>@localhost:27017/<databaseName>

Here <databaseName> is the database to retrieve the content from.

Use TLS

Whether or not to use TLS in case your mongoDB requires TLS.

Allow Invalid Certificates

Checking this will disable certificate validation. Warning: specifying this option in a production environment makes your application insecure and potentially vulnerable to expired certificates and to foreign processes posing as valid client instances.

Certificate Authority File

One or more certificate authorities to trust when making a TLS connection. In order to access the local filesystem, the XILL4_WORKDIRS environment variable must be set to the path of the directory to be accessed.

Example: .\ca.pem

Query

A Mongo query that can be passed in find(). Example: {"kind":"CONTAINER"}

You can use Handlebars to create dynamic queries. For nested keys. you can use the so-called dot-notation.

An empty query value or empty object {} will return all the data from the database.

Sort

Specify a sort order using the Mongo sort syntax which can be passed into Mongo sort(). Example: {"kind":1}

Projection

Specify a projection using the Mongo projection syntax which can be passed into Mongo projection(). Example: {"kind":1}

Limit Specify a limit on the number of documents that you want to retrieve.

Skip Specify the amount of documents that should be skipped before results are retrieved.

Enable allowDiskUse

By enabling allowDiskUse, MongoDB can process the sort operation even if it requires more than 100 megabytes of system memory. If this option is disabled and the operation required more than 100 megabytes of system memory, MongoDB will return an error: Executor error during find command :: caused by :: Sort exceeded memory limit

Enable case insensitive sort

By default Mongo sorts upper case characters before lower case characters. By enabling case-insensitive sorting, this behavior will be disabled.

note

This feature will decrease the performance of the query and should only be used when case insensitivity matters.

Enable parsing numberDecimal

By default Mongo will retrieve NumberDecimals (in this example 1.05) as { $numberDecimal: "1.05" }, By enabling parsing numberDecimal the returning value will be a float: 1.05 instead.

note

Rounded numbers will parse to a number that does not have its decimals anymore. For example { "testKey": { $numberDecimal: "1.0" } } will parse to { "testKey": 1 }

Include related data

Only include related data that is flagged for migration

Only related data, that has migration.migrate set to true, will be retrieved. Otherwise all related data will be retrieved.

Include source/target versions

Enable to also retrieve other versions of the queried documents. The fields sourceVersions:[] and targetVersions:[] will be added to each document if either feature is enabled, respectively. When enabled, a projection, skip and limit option can be set to only retrieve a subset of all versions. Limit can be left 0 to impose no limit.

Include source/target binaries

Enable to also retrieve the binaries associated with a document. The source.binaries and target.binaries arrays will be replaced with the retrieved binaries. When enabled, a projection, skip and limit option can be set to retrieve a subset of the arrays as seen in the document. Limit can be left 0 to impose to limit.

Include source/target ACLs

Enable to also retrieve the access control lists associated with a document. The source.acls and target.acls arrays will be replaced with the retrieved control lists. When enabled, a projection, skip and limit option can be set to retrieve a subset of the arrays as seen in the document. Limit can be left 0 to impose to limit.

include source/target Audit logs

Enable to also retrieve the audit logs lists associated with a document. The source.auditLogs and target.acls arrays will be replaced with the retrieved control lists. When enabled, a projection, skip and limit option can be set to retrieve a subset of the arrays as seen in the document. Limit can be left 0 to impose to limit.

Rate limit

The rate limit settings are used to throttle the number of documents that are sent into the flow. The minimum interval is set at 10 milliseconds. The minimum batch size is 1 outgoing message per interval.

Batch size

Allows you to configure the batch size.

Interval

The interval in milliseconds in which the batches are sent.

Inputs

• Name: Input

  • Description: The incoming message isn't used unless the query uses a variable from this object. The incoming message can be empty if you don't need any variables.

  • Example:

    //query in config:
    { "kind":"{{kind}}" }
    
    //incoming message:
    {
      _id: "135176441e008d9d0c5ba0cee61783ce",
      kind: "CONTAINER"
    }

Outputs

• Name: Output

  • Description: Each retrieved document, combined with the incoming message.

  • Example:

    {
      value: {
        // the incoming message, if there isn't any this is an empty object
      },
      result : {
        // the resulting document
      }
    }

• Name: Finished Output

  • Description: Send result count as output signal after every document is outputted from a query.

  • Example:

    {
      "resultCount": 10
    }