Deduplicate

The deduplicate component will look at a property-key from the incoming message. It will only send unique messages to the Output.

When a key is not present in an object an error will be sent to the Error-output, or will be logged if nothing is connected to this output.

Configuration

In the configuration you must specify what the property-key is to inspect.

• Property Key: The path of the property to inspect.
• No Show: Whether or not to show the most frequent duplicates and their occurrence counts.
• Limit: How many entries to show.

Property Key

A property key in its simplest form is just a single property name. For example, the property key _id will look for that field of the incoming object, resulting in the key abcd-e1fgh-i1jkl from the following object:

{
  "_id": "abcd-e1fgh-i1jkl",
  "data": "some value"
}

When a property is nested deeper in an object, you can append multiple keys separated by a dot (.). For example user._id. This will look recursively for the key.

Inputs

• Name: Input

  • Description: Any message that is an object can be sent through, but should by design be an object that has the key defined in the Property Key.

Outputs

• Name: Output

  • Description: If and only if this key is encountered for the first time, the incoming message will be sent to this output.

• Name: Duplicate

  • Description: If a key is detected for a second time, the incoming message will be sent to this output.