Skip to main content
Version: 4.51.0

Deduplicate

The deduplicate component will look at a property-key from the incoming message. It will only send unique messages to the Output.

When a key is not present in an object an error will be sent to the Error-output, or will be logged if nothing is connected to this output.

Configuration

Display Settings

In the configuration you must specify what the property-key is to inspect.

  • Property Key: The path of the property to inspect.
  • No Show: Whether or not to show the most frequent duplicates and their occurrence counts.
  • Limit: How many entries to show.

Property Key

A property key in its simplest form is just a single property name. For example, the property key _id will look for that field of the incoming object, resulting in the key abcd-e1fgh-i1jkl from the following object:


{
"_id": "abcd-e1fgh-i1jkl",
"data": "some value"
}

When a property is nested deeper in an object, you can append multiple keys separated by a dot (.). For example user._id. This will look recursively for the key.

Inputs | Outputs


Deduplicate
Input
Output
Duplicate
Error
0 0 0 (ilyxsu21)
Any message that is an object can be sent through, but should by design be an object that has the key defined in the Property Key.