Deduplicate

The deduplicate component will look at a property-key from the incoming message. It will only send unique messages to the Output.

When a key is not present in an object an error will be sent to the Error-output, or will be logged if nothing is connected to this output.

Configuration

In the configuration you must specify what the property-key is to inspect.

• Property Key: The path of the property to inspect.
• No Show: Whether or not to show the most frequent duplicates and their occurrence counts.
• Limit: How many entries to show.

Property Key

A property key in its simplest form is just a single property name. For example, the property key _id will look for that field of the incoming object, resulting in the key abcd-e1fgh-i1jkl from the following object:

    {
        "_id": "abcd-e1fgh-i1jkl",
        "data": "some value"
    }
        

When a property is nested deeper in an object, you can append multiple keys separated by a dot (.). For example user._id. This will look recursively for the key.

Inputs | Outputs

Deduplicate

Input

Output

Duplicate

Error

00 0 (ilyxsu21)

Any message that is an object can be sent through, but should by design be an object that has the key defined in the Property Key.