...
Step-by-step guide: Removing Duplicates
...
- Load all data (including duplicates) into a stream
- Create a new stream from this stream - make it an aggregate stream.
On the pipe linking the two streams, set the maximum number of records to be one and group it on the field with duplicated data.
Panel bgColor #DCDCDC titleBGColor #808080 title Optional If this is case 3, apply sorting on another attribute, depending on which record you want. E.g. to get the latest record, sort by the last updated date. See 'outMult' in the screenshot below for illustration.
- Run the analysis and inspect the resultant stream data. There should now be only unique records.
...