Create Unique Indexes / Record Identifiers
Scenario
When processing large volumes of data, where the data is uniform, but coming from many different sources (e.g. a mediation system will process data from multiple switches) and doing a great many transformations, tagging each record at the point it enters the system with a unique number is useful for auditing and tracking.
Generate a counter – a unique number – for all the records in a stream set.
Combine 2 or more stream sets and for each stream set have the counter continue to increment
over the value of the last record of the previous stream set.
Solution:
- Create a Stream to read in data from a DB table or file
- Create an attribute of that stream to contain a unique number
- Use the built in cache() function to set the attribute. Increment the cache for the attribute to ensure that the next record contains a unique number.
See Also
- href="../../../../../../doc/training/Exercises.pdf">Patterns and Exercise Training Document
- href="http://www.phixflow-online.com:8081/patterns">PhixFlow Online Patterns and Training