PhixFlow Help

Create Unique Indexes / Record Identifiers

Scenario

When processing large volumes of data, where the data is uniform, but coming from many different sources (e.g. a mediation system will process data from multiple switches) and doing a great many transformations, tagging each record at the point it enters the system with a unique number is useful for auditing and tracking.

Generate a counter - a unique number - for all the records in a stream set.
Combine 2 or more stream sets and for each stream set have the counter continue to increment over the value of the last record of the previous stream set.

Solution:

  • Create a Stream to read in data from a DB table or file
  • Create an attribute of that stream to contain a unique number
  • Use the built in cache() function to set the attribute. Increment the cache for the attribute to ensure that the next record contains a unique number.

 

Please let us know if we could improve this page feedback@phixflow.com