Page Comparison

...

Configuration of In Memory streams is illustrated in this article with an example:

You have a file of record records with, on each line, two key values: KeyVal (e.g. "Key5") , and Node (e.g. "AA112"); and a count of events for that key combination, EventCount. A snippet of the files appears as file is shown below:

103,"AA112","Key5"
115,"AA174","Key3"
126,"AA100","Key1"
145,"AA169","Key5"
126,"AA164","Key3"
158,"AA196","Key0"
117,"AA150","Key2"

...

The following configuration will result in the model:

To do create this model:

Create a file collector

• Create a file collector on the file as normal, with attributes:

  • EventCount

  • Node

  • EventCount

Create an In Memory Stream

Create an In Memory stream:

• Stream Function = Calculate
• Storage Type = In Memory
• Linked to the file collector with a pipe "in"Attributes

Create the stream attributes

Create the stream attributes as below:

The stream attributes above are configured as:

KeyVal - a key value

...

 

Populated with the expression in.KeyVal to take

...

KeyVal from the file collector

To make

...

this attribute a key value the Key flag has been ticked on the stream attribute configuration form:

This means that a record will be created in the cache for each new value of KeyVal

...

Note - a key value

...

Configured similarly to KeyVal, with the flag Key ticked

This means that a record will be created in the cache of each new combination of KeyVal and Node

EventCountTotal - an accumulated total count

Populated with the expression: _cacheRecord.EventCountTotal+in.EventCount

The variable _cacheRecord is an Internal Variable: a record is created in the cache for each combination of key values, and _cacheRecord is the record for the current key. The record has a value for each attribute defined in the stream, e.g. EventCountTotal.

So the expression _cacheRecord.EventCountTotal+in.EventCount takes the current total count (_cacheRecord.EventCountTotal) then adds the new value read in from the file to it (+in.EventCount)

RecordCount -

...

a record count

Similarly to EventCountTotal, this attribute is building a total across records which share a key value combination

In this case we are generating a simple count of records from the file with the expression _cacheRecord.RecordCount+1

Store the results of the In Memory Stream

Create a stream to record the results of the In Memory stream:

...

this is a simple calculate stream which reads in the attributes of the In Memory stream, with attributes as below

Results

After running the final stream we get results like:

...