Reading all data from a input stream

By the end of this chapter you will be able to:

• Read all data from an input stream

In this exercise you will create a report of all customer sign ups so far, using a pipe to supply all data (rather than latest):

• Open the model Process Daily Signup Files – you created this in the previous exercise
• Hover over Customer SignUps Seq and press
• Call this stream Customer SignUp Totals
• Open the configuration form for the pipe
  • On the Details tab set Data to Read to All
  • Add a sort/group attribute: REGION
  • Add an aggregate attribute:
    • Aggregate Function: Sum
    • Attribute: CustomerSignUps
    • New Attribute Name: TotalCustomerSignUps
• Drag attributes Region and TotalCustomerSignUps from the pipe onto Customer SignUp Totals
• Save your changes

Because this new stream consider all runs so far, you will need to clear out the data in the input stream - ReadCustSignUpFilesSeq. To do this, you will do a rollback:

• Right-click on the stream ReadCustSignUpFilesSeq
• Select Rollback Stream
• In the rollback form:
  • Select Rollback all data
  • Untick the box Keep Old Stream Sets
  • Press OK

The Console will pop up, and you will see a log file telling you how many data records were cleared out of the Stream.
You also need to refresh your sequence number:

• Find SignUpSeq in the list of sequences
• Delete it
• Re-create it as above

Run your new stream:

• Run Analysis on Customer SignUp Totals three times
• Review the data – the totals in Customer SignUp Totals for each region accumulate with each run; this is because the input pipe has Data to Read set to All