By the end of this chapter you will be able to:

• Combine data sets (a union) with a calculate stream
• Record the source for each record in a stream
• Use the internal variable _pipeName

In this exercise you will produce a single list of updates to customers' Cable TV packages. Customers can update their packages both through a customer care system and in shops. You will bring all these updates together into a single list.

Read in Data Sets from Files

For this exercise you need two files from the zip file you downloaded in  2 Using Excel Templates, step 5. The files contain updates from the two sources: the customer care service and the shop.

• <path>\train\inputData\ChannelPackages\CustomerCareUpdates\CustomerCareUpdates.txt
• <path>\train\inputData\ChannelPackages\ShopUpdates\ShopUpdates.txt

1. Create a new model with the Name Channel Package Check.
2. Add a file collector to your model and load the data.
   1. PhixFlow displays a form for the new file collector. Set the Name to Customer Care Updates.
   2. In the model, hover over the Channel Package Check file collector to display the context toolbar.
   3. Click  Upload Managed File. 
   4.  Navigate to the directory <path>\train\inputData\ChannelPackages\CustomerCareUpdates.
   5. Select the file CustomerCareUpdates.txt. and clickUpload.
   6. PhixFlow adds a new stream to the model, called CustomerCareUpdates.
   7. In the model, hover over the new CustomerCareUpdates stream and click Run Analysis.
   8. Check the data is loaded. There should be 8 lines of data, with columns for Customer Ref, Sales Date and Package.
3. Add another file collector to your model and repeat the process described in step 2. This time:
   1. Set the file collector Name to Shop Updates.
   2. Upload <path>\train\inputData\ChannelPackages\ShopUpdates\ShopUpdates.txt.
   3. PhixFlow loads 7 rows of data to this stream.

Snapshot of the Channel Package Check model so far:

Combine Data Sets

1. Create a new stream in your model and populate its attributes.
   1. Hover over CustomerCareUpdates and click  Add a new Stream.
   2. In the form, set the Name to  Combined Updates.
   3. In the model, show the stream attributes for CustomerCareUpdates .
   4. Select all the attributes and drag them into the form for the Combined Updates stream. Drop them into the Attributes section.
2. Save your model layout.
3. Connect the ShopUpdates stream to the Combined Updates stream.
   1. Hover over ShopUpdates and click  Add a new Connector.

   2. Click Combined Updates to link the pipe to the stream.

      When PhixFlow adds data to a stream via a pipe, it automatically adds a reference to the input pipe expression. To merge the data sucessfully you must update the attribute expressions to remove the in. prefix.

4. Fix all the attribute expressions.
   1. Double-click on Combined Updates to open its configuration form. For each attribute:
   2. In the Attributes section, double-click on an attribute to open its form.
   3. In the form for the attribute, in the Basic Settings section, edit the Expression to remove in. and press .
      
5. In the Combined Updates stream form, press  to save the changes.
6. Run analysis on Combined Updates.
7. Check that you have the correct number of records in this stream when the analysis is complete.
8. Check the data set you generated to make sure that every field in every output record has been populated.

Snapshot of the Chanel Package Check model so far:

Record the Source for Each Update

You will now update your configuration to record the source for each update, in the set of combined updates. To do this:

1. Update the name of the pipe from the stream CustomerCareUpdates.
   1. Double-click on the pipe to open its configuration form.
   2. Update the Name to CC.
   3. Press  to save your changes.
2. Update the name of the pipe from ShopUpdates to shop.
3. Double-click the stream Combined Updates to open its configuration form.
4. In the Attributes section, add a new attribute and set:
   1.  Name: Source.

   2. Expression :

      if (_pipeName == "CC",
       "Customer Care"
       ,
       "Shop"
       )

5. To make Source the first attribute in the stream, drag it to the top of the attributes list.
6. Press  in the main stream configuration form.
7. Run analysis on Combined Updates.
8. Check the output data set and make sure that the source has been recorded on each record correctly.