By the end of this chapter you will be able to:

• Create lookup pipes
• Enrich a data set using a lookup pipe

Copy data from merge

First, you will create a new stream to copy over all data from your merge:

1. Hover over the stream Customer All Details, and press  Add a new Stream
2. Call this stream: Customer Details with Region
3. Copy all attributes into your new stream:
   1. Hover over Customer All Details and in the hover menu press  Show Stream Attributes
   2. Select all attributes in the list:
      1. Select the top attribute
      2. Keep shift pressed down and select the bottom attribute in the list
   3. Drag the attributes on top of the stream Customer Details with Region
4. Run your new stream and look at the data in it to verify that you have copied data from the merge stream

5. Remember to save your model layout

   Here you have copied data from a merge Stream into a new stream – but this method works for any type of input stream. Copying data into a new stream is a common starting point for a modelling step.

   Create a lookup into reference data

Now you will add a lookup pipe from Regions by County to enrich the data in Customer Details with Region, by finding the region for each customer address:

1. Hover over RegionsByCounty and press  Add a new Connector
2. Move the pointer over the top of Customer Details with Region and select – this will attach the end of the pipe to the selected stream
3. In the pipe details form that pops up:
   • Enter the Name: region
   • Type: Look-up
   • Data To Read: leave as the default – Latest
   • Press  Apply
   • In the Order/Index section we will set up the fields to match across the two data sets:
     1. Press  Show Source and Target Attributes
     2. Drag the attribute County from the left hand list (the input attributes from the stream RegionsByCounty) into the list of Order/Index attributes
     3. Drag the attribute County from the right hand list (the output attributes from the stream Customer Details with Region) on top of the Order/Index attribute County
        • In the Order/Index section, you should see that the Index Expression column has updated to _out.County


        • This means that the lookup will match records in the reference data (RegionsByCounty) where County = County in the output data (Customer Details with Region)
     4. Press 
   • Note that the pipe linking RegionsByCounty to Customer Details with Region is dashed; this indicates that this is a lookup pipe
   • When adding lookup pipes remember that the arrow points in the direction that the data flows; here you are getting data from RegionsByCounty to add to Customer Details with Region

Enrich a data set using a Lookup Pipe

Now you can add the region – from RegionsByCounty – to Customer Details with Region:

1. Hover over RegionsByCounty and in the hover menu press  Show Stream Attributes
2. The list of the stream attributes will pop up
3. Drag the attribute Region from RegionsByCounty on top of Customer Details with Region
4. Run analysis on Customer Details with Region
5. Have a look at the data in this stream – check that the Region attribute has been populated with the value looked up from RegionsByCounty