Introduction

Merges can be used to consolidate records from two or more tables. In this page we will work through a simple merge scenario to explain how merges work. All resource used in this example are available here: 

• Businesses1.xlsx
• Businesses2.xlsx

Scenario

We have two sets of company information, Businesses 1.xlsx and Businesses 2.xlsx, that we want to merge together and remove duplicates. There are a number of challenges we must resolve to achieve this:

1. The attribute names in each file are different.

2. There are duplicate companies that exist in both files.

3. There are duplicates within the Businesses

   1

   2.xlsx file.

Solution

1. Import the data

   1. Using a file collector import each of the

      Busiesss

      Businesses Excel documents into their own table and run analysis on each to populate them with data.

   2. Make both of the tables static using the 

      Insert excerpt
      _static _static nopanel true

       option from the table hover over toolbar.

      1. This ensures we only read the files once.

2.  Create the Merge Table

   1. From the Create section of the analysis toolbar, click 

      Insert excerpt
      _datasource_show_tables _datasource_show_tables nopanel true

   2. drag the Image Modified Merge Table  onto your model.

   3. In the settings, set the Name for example Businesses Merged

   4. The attributes can be created manually using the plus button in the Attributes section. Here we will use the attributes from the Businesses 1 table to automatically setup our merge attributes.

      1. Hover over the Businesses 1 table and from the hover over toolbar click 

         Insert excerpt
         _attributes_show_hide _attributes_show_hide nopanel true

         .

      2. Tick all of the attributes and then drag them into the attributes section of the Merge Table. 

   5. Click Image Modified OK to save and close the settings for the merge table.

3. Connect the Inputs

   1. In the analysis model, hover over the Businesses 1 table.

   2. In

      From the hover over toolbar, select Image Modified Connector, then click the end of the arrow onto the merge table. This connector will pipe data into the merge table.

   3. In the

      A properties window

      that

      opens for the pipe

      in

      , set the name to B1 to better indicate the use of the pipe.

      It is good practice to utilise meaningful names.

   4. In the Sort/Group section of the properties click 

      Insert excerpt
      _attributes_show _attributes_show nopanel true

       and PhixFlow displays the list of attributes (columns).

   5. Drag the Name attribute into the Sort/Group section. This defines the attribute that will be used to merge on, we call this the key. Keys are listed in order and we reference them by number, therefore the keys must appear in the same order on all pipes leading into the merge table. This means attributes we are merging on do not need to have the same name.

   6. Similarly, add a

      connector 

      connector from the table Businesses 2 to the merge table and add the Company Name to the Sort/Group.

      1. Businesses 2 has duplicates, we can define which records to keep. Here we will use the most recent Onboarding Date.

      2. Drag the Onboarding Date attribute into the Sort/Group section.

      3. Double click on the attribute and untick group and set Direction to Z-A. What we have done here is to use th 

4. Configure the Merge Table

   1. Finally we now need to tell the Merge Table what we are using to merge on, and which values from the tables we want to keep. For example, if there a company in both Business Tables, we can set the precedence to use the one from Business 1.

   2. Click on the Merge Table, double click on the Name attribute. We are merging on the names set the merge key we want to use, we only have one 

Configure groupings on the pipes

1. Double-click on the pipe from CustomerAddresses  to Customer All Details to open its settings.
   1. In the Basic Settings section, set:
      • Name: this pipe is automatically named in.

   2. In the Sort/Group section click  Show Attributes. PhixFlow displays the list of attributes (columns).

   3. Drag CustomerRef from the list into the settings.
   4. Close the list of attributes.
2. Double-click on the pipe from CustomerAddresses to SOURCE_CUSTOMER_PHONE_NUMBERS. This pipe will be called in_2.

   1. In the Sort/Group section, click  Show Attributes. PhixFlow displays a list of the attributes (columns) in the CustomerAddresses  data.

   2. Drag CUSTOMER_REF into the Sort/Group section.
   3. Close the list of attributes.
3. Now add attributes from CustomerAddresses to the merge stream:
   1. Hover over CustomerAddresses and in the pop-up toolbar, click  Show Attributes.

   2. Drag all attributes from this list and drop them onto the stream icon for Customer All Details.

      How to select all the attributes in a list

      To select all attributes, you can:

      • either click the first attribute then Shift+click the last attribute
      • or click the check box next to the Name header. 

      

   3. PhixFlow asks for confirmation. If it is not already selected, select Use original attribute name. Then click the arrow  to confirm.
      
      
4. Similarly, drag all attributes from SOURCE_CUSTOMER_PHONE_NUMBERS to Customer All Details.
5. To see the resulting settings for Customer All Details, double-click its icon to open the settings tab and look at the Attributes section. You can see a list of 8 attributes (data columns).

Run analysis and view the stream data

1. To run analysis on your new stream, hover your mouse pointer over the stream Customer All Details, and in the pop-up toolbar, click  Run Analysis.
2. To view the data you just loaded, hover over the stream and click  Show view. In the drop-down list, select  Default View. You will see the address data you loaded from the file combined with the phone numbers you loaded from the database.