Introduction

Merges can be used to consolidate records from two or more tables. In this page we will work through a simple merge scenario to explain how merges work. All resources used in this example are available here:

Scenario

We have two sets of company information, Businesses 1.xlsx and Businesses 2.xlsx, that we want to merge together and remove duplicates. There are a number of challenges we must resolve to achieve this:

The attribute names in each file are different.
There are duplicate companies that exist in both files.
There are duplicates within the Businesses 2.xlsx file.

Solution

Step 1 Import the data

Using a File Collector, import each of the Businesses Excel documents into their own table and run analysis on each to populate them with data.
Make both of the tables static using the Static option from the table hover over toolbar.
This ensures we only read the files once.

Step 2 Create the Merge Table

In the analysis toolbar, click Table
Drag the Merge Table onto your model.
In the settings, set the Name for example Businesses Merged
The attributes can be created manually using the plus button in the Attributes section. Here we will use the attributes from the Businesses 1 table to automatically setup our merge attributes.
1. Hover over the Businesses 1 table and from the hover over toolbar click Show or Hide Attributes.
2. Tick all of the attributes and then drag them into the attributes section of the Merge Table.
3. If PhixFlow asks for confirmation. Select Use original attribute name. Then click the arrow to confirm.
Click Apply and Close to save and close the settings for the merge table.

Step 3 Connect the Inputs

In the analysis model, hover over the Businesses 1 table.
From the hover over toolbar, select Connector, then click the end of the arrow onto the merge table. This connector will pipe data into the merge table.
A properties window opens for the pipe, set the name to B1 to better indicate the use of the pipe.
In the Sort/Group section of the properties click Show Attributes and PhixFlow displays the list of attributes (columns).
Drag the Name attribute into the Sort/Group section. This defines the attribute that will be used to merge on, we call this the key. Keys are listed in order and we reference them by number, therefore the keys must appear in the same order on all pipes leading into the merge table. This means attributes we are merging on do not need to have the same name.
Similarly, add a connector from the table Businesses 2 to the merge table and add the Company Name to the Sort/Group.
1. Businesses 2 has duplicates within its data, we can remove these and decide which record to keep. Here we will use the most recent Onboarded Date to decide which record to keep.
2. Drag the Onboarded Date attribute into the Sort/Group section.
3. Double-click on the attribute and untick Group and set Direction to Z-A. By unticking group we are telling PhixFlow to sort by this attribute but omit it from grouping.
4. PhixFlow will group and sort on the Company name , and use the Onboarded Date to also sort the groups. We need to select the top record from the group, and we do this by setting the Maximum Number of Records Per Group to 1. This will ensure we get the most recently onboarded business. The configuration should look like this:

Step 4 Configure the Merge Table

We now need to tell the Merge Table how to populate its attributes.
Click on the Merge Table, double click on the Name attribute. We are merging on the first merge key, the name of the business, enter _key[1] in the expression:

If you were merging on a second attribute enter _key[2], and this can be repeated for any number of attributes.
For the Address1 attribute, we can decide on which source will have the best data to create a merged record from. Here we will let Businesses 1 take precedence. Double click on Address 1 and enter ifNull(B1.Address1, B2.AddressLine1). This tell PhixFlow to use the Address from the Businesses 1, but if it is null then take it from Businesses 2.
Repeat these steps for the remaining attributes and your configuration will look like this:

Step 5 Run Analysis

Run analysis to populate the Merge Table.
Hover over the merge table and click Create Table Using Attributes → Default View.

Troubleshooting

How do I clear the data?
To remove data from a table see Removing Recordsets.
Why is there no data when I run my import?
When you run an import the file you uploaded will be archived. If you run the import a second time without uploading a new file there is nothing to import. Upload your file again and run analysis on the table.
PhixFlow says it can't find the attribute.
In the merge table ensure the values in your expressions match the names of the attributes coming from your input tables.

Browser not supported