Combining Data (Union)
Scenario
Where we have two sources of data that share the same structure, i.e. the same attribute names, we can combine these into a single table without the need to perform a merge. The intention is that we will keep all records, so for every record passed in, a record is passed out.
This technique is useful when performing a data migration that consolidates data from 3 separate systems that all follow the same data structure.
Solution
It is assumed that all data sources are imported into their own tables in an Analysis Model. We will refer to all tables that are due to be combined as input tables and the table where the data is being combined as the output table.
- From the Table drop down on the analysis toolbar, click and drag a Calculate table on your model
- In the Properties, set the Name, e.g.
Output Table
- In the Properties, set the Name, e.g.
For each of the input tables, from its hover over toolbar, select Connector, then click the end of the arrow onto the output table
Hover over the first input table and from the popup toolbar click
- In the Attributes window, tick all of the attributes and drag them onto the output table
- Select Use original attribute name
- Click Confirm
- This automatically creates the attributes for us in the output table
- Close the Attributes window
- We now need to set up the attributes to pull information from all input tables
- Click on the output table and Pin the tab to keep it open
- In the Attributes section, double click the first attribute
- Remove the pipe prefix, typically this will be "in."
- For example,
in.CompanyName
becomesCompanyName
. - By doing this we are telling PhixFlow to look for this attribute name regardless of which pipe it is coming from
- For example,
- Repeat this for all of your attributes, the configuration is illustrated below:
- f
- Click Apply and Close to save and close the settings for the output table