Analysis Models Overview

Analysis models are the graphical representation of the data processing that will be carried out by PhixFlow.  They Designed to handle large data sets, analysis models define how data is imported, transformed, viewed, analysed and exported. They are made up of modelling objects, each of which performs one or more of these functions Each tasks is performed by a modelling object. For example, a datasource is an object that allows PhixFlow to connect to an external database. A table (formerly known as a stream) allows , and a table which allows you to write functions to manipulate data such as calculating a value.The analysis models are the main back-end process in PhixFlow and run whenever a table is requested. This can be triggered in two

Calling Analysis

Analysis models can be run in one of the following ways:

1. Run analysis: An ad-hoc request from a model, by clicking the 

   Insert excerpt
   _run_analysis _run_analysis nopanel true

    in the table's popup toolbar.
2. Scheduling a task: By a

   Insert excerpt
   _taskplan _taskplan nopanel true

   .
3. Triggered by an ActionFlow: actionflows can call an analysis model. 

As PhixFlow runs the analysis steps, it reports on progress using messages in the model the steps are recorded in the System Console. You can open the system console up at any time by clicking the icon

Analysis Model Window Layout

Image Added

Expand
title More Detail
The Toolbar provides screen specific options, such as displaying existing tables which can be added to the canvas and the option to create new tables or file collectors. The Repository and Property Editors are displayed to the right of the canvas. They provide navigation to locate existing items, the ability to create new items and access to the options for each editable item. The Canvas area where the Analysis Model will be configured. New items, such as Tables and File Collectors, can be added to the canvas by dragging on the associated toolbar icon, existing items are added by dragging the items from the repository onto the canvas.

Steps

Modelling Objects, such as Tables and Datasources appear on the canvas and are connected by pipes. Pipes perform different roles such as allowing data to flow through (solid lines) or performing lookups (dashed line) to retrieve additional information such as looking up a product code and returning its name.

Image Removed

Getting Started

Importing Data

PhixFlow supports a range of methods for importing data including files, emails, databases and APIs. See 2. Importing Data.

Transforming Data

Transforming and enriching data is at the heart of analysis modelling, from looking up reference information to performing fuzzy logic deduplication of customers. There are a host of options and strategies to support them, begin by looking at Transforming Data.

Transformations are performed using a combination of modelling objects, such as tables, and Functions within those objects, such as replaceAll.

Enriching data is also extremely useful, such as structuring unstructured address data into individual address lines or deriving additional information, such deriving asset categories base on information found in a description. See Enriching Data.

Reconciliation can be performed in analysis models, from simple master data checks which provide details of records processed vs records output with supporting information, through to transactional reconciliation where calculation are performed on data to ensure the results of the processed data match the expected result. Reconciliation is particularly useful for data migrations to validate the accuracy of the data moved. See Reconciliation

Lookups enable data to be read from different tables for the purposes of enriching another. For example it could be to check the status of an order. For more information on looking up data from other tables see, Enriching Data → Lookup Information.

Performance

Performance is key when handling large data sets, therefore PhixFlow provides a number of features to assist in the area of performance. These include caching data, in memory lookups and indexing. See Performance and Tuning.

Modelling Objects

Modelling Objects, such as Tables and Datasources appear on the canvas and are connected by pipes. Pipes perform different roles such as allowing data to flow through (solid lines) or performing lookups (dashed line) to retrieve additional information such as looking up a product code and returning its name.

See Analysis Properties.

Candidate Sets

Candidate sets are a fundamental concept of function calculation in PhixFlow.

Every time a function calculation is carried out, all the required input data is brought together and organised into sets of data - one set for each Key group.

The Key groups are worked out using the Pipe Grouping Attributes defined on input Pipe for each table.

Recordsets

Each time you run analysis on a model, PhixFlow creates a new set of data in each table; see Table. A recordset is a collection of data within a table for a given period. Recordsets, and the data they contain, remain in the table until you archive the data (see Task) or manually remove it (see Rollback Recordsets). 

Within analysis models, all data is processed before commencing to the next modelling object. This is in contrast to actionflows which process each record in their entirety before moving to the next one.

Periods

The time period of a table determines how data in the table will be handled. The period is typically set to:

1. Transactional: allows multiple users to run independent analysis tasks at the same time.
2. Variable: generate or collect data since the most recent run of the table to the current date.

Data Range

The data range determines the recordset that will be displayed:

• Latest displays the records from the latest recordset only.
  
• All displays the records from all recordsets.

Rolling back

To clear data the record set must be rolled back, see Rollback Recordsets.

Exports

PhixFlow supports a range of methods for exporting data including files, direct writes to databases and APIs. Details of each can be found here Exporting Data from an Analysis Model

Learn More

      Child pages (Children Display)depth1.

Scheduling

When working with data, applications and IT systems, there are routine processes that you need to run on a schedule. PhixFlow makes it easy for you to set up and manage these processes using task plans, to which you add tasks. See Task Plans.

Example Analysis Model

What Next

PhixFlow Fundamentals, this course provides a practical guide to using PhixFlow including the design and creation of screens in our applications.