Using Tasks and Task Plans

Owned by Fiona Sargeant (Unlicensed)
Last updated: Jun 22, 2022
4 min read

This topic is for data modellers, application designers and system administrators. The pages in this topic explain how to create and run automated tasks in PhixFlow. It also explains strategies for tasks that affect tables. 

The pages in this topic are:

Overview

When working with data, applications and IT systems, there are routine processes that you need to run. PhixFlow makes it easy for you to set up and manage these processes using tasks and task plans. For example:

  • application designers or system administrators can automatically update data in applications
  • data modellers can run analysis on models or specific tables
  • administrators can clear old information from the PhixFlow system, by archiving data from:
    • record sets
    • processing logs
    • performance statistics.

Use the task plan properties tab to configure Task Plans. Set the options and add individual Tasks.

If you need to temporarily prevent scheduled task plans from running during system maintenance, see Preventing Task Plans From Running.

You can run a task plan:

  • manually, using the property tab toolbar button  Start Task Plan.
  • automatically at a scheduled date and time, using the Scheduled check box and related Schedule section.
  • by linking it to an application action, in the action properties → Run Task Plan section; see Table-Action.
    When the user clicks something in the application, such as a button or a data row, the task plan runs.

When a task plan runs, its tasks are run in the order they are listed.

You can optionally set up a list of users who will receive an email about the outcome of running the task plan.

Structuring Tasks

When setting up task plans with tasks that affect tables, you must consider whether the tables have any dependencies. This affects your options when structuring tasks in task plans.

Multiple tasks each affecting one table

If there are dependencies between tables, configure PhixFlow to have one task per table and configure each task to be mandatory. 

  1. Create a set of tasks.
    • Each task works on a single table.
    • Set the tasks to be mandatory.
  2. Add the set of tasks to the task plan.

PhixFlow processes the tasks, and therefore the tables, in order. PhixFlow must finish a task successfully before it starts to update the next table. This is the safest way to set up tasks that affect tables. Always use this method when there are dependencies between tables.

Example

A model contains connected tables that should be run twice daily to update a model with the latest data from an external database. A scheduled task plan contains:

  • TaskA to run table-a
  • TaskB to run table-b
  • TaskC to run table-c.

PhixFlow will process TaskA until changes to table-a are finished. As this now has the latest data, when TaskB runs, it pulls the new data into table-b. Finally TaskC pulls new data from table-b into table-c.


Or one task affecting multiple tables

If there are no dependencies between tables, you can add multiple tables to the task. In this case you have no control over the order in which tables will be processed. Only use this method if there is you are sure there are no dependencies between the tables.

  1. Create a task
  2. Add multiple tables to a task
  3. Add the task to the task plan. 

This has the advantage that you only need to configure one task to change multiple tables.

Example

Three tables, a, b and c are in different models and have not data dependencies. A task plan to update the table data contains a single task which runs analysis on table-a, table-b and table-c. PhixFlow processes all the tables concurrently.

Mandatory Tasks

The tasks are run in the order they are listed in the the task plan.

You can specify that a task in a task plan is mandatory. This means the task must complete before the next task in the sequence can run. If a mandatory task fails, PhixFlow will not run the following tasks. You must resolve the issue preventing the task from running and then restart the task.

Rerunning a task plan that previously failed means it continues that run from where it failed.

If data related to an previous task has changed this is not reflected in the results of the rerun. 

This means that after you have rerun a previously failed task, consider then running the task again, to ensure all data or processes are up-to-date.

Types of Task

  • Analysis Tasks
    Use an analysis task to run analysis on the table(s) in the task. 

  • Rollback Tasks
    Use a rollback task to effectively undo run analysis on a table.  When you run a rollback task it rolls back all data in each of the listed tables, deleting the record-sets. The list of record-sets is empty and there are no data records in the table. For information about how to rollback tables manually, see Rollback Recordsets.

  • Configuration Export Tasks
    Use a configuration export task to export selected applications and packages or a full configuration. The exported file is saved to the download area; see Using the Download Area. 

  • System Tasks
    Use a system task to perform system-wide housekeeping activities. These include deleting:

    • managed file entries
    • email entries
    • temporary files created by file exporters to send by email
    • data from incomplete record-sets
    • log messages and optionally archiving them
    • from the PhixFlow database:
      • table views for which there is no table
      • tables for which there is no table object, when they are older than the period set in System Configuration → Delete Orphaned Table after Days. 
        Tables become orphaned when their table object is deleted or removed by an import process.
      • temporary tables created during publishing, when they are older than the period set in System Configuration → Delete Temp Tables after Days.
        Temporary tables are created by PhixFlow during the publishing process.

    The system task should be run regularly, for example daily or weekly. When the system task is not run regularly, incomplete record-sets can accumulate. If these are large, or many have accumulated, this can slow down PhixFlow's performance. Depending on your PhixFlow database, queries that have to exempt many incomplete record-sets can reach system limits. This can prevent PhixFlow and its applications from running.

    How long PhixFlow keeps system information, and whether to archive log messages before they are deleted, are controlled by the following parameters in System Configuration:

  • Table-Data-Delete Tasks
    Use a table-data-delete task to delete data records and record-sets that you no longer need in PhixFlow. How long PhixFlow keeps table data is configured in table properties → Data Retention Settings.
    Running a table-data-delete task on a large number of tables, or on a table with a large data set, can take some time. We recommend scheduling these tasks for times when the system is quiet, for example overnight.
    When tables are deleted from the repository, their names are automatically be removed from tasks that refer to them. This means a saved task can become empty.

    Phixflow only runs a table-data-delete task on tables that have Data Retention Settings specified. If no data retention values are set, PhixFlow never deletes the data. table data will accumulate, leading to space and performance issues.

    See also Deleting Old Data.