Tasks, Task Plans and Archiving

Overview

When working with data, applications and IT systems, there are routine processes that you need to run. PhixFlow makes it easy for you to set up and manage these processes using tasks and task plans. For example:

• application designers or system administrators can automatically update data in applications
• data modelers can run analysis on models or specific streams 
• administrators can clear old information from the PhixFlow system, by archiving data from:
  • stream sets
  • processing logs
  • performance statistics.

Use the task plan properties tab to configure Task Plans. Set the options and add individual Tasks.

You can run a task plan:

• manually, using the property tab toolbar button  Start Task Plan.
• automatically at a scheduled date and time, using the Scheduled check box and related Schedule section.
• by linking it to an application action, in the action properties → Run Task Plan section; see Action.
  When the user clicks something in the application, such as a button or a data row, the task plan runs.

When a task plan runs, its tasks are run in the order they are listed.

You can optionally set up a list of users who will receive an email about the outcome of running the task plan.

Structuring Tasks

When setting up task plans with tasks that affect streams, you must consider whether the streams have any dependencies. This affects your options when structuring tasks in task plans.

Either each task affects one stream

If there are dependencies between streams, configure PhixFlow to have one task per stream and configure each task to be mandatory. 

1. Create a set of tasks.
   • Each task works on a single stream.
   • Set the tasks to be mandatory.
2. Add the set of tasks to the task plan.

PhixFlow processes the tasks, and therefore the streams, in order. PhixFlow must finish a task successfully before it starts to update the next stream. This is the safest way to set up tasks that affect streams. Always use this method when there are dependencies between streams.

Or one task affects multiple streams

If there are no dependencies between streams, you can add multiple streams to the task. In this case you have no control over the order in which streams will be processed. Only use this method if there is you are sure there are no dependencies between the streams.

1. Create a task
2. Add multiple streams to a task
3. Add the task to the task plan. 

This has the advantage that you only need to configure one task to change multiple streams.

Mandatory Tasks

The tasks are run in the order they are listed in the the task plan.

You can specify that a task in a task plan is mandatory This means the task must complete before the next task in the sequence can run. If a mandatory task fails, PhixFlow will not run the following tasks. You must resolve the issue preventing the task from running and then restart the task.

Types of Task

• Analysis Tasks
  Use an analysis task to run analysis on the stream(s) in the task. 
  
  
• Rollback Tasks
  Use a rollback task to effectively undo run analysis on a stream.  When you run a rollback task it rolls back all data in each of the listed streams, deleting the stream-sets. The list of stream sets is empty and there are no data records in the stream. For information about how to rollback streams manually, see Rollback.
  
  

• Configuration Export Tasks
  Use a configuration export task to export selected applications and packages or a full configuration. The exported file is saved to the download area; see Using the Download Area. 
  
  

• System Tasks
  Use a system task to perform system-wide housekeeping activities. These include deleting:

  • managed file entries
  • email entries
  • temporary files created by file exporters to send by email
  • data from incomplete stream sets
  • orphaned stream views from the PhixFlow database
  • log messages and optionally archiving them.

  The system task should be run daily, or at a minimum every week. If incomplete stream sets are not deleted, they can slow down PhixFlow's performance. Depending on your PhixFlow database, queries that have to exempt many incomplete stream sets can reach system limits. This can prevent PhixFlow and its applications from running.

  The archiving periods, and whether to archive log messages before they are deleted, are controlled by the following parameters in System Configuration:

  • Delete Log Messages After Days
  • Archive Deleted Log Messages To File
  • Delete Statistics After Days
  • Delete Emails After Days
  
  
  

• Archive Tasks
  Use an archive task to delete or archive stream data or stream sets that you no longer need in PhixFlow. Archive data is saved into a zip file - to get at this? You can (in theory) reinstate archive data. Whether or not PhixFlow deletes or archives stream sets depends on how the archive settings in the stream are configured. 
  Running an archive task on a large number of streams, or on a stream with a large data set can take some time. We recommend scheduling archiving tasks for times when the system is quiet, for example overnight.
  

  Phixflow can only run an archive task on streams that have Archive Settings specified. If a stream does not have any archive settings, PhixFlow never deletes or archives the data. Stream sets will accumulate, leading to space and performance issues.

Archiving Data

Archiving Specific Streams

You can create an archiving task for a specific stream or streams. PhixFlow creates a single zip file containing the specified streams.  For example, you might want to create a single archive for:

• data from several related streams
• a stream that contains a large amount of data  
• a stream that you want to be able to restore from the backup. This is easier when the data is in its own zip.

Archiving Any Stream

You can create a single archive task to delete or archive the data in streams:

• that are not in another archive task
• and have archiving settings configured.

To do this, select the Archive All option

When you run the task, PhixFlow:

• deletes stream sets that are older than the specified archive settings in the stream
• creates a zip archive of all the stream sets within the specified archive settings in the stream.
  This is a single archive containing all the data from all the streams processed.

Archive tasks must have:

• either at least one stream added
• or have the Archive All option ticked.

If streams are deleted from the repository, their names will automatically be removed from tasks that refer to them. This means a saved task can become empty/redundant. PhixFlow will no longer run the task - log mesages??

Finding the Archive file

When a stream is archived, PhixFlow logs information in the system console →  Archive Log. This includes information about the data archived and any archive file created. This is a single archive containing all the data from the streams processed.. Optionally, the archived data can be saved to a file. If this is done, the data can reloaded into PhixFlow using Restore Archive.