Insert excerpt_Banners_Bannersnameanalysisnopaneltrue

Overview

Streams are a key model component. They represent a structured store of data within PhixFlow receiving data from one or more components, processing it then storing it.

Insert excerpt_standard_settings_standard_settingsnopaneltrueThe tab toolbar has the following additional buttons: Insert excerpt_record_sets_show_record_sets_shownopaneltrue, Insert excerpt_view_show_view_shownopaneltrue, Insert excerpt_run_analysis_run_analysisnopaneltrue.

When you make changes to streams and stream attributes, PhixFlow automatically publishes them to the stream data tables in the database; see Publishing Streams.

Stream Properties

Basic Settings

FieldDescriptionNameThe name of the stream.Model Validation Status Insert excerpt_validate_validatenamestatusnopaneltrueEnabled Insert excerpt_check_box_tick_check_box_ticknopaneltrue to include the steam during analysis runs.Static Data Insert excerpt_check_box_tick_check_box_ticknopaneltrue to set the stream to hold static data. Static data is reference or "look up" data that is used as part of a stream calculation.

Updating a static stream

In general, static streams are not updated during an analysis run that is initiated on another stream in the model. The exception is when an incoming push pipe is triggered, as this will force the stream to update itself.

Instead, to update the data in a static stream you must run analysis directly on the stream via:

the model
a task plan.

Period

Select the time period for the stream:

Excerpt
Transactional: allows multiple users to run independent analysis tasks at the same time. Non-transactional Daily: generate or collect data every day. Monthly: generate or collect data every month. Variable: generate or collect data since the more recent run of the stream to the current date.

As transactional streams need a UID attribute, PhixFlow automatically creates this stream attribute if it does not already exist.

Default Data Range

This option affects the following items for which this is the backing stream:

stream views
dashboards
card layout components.

Insert excerpt_data_range_data_rangenopaneltrue

If no option is is selected, PhixFlow assumes the default data range is All.

You can override this setting using the options Use Custom Data Range and Data Range available:

on a Screen.
on a card component; see Component.

You cannot override this setting for a stream view.

Stream Type

Select the stream type. The type indicates the function that operates on the data.

Aggregate Stream
Calculate Stream
CalculateBySet Stream
Cartesian Stream
Merge Stream

Supersede Items on Pipe

Select a loop pipe from the list of those available.

A loop pipe links the stream back to itself. It compares new data records to the existing data records. If there is a repeated record PhixFlow uses the new one in the stream and marks the old one as superseded.

Audit Manual Changes

This check box is available when the Period is Transactional.

Insert excerpt_check_box_tick_check_box_ticknopaneltrue to update and delete initiated by stream actions (not those carried out by analysis runs) will automatically mark the existing record as superseded and create a new streamset. The new versions of the updated records will be placed in the new streamset. Inserts will simply create a new streamset, and add the inserted record into that streamset.

When Audit Manual Changes is first set, the attributes UpdateAction, UpdatedByName, UpdatedByID and UpdatedTime will be created if they do not already exist. If you do not require these attributes, delete them. For the UpdatedByName attribute, PhixFlow creates a field of 250 characters. (In versions up to 8.0.4, the field was 50 characters long.)

UpdateAction must be set to the type of action, such as INSERT, UPDATE or DELETE. The other attributes will be populated if they exist on the stream:

UpdatedByName - the name of the user that performed the update,
UpdatedByID - the internal id of the user that performed the update
UpdatedTime - the date and time the update was made

Attributes

The grid contains a list of the stream attributes in the stream. This section has a toolbar with standard buttons and Insert excerpt_table_show_table_shownopaneltrue and Insert excerpt_file_collector_show_file_collector_shownopaneltrue.

The grid lists the attributes (data columns) in the stream data. To edit the properties of an attribute, double-click the attribute name to open the Attribute properties.

To remove an attribute from the stream, select the attribute and click Insert excerpt_delete_deletenopaneltrue in the toolbar.To add a new attribute to the list, click Insert excerpt_new_newnopaneltrue to open a new Attribute properties tab.

To add an existing attribute to the list:

Click Insert excerpt_table_show_table_shownopaneltrue to list the streams available in the repository.

Expand a stream to show its attributes.

Drag an attribute into this list to add it.

To edit only the expression:

Right-click an attribute name to display the context menu.
Select Edit the expression field.
PhixFlow opens a simple text editor box
Make changes to the attributes expression.
Click Image Removed to save your changes.

AnchormultipliersmultipliersMultipliers and FiltersFieldDescriptionInput MultiplierThe input multiplier expression should evaluate to a list of one or more values. For each value in the list, the internal variable _inputMultiplier will be set to that value and the whole stream processing will be repeated i.e. the pull pipes will be read and the data from those pipes processed to generate output stream items to be added to the current streamset. For example :

do ( $aRange = [], addElement($aRange, rng.RangeFrom), addElement($aRange, rng.RangeTo), $bRange = [], addElement($bRange, $aRange), $bRange )

Where rng.RangeFrom = 500 and rng.RangeTo = 1000, the above example evaluates to [[500,1000]], which is a list containing 1 element, which is itself a list containing 2 elements. An Input Multiplier that evaluates to [3,4,7,8] would run the Stream 4 times.
Because Input Multipliers are evaluated first in the PhixFlow Timing Cycle they are often used to look up values that can be passed to Database or other Collectors.Log Input Multiplier Messages Insert excerpt_check_box_tick_check_box_ticknopaneltrue to cause PhixFlow to write a message to the log every time an Input Multiplier value changes during an analysis run.Output MultiplierThis field is an expression which should evaluate to an array of values. A separate output record will be produced for each value in the array and this value is available as _outputMultiplier in each of the stream output attribute expressions (each value in the array is also available through _type, although this is not recommended usage). In effect this will multiply each of the output records by the number of elements from the returned list.
For example :

ifNull(in.ASSET, [1,10,12] , // else do [5,7] )

will create 3 records for every record in the stream if in.ASSET contains a value (setting _type = 1, 10 and 12 in each case). Otherwise it will create 2 records for every record in the stream (and set _type = 5 and 7). An Output Multiplier may also evaluate to a record, or a group of records. For example an Output Multiplier with the expression:

do( lookup(lkin, $num = in.BNumber), lkin )

will return a list of records which match the lookup on the lkin pipe. In this case the required data can be extracted from the Output Multiplier using the following expression :

do ( $values = _type, $values.account_num )

If the output multiplier expression evaluates to _NULL, an empty list of values or an empty list of records then a single output record will be produced with _type set to _NULL, _NULL or an empty record respectively.

Select Records to Output

You can select a subset of records to output to the next modelling object. You select records using the data in any attribute that is in the stream output. Enter an expression that should evaluate to true or false (equivalently 1 or 0).

When the expression evaluates to true or 1, the records found by the expression are selected and output to the next modelling object.

For example:

A stream containing data about accounts has an attribute called "active".
You only want to include "active" accounts in the following streams.
Active accounts have the value 1.

To select active accounts, enter the expression _out.active.
All records that have the active attribute set to any other value are ignored.

Actions

This section has a toolbar with standard buttons. The grid contains a list of actions that operate on the stream.

To edit the properties of an action, double-click its name. To add an action to the list, click Insert excerpt_new_newnopaneltrue. PhixFlow opens the Action properties tab.To remove an action from the stream, select the action and click

Insert excerpt

	_Banners
	_Banners
name	analysis
nopanel	true

Available when Period is Transactional.

Insert excerpt_check_box_tick_check_box_ticknopaneltrue to ensure only a single streamset can be generated at a time even if the stream receives several concurrent requests to generate data.

This can be useful where you want to make sure that two analysis runs do not attempt to update the same records at the same time e.g. as a result of two people selecting the same records in a view and then hitting the same action button at the same time to process those records.

Run Alone Insert excerpt_check_box_tick_check_box_ticknopaneltrue so that, whenever the analysis engine needs to generate data for this stream, it will first wait for all running tasks to complete before it starts.

Any additional analysis tasks submitted while this stream is waiting to start, or while it is generating data, will wait until this stream has completed its analysis before they start.

Key TolerancesWhen building a candidate set, data can be grouped together using a specific key value, e.g. Account Number. If the key value is a number, enter a key tolerance to identify numbers within the given tolerance as the same key value.Default ViewSelect the default view for the stream; see Views for details of creating views on streams.Last Run Date (Read only)PhixFlow displays the date and time that analysis was last run for this stream. This date is taken from the "to date" of the most recent streamset for this stream.Last Run ByPhixFlow displays the user that last ran analysis on this stream.

Access Permissions

By default, the permissions to access stream data is set in System Configuration → Allow access to data by default.

You can add specific user groups to restrict access to the stream; see Common Properties → Access Permissions.

Insert excerpt_audit_auditnopaneltrue

Overview

Tables are a key model component. They represent a structured store of data within PhixFlow receiving data from one or more components, processing it then storing it. When you make changes to tables and attributes, PhixFlow automatically publishes them to the database; see Publishing Tables.

Excerpt

name	FileTable

For

Insert excerpt

	_tableFile
	_tableFile
nopanel	true

s, see File Table.

Insert excerpt

	_property_toolbar
	_property_toolbar
nopanel	true

The toolbar has the following additional buttons:

Insert excerpt

	_recordsets_show
	_recordsets_show
nopanel	true

,

Insert excerpt

	_view_show
	_view_show
nopanel	true

,

Insert excerpt

	_run_analysis
	_run_analysis
nopanel	true

.

Insert excerpt
_property_tabs
_property_tabs
name basic-h
nopanel true

Insert excerpt

	_parent
	_parent
nopanel	true

Basic Settings

Field	Description
Name	The name of the table.
Description	We recommend that you always enter a description to explain the purpose of this item.

Attributes

The grid lists the attributes (data columns) in the table data.

Excerpt

name	Attributes

To add a new attribute to the list, click the
Insert excerpt
_add_icon
_add_icon
nopanel true
icon
To edit the properties of an attribute, double-click the attribute name to open the Attribute properties
To remove an attribute from the table, right-click the attribute and select
Insert excerpt
_delete
_delete
nopanel true
To add an attribute from another table, open the table and drag the attributes from one table to the other

Excerpt

name	Views

Views

The grid contains a list of the data views for this table.

To edit the properties of a view, double-click its name. To add a view to the list, click
Insert excerpt
_new
_new
nopanel true
. PhixFlow opens the View properties tab.
To remove a view from the table, select the view and click
Insert excerpt
_delete
_delete
nopanel true
in the toolbar.

Anchor
multipliers
multipliers
Analysis Options

Field Description

Enabled

Insert excerpt

	_check_box_tick
	_check_box_tick
nopanel	true

to include the table during analysis runs.

Static Data

Insert excerpt

	_check_box_tick
	_check_box_tick
nopanel	true

to set the table to hold static data. Static data is reference or "look up" data that is used as part of a table calculation.

Updating a static table

In general, static tables are not updated during an analysis run that is initiated on another table in the model. The exception is when an incoming push pipe is triggered, as this will force the table to update itself.

Instead, to update the data in a static table you must run analysis directly on the table via:

the model
a task plan.

Period

Select the time period for the table:

Excerpt

name	Period

Transactional

This allows multiple users to run independent analysis tasks at the same time.

As transactional tables need a UID attribute, PhixFlow automatically creates this attribute if it does not already exist.

Daily

This is a non-transactional or periodic table type, and generates or collects data every day.

Monthly

This is a non-transactional or periodic table type, and generates or collects data every month.

Variable

This is a non-transactional or periodic table type, and generates or collects data since the more recent run of the table to the current date.

Default Data Range

This option affects the following items for which this is the backing table:

views
screens
card components.

Insert excerpt

	_data_range
	_data_range
nopanel	true

If no option is is selected, PhixFlow assumes the default data range is All.

You can override this setting using the options Use Custom Data Range and Data Range available:

on a Screen.
on a card; see Component.

You cannot override this setting for a view.

Table Type

Select the table type; see Types of Table and When to Use Them. The type indicates the function that operates on the data.

Aggregate Table
Calculate Table
CalculateBySet Table
Cartesian Table
Merge Table

Supersede Items on Pipe

Select a loop pipe from the list of those available.

A loop pipe links the table back to itself. It compares new data records to the existing data records. If there is a repeated record PhixFlow uses the new one in the table and marks the old one as superseded.

Input Multiplier

The input multiplier expression should evaluate to a list of one or more values. For each value in the list, the internal variable _inputMultiplier will be set to that value and the whole table processing will be repeated i.e. the pull pipes will be read and the data from those pipes processed to generate output records to be added to the current recordset. For example :

do ( $aRange = [], addElement($aRange, rng.RangeFrom), addElement($aRange, rng.RangeTo), $bRange = [], addElement($bRange, $aRange), $bRange )

Where rng.RangeFrom = 500 and rng.RangeTo = 1000, the above example evaluates to [[500,1000]], which is a list containing 1 element, which is itself a list containing 2 elements. An input multiplier that evaluates to [3,4,7,8] would run the Table 4 times.
Because Input Multipliers are evaluated first in the PhixFlow Timing Cycle they are often used to look up values that can be passed to Database or other Collectors.

See Expressions and PhixScripts.

Log Input Multiplier Messages

Insert excerpt

	_check_box_tick
	_check_box_tick
nopanel	true

to cause PhixFlow to write a message to the log every time an Input Multiplier value changes during an analysis run.

Output Multiplier

This field is an expression which should evaluate to an array of values. A separate output record will be produced for each value in the array and this value is available as _outputMultiplier in each of the table output attribute expressions (each value in the array is also available through _type, although this is not recommended usage). In effect this will multiply each of the output records by the number of elements from the returned list.
For example :

ifNull(in.ASSET, [1,10,12] , // else do [5,7] )

will create 3 records for every record in the table if in.ASSET contains a value (setting _type = 1, 10 and 12 in each case). Otherwise it will create 2 records for every record in the table (and set _type = 5 and 7). An output multiplier may also evaluate to a record, or a group of records. For example an Output Multiplier with the expression:

do( lookup(lkin, $num = in.BNumber), lkin )

will return a list of records which match the lookup on the lkin pipe. In this case the required data can be extracted from the Output Multiplier using the following expression :

do ( $values = _type, $values.account_num )

If the output multiplier expression evaluates to _NULL, an empty list of values or an empty list of records then a single output record will be produced with _type set to _NULL, _NULL or an empty record respectively.

See Expressions and PhixScripts.

Select Records to Output

(Formerly Output Filter)

You can select a subset of records to output to the next modelling object. You select records using the data in any attribute that is in the table output. Enter an expression that should evaluate to true or false (equivalently 1 or 0).

When the expression evaluates to true or 1, the records found by the expression are selected and output to the next modelling object.

For example:

A table containing data about accounts has an attribute called "active".
You only want to include "active" accounts in the following tables.
Active accounts have the value 1.

To select active accounts, enter the expression _out.active.
All records that have the active attribute set to any other value are ignored.

See Expressions and PhixScripts.

Actions

The grid contains a list of actions that operate on the table.

To edit the properties of an action, double-click its name
To add an action to the list, click
Insert excerpt
_new
_new
nopanel true
and PhixFlow opens the Table-Action properties tab
To remove an action from the table, select the action and click
Insert excerpt
_delete
_delete
nopanel true
in the toolbar

Record-Actions

The grid contains a list of record-actions that can update records in the table.

To edit the properties of a record-action, double-click its name
To add a record-action to the list, click
Insert excerpt
_new
_new
nopanel true
and PhixFlow opens the Record-Action properties tab
To remove a record-action from the table, select the record-action and click
Insert excerpt
_delete
_delete
nopanel true
in the toolbar

Excerpt

name	SortOrders

Sort Orders

The grid contains a list of the sort orders for this table.

To edit the properties of a sort order, double-click its name
To add a sort order to the list, click
Insert excerpt
_new
_new
nopanel true
and PhixFlow opens the Sort Order properties
To remove a sort order from the table, select the sort order and click
Insert excerpt
_delete
_delete
nopanel true
in the toolbar

Excerpt

name	Errors

Errors
Anchor
error
error

Insert excerpt

	_validate
	_validate
name	error
nopanel	true

Excerpt

name	ERDs

ERDs

The grid displays a read-only list of the Entity Relationship Diagrams (ERDs) that this table is part of. Double-click an ERD to open it. See Entity Relationship Diagrams.

Excerpt

name	Relations

Relations

The grid displays a read-only list of the relationships that this table has to attributes in other tables. Double-click a relationship to display it's properties; see Relationship and Entity Relationship Diagrams.

Excerpt

name	Filters

Filters

The grid contains a list of the filters that can be applied to this table. You can add filters to view based on this table.

All filters in this list are available in the default view for this table. For views that you create, in the View properties tab → Filters section, you can add filters. The list of available filters is populated by the filters defined for the table.

To edit the properties of a filter, double-click its name. To add a filter to the list, click

Insert excerpt

	_new
	_new
nopanel	true

. PhixFlow opens the Filter properties tab.

To remove a filter from the table, select the filter and click

Insert excerpt

	_delete
	_delete
nopanel	true

in the toolbar.

Tip
In a view grid, the header row has a drop-down list of filters. The drop-down list is populated by the filters that are selected on the view. The filters must first be defined for the table here.

Inputs

This section lists all pipes that connect into the table. Double-click a pipe to view more details and make changes. See Pipe.

This section has a toolbar with

Insert excerpt

	_detailed_list
	_detailed_list
nopanel	true

and

Insert excerpt

	_refresh
	_refresh
nopanel	true

buttons; see standard buttons, see Menu Options and Toolbar Buttons.

Note

This list can include pipes that have no input. This occurs if the source table has been deleted or if a model has been moved to a different PhixFlow instance (export/import), leaving behind a referenced table. Any pipes with no input are highlighted in yellow. To resolve pipes with no input you can:

recreate the missing table
import the missing table
keep the connection, if it will be restored when the model is moved to a different PhixFlow instance
delete the pipe, if it is no longer required.

Download Limits
Anchor
download_limits
download_limits

The following properties affect the number of data records a user can download from a view, such as a grid, chart or card; see Download CSV or Excel Data.

Field

Description

Table Download Limit

Enter the maximum number of records that a user can download from a view on this table. If no value is set, PhixFlow uses the default limit set in System Configuration→ System Tuning → Default Download File Limit.

Allow Unlimited Downloads

Insert excerpt

	_check_box_untick
	_check_box_untick
nopanel	true

to restrict all users to the Table Download Limit.

Insert excerpt

	_check_box_tick
	_check_box_tick
nopanel	true

to display a User Groups section where you can specify user groups whose members can download unlimited data from a view on this table. For example, members of the Administrators user group may need to download all data.

User Groups

Available when Allow Unlimited Downloads is ticked.

Use this section to select the user groups whose members can download an unlimited number of records.

This section has a toolbar with standard buttons, see Menu Options and Toolbar Buttons, and

Insert excerpt

	_user_group
	_user_group
nopanel	true

.

The grid contains a list of user groups that are exempt from the Table Download Limit. To add a user group to the list:

Click
Insert excerpt
_user_group
_user_group
nopanel true
to list all the user groups in the repository.
Drag user groups into this list to add them to the role.

To remove user groups, use the

Insert excerpt

	_delete
	_delete
nopanel	true

in the toolbar.

Stream Item Actions

This section has a toolbar with standard buttons. The grid contains a list of stream item actions that can update records in the stream.

To edit the properties of a stream item action, double-click its name. To add a stream item action to the list, click Insert excerpt_new_newnopaneltrue. PhixFlow opens the Record-Action properties tab.To remove a stream item action from the stream, select the stream item action and click Insert excerpt_delete_deletenopaneltrue in the toolbar.

Views

This section has a toolbar with standard buttons. The grid contains a list of the data views for this stream.

To edit the properties of an stream view, double-click its name. To add a stream view to the list, click Insert excerpt_new_newnopaneltrue. PhixFlow opens the View properties tab.To remove a view from the stream, select the view and click Insert excerpt_delete_deletenopaneltrue in the toolbar.

Sort Orders

This section has a toolbar with standard buttons. The grid contains a list of the sort orders for this stream.

To edit the properties of a sort order, double-click its name. To add a sort order to the list, click Insert excerpt_new_newnopaneltrue. PhixFlow opens the Sort Order properties tab.To remove a sort order from the stream, select the sort order and click Insert excerpt_delete_deletenopaneltrue in the toolbar.Errors Anchorerrorerror Insert excerpt_validate_validatenameerrornopaneltrue

Relations

This section has a toolbar with standard buttons. The grid displays a read-only list of the relationships that this stream has. Double-click a relationship to display it's properties; see Relationship.

Filters

This section has a toolbar with standard buttons. The grid contains a list of the filters that can be applied to this stream and can be added to this stream's views.

All filters in this list are available in the default view for this stream. For stream views that you create, in the View properties tab → Filters section, you can add filters. The list of available filters is populated by the filters defined for the stream.

To edit the properties of a filter, double-click its name. To add a filter to the list, click Insert excerpt_new_newnopaneltrue. PhixFlow opens the Filter properties tab.To remove a filter from the stream, select the filter and click Insert excerpt_delete_deletenopaneltrue in the toolbar.

Tip
In a stream view grid, the header row has a drop-down list of filters. The drop-down list is populated by the filters that are selected on the stream view. The filters must first be defined for the stream here.

Inputs

This section has a toolbar with Insert excerpt_detailed_list_detailed_listnopaneltrue and Insert excerpt_refresh_refreshnopaneltrue buttons; see standard buttons. In the grid, PhixFlow lists all pipes that connect into the stream. You cannot edit this grid.

Note

This list can include pipes that have no input. This occurs if the source stream has been deleted or if a model has been moved to a different PhixFlow instance (export/import), leaving behind a referenced stream. Any pipes with no input are highlighted in yellow. To resolve pipes with no input you can:

recreate the missing stream
import the missing stream
keep the connection, if it will be restored when the model is moved to a different PhixFlow instance
delete the pipe, if it is no longer required.

Download Limits Anchordownload_limitsdownload_limits

The following properties affect the number of data records a user can download from a stream view, such as a grid, chart or card; see Download CSV or Excel Data.

FieldDescriptionStream Download LimitEnter the maximum number of records that a user can download from a stream view or card view on this stream. If no value is set, PhixFlow uses the default limit set in System Configuration→ System Tuning → Default Download File Limit.Allow Unlimited Downloads Insert excerpt_check_box_untick_check_box_unticknopaneltrue to restrict all users to the Stream Download Limit. Insert excerpt_check_box_tick_check_box_ticknopaneltrue to display a User Groups section where you can specify user groups whose members can download unlimited data from a stream view or card view on this stream. For example, members of the Administrators user group may need to download all data.User Groups

Available when Allow Unlimited Downloads is ticked.

Use this section to select the user groups whose members can download an unlimited number of records.

This section has a toolbar with standard buttons and Insert excerpt_user_group_user_groupnopaneltrue.

The grid contains a list of user groups that are exempt from the Stream Download Limit. To add a user group to the list:

Click Insert excerpt_user_group_user_groupnopaneltrue to list all the user groups in the repository.

Drag user groups into this list to add them to the role.

To remove user groups, use the Insert excerpt_delete_deletenopaneltrue in the toolbar.Data Retention Settings Anchordata-retaindata-retain

Use the following options to specify how PhixFlow manages old data when a stream-data-delete task runs.

Warning

We recommend that you always specify the data retention values for every stream you create.
PhixFlow checks that a stream has:

either data retention settings (recommended)
or Retain All Data selected.

If no data retention option is set, PhixFlow displays an error in the Errors section, above.

Using Tasks and Task Plans for information about tasks.
Deleting Old Data to see how you can use these values in data management strategies.

FieldDescriptionWhat Data is Deleted

Retain All Data

Insert excerpt_check_box_untick_check_box_unticknopaneltrue to specify the criteria for keeping data in the Keep... properties below (recommended). Insert excerpt_check_box_tick_check_box_ticknopaneltrue to allow old data to accumulate in PhixFlow.

Warning
Streams with Retain All Data ticked never have their data deleted and the data will accumulate. This can lead to performance issues.

Keep for N Days

AnchorkeepForXDayskeepForXDaysEnter the number of days to keep data in the stream.

The age of data in a streamset is its 'to' date, relative to the 'to' date of the newest valid streamset in the stream.

When a stream-data-delete task runs for a stream and:

Keep for X Days only is set, PhixFlow deletes stream data that is older than N days.
Keep for Y Streamsets only is set, PhixFlow keeps the most recent X streamsets and deletes streamsets that exceed X.
both Keep for X Days and Keep for Y Streamsets are set, PhixFlow deletes the stream data that meets both conditions.
neither Keep for N Days nor Keep for X Streamsets are set, PhixFlow keeps stream data indefinitely. This can lead to performance issues.

Keep for X Streamsets

The number of streamsets to keep in the stream.

Keep Superseded for N Days

AnchorkeepSupersededForXDayskeepSupersededForXDaysAvailable when Track Superseded Data is selected.

Enter the number of days to keep superseded records in the stream data.

Tip

Remember to keep superseded records for as long as you want the option to roll back the data.

For example, when Keep Superseded for N Days is set to 4, you can run roll back on data that was superseded up to 4 days ago.

A stream-data-delete task removes superseded records older than 4 days, so PhixFlow no longer has the data to be able to roll it back.

In a stream where the superseded date is tracked, the stream data contains a mixture of active records and records that have been superseded.

When a stream-data-delete task runs for a stream and:

Keep Superseded for N Days only is set, PhixFlow deletes data records that were superseded more than N days ago.
Keep Superseded for X Streamsets only is set, PhixFlow deletes superseded records from the streamsets that exceed X.
both Keep Superseded for N Days and Keep Superseded for X Streamsets are set, PhixFlow deletes the superseded records that meet both conditions.
neither Keep Superseded for N Days nor Keep Superseded for X Streamsets are set, PhixFlow keeps superseded records indefinitely. This can lead to performance issues.

Keep Superseded for X Streamsets

Available when Track Superseded Data is selected.

Enter the number of recent streamsets that will retain their superseded data records.

Apply Delete Filter

Image Removed: Insert excerpt_check_box_tick_check_box_ticknopaneltrue to open a filter tab where you can set filter conditions; see Filter. PhixFlow applies this filter when it runs a stream-data-delete task. Only the records which match the filter are deleted.

Store Stream Views

This section has a toolbar with standard buttons. In the grid PhixFlow displays a list of the stream views to store for this stream.

Analysis Models

A list of the analysis models that this stream appears on.

Advanced

Only change the advanced properties with advice by PhixFlow support.

FieldDescriptionAdvanced PropertiesOnly use this field with advice from PhixFlow support. Database View

Use this property if your database administrator wants to query stream data in the PhixFlow database.

Enter a string in the format v_xxxx. When the stream is published to the database, PhixFlow creates a stream view. If the stream is deleted, the next system task that runs will clear the associated stream view from the database.

By default, this field is blank and no stream view is created in the database.

Storage OptionsIndex Scheme Anchorindex_schemeindex_schemeSelect an option to determine how indexes on the stream are organised.

All: indexes on the stream are optimised for selecting from all streamsets (non-historied reads).
Latest: indexes on the stream are optimised for selecting from the latest streamset (i.e. for historied reads).
Superseded: indexes on the stream are optimised for self-updating streams which have a mostly superseded records.
None: no indexes are created on the stream.
Partitioned: this index scheme is specific to partitioned streams; see Storage Type below. Partitioned streams must use this index scheme, and only partitioned streams can use it.

Storage TypeSelect an option to specify how data for the stream should be stored:

Database: Store the data in a regular table within the PhixFlow database. This is the most common option
Database (Partitioned): Store the data in a partitioned table within the PhixFlow database. This option provides improved performance for rollback and retaining very large streamsets. The option is only available if "partitioning" is available within your database installation.
In Memory: Data for the Stream will not be written to the database. This option can be used (for example) when you want to aggregate large amounts of unsorted data which can then be written to a stored Stream.

Data Generation Options Anchordata-gendata-genStart DateEnter the date on which this stream starts or click Insert excerpt_calendar_calendarnopaneltrue to select a date. Data will be populated into the stream from this date onwards.Primary Key Generator

This option relates to an actionflow with a save node that inserts a new stream-item/record into a stream/table. Each new record requires a unique value for the record's primary key. Use this option to specify how PhixFlow generates the primary key value.

Internal: PhixFlow generates unique primary key values automatically. This is equivalent to setting the stream attribute expression to _NULL. Check the stream attr expression doc
Sequence: display the Primary Key Generator Sequence option, in which you can select a sequence of values.

Primary Key Generator Sequence

Available when Primary Key Generator is Sequence.

Select a sequence to assign a new value to the primary key attribute of a new stream-item/record; see Sequence.

new stream item takes the next primary key value in a sequence. This is equivalent to setting the stream attribute expression to nextValue("sequenceName".
Sequence

In Memory Cache SizeEnter the size of the cache that will be maintained when you are using an in-memory stream. See Manage Unsorted Data Using an In-Memory Stream.Allow Partial Set Processing Insert excerpt_check_box_untick_check_box_unticknopaneltrue to restrict all users to the Stream Download Limit. Insert excerpt_check_box_tick_check_box_ticknopaneltrue so that, when analysis reaches the end of a buffer block, it submits the candidate set for processing, even if the next buffer block has a different key.Prevent Parallel Processing

Excerpt

name	DataRetentionSettings

Data Retention Settings
Anchor
data-retain
data-retain

Use the following options to specify how PhixFlow manages old data when a table-data-delete task runs.

Warning

We recommend that you always specify the data retention values for every table you create.
PhixFlow checks that a table has:

either data retention settings (recommended)
or Retain All Data selected.

If no data retention option is set, PhixFlow displays an error in the Errors section, above.

Using Tasks and Task Plans for information about tasks.
Deleting Old Data to see how you can use these values in data management strategies.

Field

Description

What Data is Deleted

Retain All Data

Insert excerpt

	_check_box_untick
	_check_box_untick
nopanel	true

to specify the criteria for keeping data in the Keep... properties below (recommended).

Insert excerpt

	_check_box_tick
	_check_box_tick
nopanel	true

to allow old data to accumulate in PhixFlow.

Warning
Tables with Retain All Data ticked never have their data deleted and the data will accumulate. This can lead to performance issues.

Keep for N Days

Anchor

	keepForXDays
	keepForXDays

Enter the number of days to keep data in the table.

The age of data in a recordset is its 'to' date, relative to the 'to' date of the newest valid recordset in the table .

When a table-data-delete task runs for a table and:

Keep for X Days only is set, PhixFlow deletes table data that is older than N days.
Keep for Y Recordsets only is set, PhixFlow keeps the most recent X recordsets and deletes recordsets that exceed X.
both Keep for X Days and Keep for Y Recordsets are set, PhixFlow deletes the table data that meets both conditions.
neither Keep for N Days nor Keep for X Recordsets are set, PhixFlow keeps table data indefinitely. This can lead to performance issues.

Keep for X Recordsets

The number of recordsets to keep in the table.

Keep Superseded for N Days

Anchor

	keepSupersededForXDays
	keepSupersededForXDays

Available when Track Superseded Data is selected.

Enter the number of days to keep superseded records in the table data.

Tip

Remember to keep superseded records for as long as you want the option to roll back the data.

For example, when Keep Superseded for N Days is set to 4, you can run roll back on data that was superseded up to 4 days ago.

A table-data-delete task removes superseded records older than 4 days, so PhixFlow no longer has the data to be able to roll it back.

In a table where the superseded date is tracked, the table data contains a mixture of active records and records that have been superseded.

When a table-data-delete task runs for a table and:

Keep Superseded for N Days only is set, PhixFlow deletes data records that were superseded more than N days ago.
Keep Superseded for X Recordsets only is set, PhixFlow deletes superseded records from the recordsets that exceed X.
both Keep Superseded for N Days and Keep Superseded for X Recordsets are set, PhixFlow deletes the superseded records that meet both conditions.
neither Keep Superseded for N Days nor Keep Superseded for X Recordsets are set, PhixFlow keeps superseded records indefinitely. This can lead to performance issues.

Keep Superseded for X Tablesets

Available when Track Superseded Data is selected.

Enter the number of recent recordsets that will retain their superseded data records.

Apply Delete Filter

Image Added:

Insert excerpt

	_check_box_tick
	_check_box_tick
nopanel	true

to open a filter tab where you can set filter conditions; see Filter. PhixFlow applies this filter when it runs a table-data-delete task. Only the records which match the filter are deleted.

Insert excerpt
_model_prop
_model_prop
nopanel true

Excerpt

name	Advanced

Advanced

Only change the advanced properties with advice by PhixFlow support.

Field Description

Advanced Properties Only use this field with advice from PhixFlow support.

Database View

Use this property if your database administrator wants to query table data in the PhixFlow database.

Enter a string in the format v_xxxx. When the table is published to the database, PhixFlow creates a view. If the table is deleted, the next system task that runs will clear the associated view from the database.

By default, this field is blank and no view is created in the database.

Audit Manual Changes

This toggle is available when the Analysis Options → Period is Transactional.

Insert excerpt

	_toggle_on
	_toggle_on
nopanel	true

toggle on to update and delete initiated by actions (not those carried out by analysis runs) will automatically mark the existing record as superseded and create a new recordset. The new versions of the updated records will be placed in the new recordset. Inserts will simply create a new recordset, and add the inserted record into that recordset.

When Audit Manual Changes is first set, the attributes UpdateAction, UpdatedByName, UpdatedByID and UpdatedTime will be created if they do not already exist. If you do not require these attributes, delete them. For the UpdatedByName attribute, PhixFlow creates a field of 250 characters.

UpdateAction must be set to the type of action, such as INSERT, UPDATE or DELETE. The other attributes will be populated if they exist on the table:

UpdatedByName - the name of the user that performed the update,
UpdatedByID - the internal id of the user that performed the update
UpdatedTime - the date and time the update was made

Storage Options

Index Scheme

Anchor

	index_scheme
	index_scheme

Select an option to determine how indexes on the table are organised.

All: indexes on the table are optimised for selecting from all recordsets (non-historied reads).
Latest: indexes on the table are optimised for selecting from the latest recordset (i.e. for historied reads).
Superseded: indexes on the table are optimised for self-updating tables which have a mostly superseded records.
None: no indexes are created on the table.
Partitioned: this index scheme is specific to partitioned tables; see Storage Type below. Partitioned tables must use this index scheme, and only partitioned tables can use it.

Storage Type

Select an option to specify how data for the table should be stored:

Database: Store the data in a regular table within the PhixFlow database. This is the most common option
Database (Partitioned): Store the data in a partitioned table within the PhixFlow database. This option provides improved performance for rollback and retaining very large recordsets. The option is only available if "partitioning" is available within your database installation.
In Memory: Data for the table will not be written to the database. This option can be used (for example) when you want to aggregate large amounts of unsorted data which can then be written to a stored table.

Data Generation Options

Anchor

	data-gen
	data-gen

Start Date

Enter the date on which this table starts or click

Insert excerpt

	_calendar
	_calendar
nopanel	true

to select a date. Data will be populated into the table from this date onwards.

Primary Key Generator

This option relates to an actionflow with a save node that inserts a new record into a table. Each new record requires a unique value for the record's primary key. Use this option to specify how PhixFlow generates the primary key value.

Internal: PhixFlow generates unique primary key values automatically. This is equivalent to setting the attribute expression to _NULL.
Sequence: display the Primary Key Generator Sequence option, in which you can select a sequence of values.

Primary Key Generator Sequence

Available when Primary Key Generator is Sequence.

Select a sequence to assign a new value to the primary key attribute of a new record; see Sequence.

The new record takes the next primary key value in a sequence. This is equivalent to setting the attribute expression to nextValue("sequenceName".

In Memory Cache Size Enter the size of the cache that will be maintained when you are using an in-memory table. See Manage Unsorted Data Using an In-Memory Table.

Allow Partial Set Processing

Insert excerpt

	_check_box_untick
	_check_box_untick
nopanel	true

to restrict all users to the Table Download Limit.

Insert excerpt

	_check_box_tick
	_check_box_tick
nopanel	true

so that, when analysis reaches the end of a buffer block, it submits the candidate set for processing, even if the next buffer block has a different key.

Prevent Parallel Processing

Available when Period is Transactional.

Insert excerpt

	_check_box_tick
	_check_box_tick
nopanel	true

to ensure only a single recordset can be generated at a time even if the table receives several concurrent requests to generate data.

This can be useful where you want to make sure that two analysis runs do not attempt to update the same records at the same time e.g. as a result of two people selecting the same records in a view and then hitting the same action button at the same time to process those records.

Run Alone

Insert excerpt

	_check_box_tick
	_check_box_tick
nopanel	true

so that, whenever the analysis engine needs to generate data for this table, it will first wait for all running tasks to complete before it starts.

Any additional analysis tasks submitted while this table is waiting to start, or while it is generating data, will wait until this table has completed its analysis before they start.

Key Tolerances When building a candidate set, data can be grouped together using a specific key value, e.g. Account Number. If the key value is a number, enter a key tolerance to identify numbers within the given tolerance as the same key value.

Default View Select the default view for the table; see Views for details of creating views on tables.

Last Run Date (Read only) PhixFlow displays the date and time that analysis was last run for this table. This date is taken from the "to date" of the most recent recordset for this table.

Last Run By PhixFlow displays the user that last ran analysis on this table.

Insert excerpt

	_access
	_access
nopanel	true

By default, the permissions to access table data is set in System Configuration → Allow access to data by default. You can add specific user groups to restrict access to the table.

Insert excerpt

	_audit
	_audit
nopanel	true

Live Search
spaceKey @self
additional none
placeholder Search all help pages
type page

Panel

borderColor	#00374F
titleColor	white
titleBGColor	#00374F
borderStyle	solid
title	Sections on this page

Table of Contents

maxLevel	3
indent	12px
style	none

Learn More

Version	Old Version 137	New Version Current
Changes made by	Fiona Sargeant (Unlicensed)	Zoe Baldwin
Saved on	Mar 16, 2022	Jan 09, 2025

Page Comparison

Versions Compared

Key

Overview

Stream Properties

Basic Settings

Updating a static stream

Attributes

Actions

Access Permissions

Overview

Insert excerpt_property_tabs_property_tabsnamebasic-hnopaneltrue

Basic Settings

Attributes

Views

AnchormultipliersmultipliersAnalysis Options

Updating a static table

Actions

Record-Actions

Sort Orders

Errors Anchorerrorerror

ERDs

Relations

Filters

Inputs

Download Limits Anchordownload_limitsdownload_limits

Stream Item Actions

Views

Sort Orders

Relations

Filters

Inputs

Store Stream Views

Analysis Models

Advanced

Data Retention Settings Anchordata-retaindata-retain

Insert excerpt_model_prop_model_propnopaneltrue

Advanced

Live SearchspaceKey@selfadditionalnoneplaceholderSearch all help pagestypepage

Insert excerpt
_property_tabs
_property_tabs
name basic-h
nopanel true

Anchor
multipliers
multipliers
Analysis Options

Errors
Anchor
error
error

Download Limits
Anchor
download_limits
download_limits

Data Retention Settings
Anchor
data-retain
data-retain

Insert excerpt
_model_prop
_model_prop
nopanel true

Live Search
spaceKey @self
additional none
placeholder Search all help pages
type page