...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
| Insert excerpt | ||||||||
|---|---|---|---|---|---|---|---|---|
|
...
...
...
...
...
...
...
...
...
...
The following fields are configured for Streams:
...
...
...
- Transactional: allows multiple users to run independent analysis tasks at the same time.
- Daily: generate or collect data every day.
- Monthly: generate or collect data every month.
- Variable: generate or collect data since the more recent run of the stream to the current date.
When the period is first set to Transactional the UID stream attribute will be created if it does not already exist.
...
The type of function used to generate this stream. Possible types are:
...
You can select a "loop" pipe - that is, a pipe linking the stream back into itself - in this field. If you do, new records will be compared to existing records, using the selected loop pipe, and if a repeated record is found, the old one will be marked as 'superseded'.
...
Only applies when period is Transactional. If ticked, updates and deletes initiated by stream actions (not those carried out by analysis runs) will automatically mark the existing record as superceded and create a new stream set. The new versions of the updated records will be placed in the new stream set. Inserts will simply create a new stream set, and add the inserted record into that stream set.
When Audit Manual Changes is first set, the attributes UpdateAction, UpdatedByName, UpdatedByID and UpdatedTime will be created if they do not already exist. If you do not require these attributes, delete them.
UpdateAction must be set to the type of action, such as INSERT, UPDATE or DELETE. The other attributes will be populated if they exist on the stream:
- UpdatedByName - the name of the user that performed the update,
- UpdatedByID - the internal id of the user that performed the update
- UpdatedTime - the date and time the update was made
...
A list of the stream attributes in the stream. The toolbar on this section has the options: 
Show list of File Collectors and 
Shows the list of Streams.
To edit the properties for an attribute, double-click the attribute name. To edit only the expression:
- Right-click an attribute name to display the context menu.
- Select Edit the expression field.
- PhixFlow opens a simple text editor box
- Make changes to the attributes expression.
- Click
to save your changes.
...
...
The order of the attributes in the stream. This is important because the stream attribute expressions are evaluated in this order. If the results of an attribute expression, or a $ variable calculated during its calculation, are required in the expression of a second attribute - the second attribute must come after the first in the attribute list.
...
- When 1 or more output pipes from the stream uses the field to 'Filter' the stream.
- When 1 or more output pipes from the stream uses the field in a 'Sort/Group' by action.
...
If ticked, new filter conditions on this field are case-insensitive by default. The filter window → Ignore Case check box inherits this setting; see Filters on Data Views. For case-insensitive filters, there is no difference if the attribute is also indexed.
...
do ( $aRange = [], addElement($aRange, rng.RangeFrom), addElement($aRange, rng.RangeTo), $bRange = [], addElement($bRange, $aRange), $bRange )
...
ifNull(in.ASSET, [1,10,12] , // else do [5,7] )
...
do( lookup(lkin, $num = in.BNumber), lkin )
will return a list of records which match the lookup on the lkin pipe. In this case the required data can be extracted from the Output Multiplier using the following expression :
do ( $values = _type, $values.account_num )
If the output multiplier expression evaluates to _NULL, an empty list of values or an empty list of records then a single output record will be produced with _type set to _NULL, _NULL or an empty record respectively.
...
A list of the filters on the stream. See Filters on Data Views.
Any filter defined on the stream may appear in the dropdown list of filters accessible from the header of each stream view. To make a filter available in a view, the filter must be added to the list of filters for that view. See Stream View for details.
All filters defined on this tab will be available on the system generated Default View for this Stream.
...
A list of pipes into the stream.
| Note |
|---|
It is possible for this list to include pipes that have no input. This occurs if the source stream has been deleted, or if a model has been moved to a different PhixFlow instance (export/import), leaving behind a referenced stream. Any pipes with no input are highlighted in yellow. To resolve pipes with no input you can:
|
...
The number of days data to keep in the stream.
When an archive task runs for a stream, all stream data is deleted if it is at least Keep for X Days old or if it is older than the Keep for Y Stream Sets most recent valid stream sets.
If both Keep for X Days and Keep for Y Stream Sets are set, stream data will be deleted only if it meets both conditions. If neither are set, stream data is kept indefinitely.
If Save Archive to File is ticked, deleted items are first saved to archive files.
The age of data in a stream set is its 'to' date relative to the 'to' date of the newest valid stream set in the stream.
See here for how to set up and schedule an Archive Task.
Please see the section below on Archiving Examples to see how this value can be used within Archiving strategies.
...
The number of stream sets data to keep in the stream.
See Keep for X Days for the main description of archiving.
...
The number of days for which to keep superseded data in the stream.
If Track Superseded Data is ticked, then this field will become visible/enabled.
In a stream where the superseded date is tracked, the stream data will contain a mixture of superseded records and "active" records - that is, records that have not been superseded.
When an archive task runs for a stream, records that were marked as superseded more than Keep Superseded for X Days days or more than Keep Superseded for Y Stream Sets stream sets ago are deleted.
If both Keep Superseded for X Days and Keep Superseded for Y Stream Sets are set, superseded records will be deleted only if they meet both conditions. If neither are set, superseded records are not deleted.
This means, for example, that if you have set Keep Superseded for X Days to 4, you will be able to roll back 3 days, making the 4th day the latest valid day.
If Save Archive to File is ticked, deleted items are first saved to archive files.
Please see the section below on Archiving Superceded Examples to see how this value can be used within Archiving strategies.
...
The number of stream sets for which to keep superseded data in the stream.
If Track Superseded Data is ticked, then this field will become visible/enabled.
See Keep Superseded for X Days for the main description of archiving superseded records.
...
If checked, this specifies that all users can view this data by default (provided they have the basic privilege to view streams).
If this field is not checked, then access to the underlying data is controlled by dropping user groups onto the stream's "User Group" tab.
Note that the default setting for this field on streams is controlled by the system parameter allowAccessToDataByDefault.
...
- All: indexes on the Stream are optimised for selecting from all stream sets (non-historied reads).
- Latest: indexes on the Stream are optimised for selecting from the latest stream set (i.e. for historied reads).
- Superseded: indexes on the Stream are optimised for self-updating streams which have a moslty superseded records.
- None: no indexes are created on the Stream.
...
- Database: Store the data in a regular table within the PhixFlow database. This is the most common option
- Database (Partitioned): Store the data in a partitioned table within the PhixFlow database. This option provides improved performance for rollback and archiving of very large Streamsets. The option is only available if "partitioning" is available within your database installation.
- In Memory: Data for the Stream will not be written to the database. This option can be used (for example) when you want to aggregate large amounts of unsorted data which can then be written to a stored Stream.
...
This field only appears if the Period is set to Transactional. If ticked, it ensures that only a single stream set can be generated at a time even if the stream receives several concurrent requests to generate data.
This can be useful where you want to make sure that two analysis runs don't attempt to update the same records at the same time e.g. as a result of two people selecting the same records in a view and then hitting the same action button at the same time to process those records.
...
If this flag is ticked then whenever the analysis engine needs to generate data for this stream it will first wait for all running tasks to complete before it starts.
Any additional analysis tasks submitted while this stream is waiting to start, or while it is generating data, will wait until this stream has completed its analysis before they start.
...
Description
...
The table below assumes the stream to be archived currently contains 8 stream sets. Two from the current day and one from each of the previous 6 days.
In the table below the value null refers to the fact that no value has been entered into this field.
Note that archiving will always retain the maximum active stream sets in the data such that no conflicting stream sets will be archived.
...
In the case where only the Keep Superseded for X Days and Keep Superseded for Y StreamSets fields are populated, the same logic in the table above will apply to the superseded records. Note that again archiving will always retain the maximum superseded stream sets in the data such that no conflicting stream sets will be archived.
In the cases where a mixture of the full archive fields Keep for X Days, Keep for Y StreamSets' and the superseded archive fields Keep Superseded for X Days, Keep Superseded for Y StreamSets are populated, then the full archive values will be first applied and the resultant stream item records will be archived and deleted. Only then will the remaining stream sets use the Keep Superseded ... values to apply a further condition to archive and delete any remaining non qualifying superseded records.
Attribute Types
...
A Bigstring is used for strings over 4000 characters long. Bigstring is a different data type to string and has some restrictions on filtering, sorting and aggregation.
For instances using an Oracle database, Bigstrings cannot be sorted or aggregated. On Oracle Bigstrings may only be filtered with the conditions (not) contains, is (not) null, (not) starts with or (not) ends with.
The maximum Bigstring size can be configured in System Configuration → System Tuning → Maximum Bigstring Size.
...
Decimal is a non-integer number which is stored to a set level of precision. Decimals have a
- significant figures property, which is the number of digits stored,
- a decimal places property, which is the number of digits after the decimal place.
The maximum number of integer digits is therefore significant figures minus decimal places. If the number of integer digits is greater than the limit, analysis will fail. Decimal places will be stored to the scale specified.
...
| Description | We recommend that you always enter a description to explain the purpose of this item. |
Attributes
The grid lists the attributes (data columns) in the table data.
| Excerpt | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||
|
| Excerpt | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||
ViewsThe grid contains a list of the data views for this table.
|
Anchor multipliers multipliers
Analysis Options
| multipliers | |
| multipliers |
| Field | Description | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Enabled |
| ||||||||
| Static Data |
Updating a static tableIn general, static tables are not updated during an analysis run that is initiated on another table in the model. The exception is when an incoming push pipe is triggered, as this will force the table to update itself. Instead, to update the data in a static table you must run analysis directly on the table via:
| ||||||||
| Period | Select the time period for the table:
| ||||||||
| Default Data Range | This option affects the following items for which this is the backing table:
If no option is is selected, PhixFlow assumes the default data range is All. You can override this setting using the options Use Custom Data Range and Data Range available: You cannot override this setting for a view. | ||||||||
| Table Type | Select the table type; see Types of Table and When to Use Them. The type indicates the function that operates on the data.
| ||||||||
| Supersede Items on Pipe | Select a loop pipe from the list of those available. A loop pipe links the table back to itself. It compares new data records to the existing data records. If there is a repeated record PhixFlow uses the new one in the table and marks the old one as | ||||||||
| Input Multiplier | The input multiplier expression should evaluate to a list of one or more values. For each value in the list, the internal variable _inputMultiplier will be set to that value and the whole table processing will be repeated i.e. the pull pipes will be read and the data from those pipes processed to generate output records to be added to the current recordset. For example : do ( $aRange = [], addElement($aRange, rng.RangeFrom), addElement($aRange, rng.RangeTo), $bRange = [], addElement($bRange, $aRange), $bRange ) Where rng.RangeFrom = 500 and rng.RangeTo = 1000, the above example evaluates to [[500,1000]], which is a list containing 1 element, which is itself a list containing 2 elements. An input multiplier that evaluates to [3,4,7,8] would run the Table 4 times. | ||||||||
| Log Input Multiplier Messages |
| ||||||||
| Output Multiplier | This field is an expression which should evaluate to an array of values. A separate output record will be produced for each value in the array and this value is available as _outputMultiplier in each of the table output attribute expressions (each value in the array is also available through _type, although this is not recommended usage). In effect this will multiply each of the output records by the number of elements from the returned list. For example : ifNull(in.ASSET, [1,10,12] , // else do [5,7] ) will create 3 records for every record in the table if in.ASSET contains a value (setting _type = 1, 10 and 12 in each case). Otherwise it will create 2 records for every record in the table (and set _type = 5 and 7). An output multiplier may also evaluate to a record, or a group of records. For example an Output Multiplier with the expression:do( lookup(lkin, $num = in.BNumber), lkin ) will return a list of records which match the lookup on the lkin pipe. In this case the required data can be extracted from the Output Multiplier using the following expression : do ( $values = _type, $values.account_num ) If the output multiplier expression evaluates to _NULL, an empty list of values or an empty list of records then a single output record will be produced with _type set to _NULL, _NULL or an empty record respectively. | ||||||||
Select Records to Output (Formerly Output Filter) | You can select a subset of records to output to the next modelling object. You select records using the data in any attribute that is in the table output. Enter an expression that should evaluate to true or false (equivalently 1 or 0). When the expression evaluates to true or 1, the records found by the expression are selected and output to the next modelling object. For example:
To select active accounts, enter the expression |
Actions
The grid contains a list of actions that operate on the table.
- To edit the properties of an action, double-click its name
- To add an action to the list, click
and PhixFlow opens the Table-Action properties tabInsert excerpt _new _new nopanel true - To remove an action from the table, select the action and click
in the toolbarInsert excerpt _delete _delete nopanel true
Record-Actions
The grid contains a list of record-actions that can update records in the table.
- To edit the properties of a record-action, double-click its name
- To add a record-action to the list, click
and PhixFlow opens the Record-Action properties tabInsert excerpt _new _new nopanel true - To remove a record-action from the table, select the record-action and click
in the toolbarInsert excerpt _delete _delete nopanel true
| Excerpt | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||
Sort OrdersThe grid contains a list of the sort orders for this table.
|
| Excerpt | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||
Errors
|
| Excerpt | ||
|---|---|---|
| ||
ERDsThe grid displays a read-only list of the Entity Relationship Diagrams (ERDs) that this table is part of. Double-click an ERD to open it. See Entity Relationship Diagrams. |
| Excerpt | ||
|---|---|---|
| ||
RelationsThe grid displays a read-only list of the relationships that this table has to attributes in other tables. Double-click a relationship to display it's properties; see Relationship and Entity Relationship Diagrams. |
| Excerpt | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||
FiltersThe grid contains a list of the filters that can be applied to this table. You can add filters to view based on this table. All filters in this list are available in the default view for this table. For views that you create, in the View properties tab → Filters section, you can add filters. The list of available filters is populated by the filters defined for the table. To edit the properties of a filter, double-click its name. To add a filter to the list, click
To remove a filter from the table, select the filter and click
|
Inputs
This section lists all pipes that connect into the table. Double-click a pipe to view more details and make changes. See Pipe.
This section has a toolbar with
and Insert excerpt _detailed_list _detailed_list nopanel true
buttons; see standard buttons, see Menu Options and Toolbar Buttons. Insert excerpt _refresh _refresh nopanel true
| Note |
|---|
This list can include pipes that have no input. This occurs if the source table has been deleted or if a model has been moved to a different PhixFlow instance (export/import), leaving behind a referenced table. Any pipes with no input are highlighted in yellow. To resolve pipes with no input you can:
|
Download Limits
| Anchor | ||||
|---|---|---|---|---|
|
The following properties affect the number of data records a user can download from a view, such as a grid, chart or card; see Download CSV or Excel Data.
| Field | Description | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Table Download Limit | Enter the maximum number of records that a user can download from a view on this table. If no value is set, PhixFlow uses the default limit set in System Configuration→ System Tuning → Default Download File Limit. | ||||||||||||||||||||||||
| Allow Unlimited Downloads |
| ||||||||||||||||||||||||
| User Groups | Available when Allow Unlimited Downloads is ticked. Use this section to select the user groups whose members can download an unlimited number of records. This section has a toolbar with standard buttons, see Menu Options and Toolbar Buttons, and The grid contains a list of user groups that are exempt from the Table Download Limit. To add a user group to the list:
To remove user groups, use the
|
| Excerpt | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Data Retention Settings
Use the following options to specify how PhixFlow manages old data when a table-data-delete task runs.
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Insert excerpt | ||||||
|---|---|---|---|---|---|---|
|
| Excerpt | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
AdvancedOnly change the advanced properties with advice by PhixFlow support.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Insert excerpt | ||||||
|---|---|---|---|---|---|---|
|
By default, the permissions to access table data is set in System Configuration → Allow access to data by default. You can add specific user groups to restrict access to the table.
| Insert excerpt | ||||||
|---|---|---|---|---|---|---|
|
| Live Search | ||||||||
|---|---|---|---|---|---|---|---|---|
|
| Panel | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
|
Learn More
- Entity Relationship Diagrams
- Displaying Data (Views)
- Analysis Models for Batch Processing Data
- Attribute
- View