...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
Insert excerpt | ||||||||
---|---|---|---|---|---|---|---|---|
|
...
...
...
...
...
...
...
...
...
...
When you make changes to streams and stream attributes, PhixFlow automatically publishes them to the stream data tables in the database; see Publishing Streams.
Stream Properties
Basic Settings
...
...
When Audit Manual Changes is first set, the attributes UpdateAction
, UpdatedByName
, UpdatedByID
and UpdatedTime
will be created if they do not already exist. If you do not require these attributes, delete them. For the UpdatedByName
attribute, PhixFlow creates a field of 250 characters. (In versions up to 8.0.4, the field was 50 characters long.)
UpdateAction
must be set to the type of action, such as INSERT, UPDATE or DELETE. The other attributes will be populated if they exist on the stream:
UpdatedByName
- the name of the user that performed the update,UpdatedByID
- the internal id of the user that performed the updateUpdatedTime
- the date and time the update was made
Attributes
...
The grid lists the attributes (data columns) in the stream data. To edit the properties of an attribute, double-click the attribute name to open the Stream Attributes properties.
...
To add an existing attribute to the list:
...
To edit only the expression:
- Right-click an attribute name to display the context menu.
- Select Edit the expression field.
- PhixFlow opens a simple text editor box
- Make changes to the attributes expression.
- Click to save your changes.
...
do ( $aRange = [], addElement($aRange, rng.RangeFrom), addElement($aRange, rng.RangeTo), $bRange = [], addElement($bRange, $aRange), $bRange )
...
ifNull(in.ASSET, [1,10,12] , // else do [5,7] )
...
do( lookup(lkin, $num = in.BNumber), lkin )
will return a list of records which match the lookup on the lkin pipe. In this case the required data can be extracted from the Output Multiplier using the following expression :
do ( $values = _type, $values.account_num )
If the output multiplier expression evaluates to _NULL, an empty list of values or an empty list of records then a single output record will be produced with _type set to _NULL, _NULL or an empty record respectively.
...
Select Records to Output
...
You can select a subset of records to output to the next modelling object. You select records using the data in any attribute that is in the stream output. Enter an expression that should evaluate to true or false (equivalently 1 or 0).
When the expression evaluates to true or 1, the records found by the expression are selected and output to the next modelling object.
For example:
- A stream containing data about accounts has an attribute called "active".
- You only want to include "active" accounts in the following streams.
- Active accounts have the value 1.
To select active accounts, enter the expression _out.active
.
All records that have the active attribute set to any other value are ignored.
Actions
The grid contains a list of actions that use the stream. This section has a toolbar with standard buttons.
...
...
Updating a static stream
In general, static streams are not updated during an analysis run that is initiated on another stream in the model. The exception is when an incoming push pipe is triggered, as this will force the stream to update itself.
Instead, to update the data in a static stream you must run analysis directly on the stream via:
- the model
- a task plan.
...
- Transactional: allows multiple users to run independent analysis tasks at the same time.
- Daily: generate or collect data every day.
- Monthly: generate or collect data every month.
- Variable: generate or collect data since the more recent run of the stream to the current date.
As transactional streams need a UID
attribute, PhixFlow automatically creates this stream attribute if it does not already exist.
...
This option affects the following items for which this is the backing stream:
- stream views
- dashboards
- card layout components.
...
If no option is is selected, PhixFlow assumes the default data range is All.
You can override this setting using the options Use Custom Data Range and Data Range available:
You cannot override this setting for a stream view.
...
Select the stream type. The type indicates the function that operates on the data.
- Aggregate Stream
- Calculate Stream
- CalculateBySet Stream
- Cartesian Stream
- Merge Stream
...
Select a loop pipe from the list of those available.
A loop pipe links the stream back to itself. It compares new data records to the existing data records. If there is a repeated record PhixFlow uses the new one in the stream and marks the old one as superseded
.
...
This check box is available when the Period is Transactional.
...
...
...
...
...
...
...
...
...
...
...
...
...
...
Views
The grid contains a list of the data views for this stream. This section has a toolbar with standard buttons.
o edit the properties of an stream view, double-click the attribute name to open its Stream View properties tab.
...
Sort Orders
A list of the sort orders on the stream. See Sort Order.
Filters
A list of the filters on the stream. See Filters on Data Views.
Any filter defined on the stream may appear in the dropdown list of filters accessible from the header of each stream view. To make a filter available in a view, the filter must be added to the list of filters for that view. See Stream View for details.
All filters defined on this tab will be available on the system generated Default View for this Stream.
Inputs
A list of pipes into the stream.
Note |
---|
It is possible for this list to include pipes that have no input. This occurs if the source stream has been deleted, or if a model has been moved to a different PhixFlow instance (export/import), leaving behind a referenced stream. Any pipes with no input are highlighted in yellow. To resolve pipes with no input you can:
|
...
The following options affect the number of data records a user can download from a stream view, such as a table (grid), chart or card; see Download CSV or Excel Data.
...
Available when Allow Unlimited Downloads is ticked.
Use this section to select the user groups whose members can download an unlimited number of records.
...
The grid contains a list of user groups that are exempt from the Stream Download Limit. To add a user group to the list:
...
The number of days data to keep in the stream.
When an stream data delete task runs for a stream, all stream data is deleted if it is at least Keep for X Days old or if it is older than the Keep for Y Stream Sets most recent valid stream sets.
If both Keep for X Days and Keep for Y Stream Sets are set, stream data will be deleted only if it meets both conditions. If neither are set, stream data is kept indefinitely.
The age of data in a stream set is its 'to' date, relative to the 'to' date of the newest valid stream set in the stream.
See also:
- Tasks, Task Plans and Archiving for information about tasks
- Archiving Stream Data Examples to see how this value can be used within archiving strategies.
...
The number of stream sets data to keep in the stream.
See Keep for X Days for the main description of archiving.
...
The number of days for which to keep superseded data in the stream.
If Track Superseded Data is ticked, then this field will become visible/enabled.
In a stream where the superseded date is tracked, the stream data will contain a mixture of superseded records and "active" records - that is, records that have not been superseded.
When an archive task runs for a stream, records that were marked as superseded more than Keep Superseded for X Days days or more than Keep Superseded for Y Stream Sets stream sets ago are deleted.
If both Keep Superseded for X Days and Keep Superseded for Y Stream Sets are set, superseded records will be deleted only if they meet both conditions. If neither are set, superseded records are not deleted.
This means, for example, that if you have set Keep Superseded for X Days to 4, you will be able to roll back 3 days, making the 4th day the latest valid day.
Please see the section below on Archiving Superceded Examples to see how this value can be used within Archiving strategies.
...
The number of stream sets for which to keep superseded data in the stream.
If Track Superseded Data is ticked, then this field will become visible/enabled.
See Keep Superseded for X Days for the main description of archiving superseded records.
...
Store Stream Views
To do
Access Permissions
By default, the permissions to access stream data is set in System Configuration → Allow access to data by default. You can add specific user groups to restrict access to the stream; see Common Properties → Access Permissions.
Analysis Models
A list of the analysis models that this stream appears on.
Advanced
The advanced properties field should only be set by, or under the guidance of, PhixFlow support.
...
Use this option if your database administrator wants to query stream data in the PhixFlow database.
Enter a string in the format v_xxxx
. When the stream is published to the database, PhixFlow creates a stream view. If the stream is deleted, the next system task that runs will clear the associated stream view from the database.
By default, this field is blank and no stream view is created in the database.
See also Configuring Access to Database Views.
...
- All: indexes on the stream are optimised for selecting from all stream sets (non-historied reads).
- Latest: indexes on the stream are optimised for selecting from the latest stream set (i.e. for historied reads).
- Superseded: indexes on the stream are optimised for self-updating streams which have a mostly superseded records.
- None: no indexes are created on the stream.
- Partitioned: this index scheme is specific to partitioned streams; see Storage Type below. Partitioned streams must use this index scheme, and only partitioned streams can use it.
...
- Database: Store the data in a regular table within the PhixFlow database. This is the most common option
- Database (Partitioned): Store the data in a partitioned table within the PhixFlow database. This option provides improved performance for rollback and archiving of very large Streamsets. The option is only available if "partitioning" is available within your database installation.
- In Memory: Data for the Stream will not be written to the database. This option can be used (for example) when you want to aggregate large amounts of unsorted data which can then be written to a stored Stream.
...
This field only appears if the Period is set to Transactional. If ticked, it ensures that only a single stream set can be generated at a time even if the stream receives several concurrent requests to generate data.
This can be useful where you want to make sure that two analysis runs don't attempt to update the same records at the same time e.g. as a result of two people selecting the same records in a view and then hitting the same action button at the same time to process those records.
...
If this flag is ticked then whenever the analysis engine needs to generate data for this stream it will first wait for all running tasks to complete before it starts.
Any additional analysis tasks submitted while this stream is waiting to start, or while it is generating data, will wait until this stream has completed its analysis before they start.
...
The table below assumes the stream to be archived currently contains 8 stream sets. Two from the current day and one from each of the previous 6 days.
In the table below the value null refers to the fact that no value has been entered into this field.
Note that archiving will always retain the maximum active stream sets in the data such that no conflicting stream sets will be archived.
...
In the case where only the Keep Superseded for X Days and Keep Superseded for Y StreamSets fields are populated, the same logic in the table above will apply to the superseded records. Note that again archiving will always retain the maximum superseded stream sets in the data such that no conflicting stream sets will be archived.
...