Pages in this topic:

• Which Stream to Use
• Stream Set List
• Copy or Move Stream Data
• Using Directed Pipes to Read from Large Data Sets
• Properties:
  • StreamTable Properties
  • Attributes 
  • View
  • Pipe 
  • Sort Order
• Rolling back stream set:
  • Rollback
  • Rollback or Repeat Cycles

See also: Importing and Exporting Data

Types of Stream

There are several types of stream.

Anchor
calculate calculate

Insert excerpt
_table_calculate _table_calculate nopanel true

Calculate streams are the most basic type. An output record will be produced for each input record.

Anchor
merge merge

Insert excerpt
_table_merge _table_merge nopanel true

Merge streams combine sets of input data. In each input pipe a grouping is defined, and an output record is produced for each key value combination that is produced by this grouping applied across all inputs.

Anchor
aggregate aggregate

Insert excerpt
_table_aggregate _table_aggregate nopanel true

Aggregate streams aggregate input data. In the input pipe a grouping is defined, and an output record is produced for each key value combination that is produced by the grouping.

Aggregate streams are functionally identical to merge streams, but by convention, when there is only one input, an aggregate stream is used - this displays as a  on the model view. Often this helps to clarify the purpose of the stream in the model.

Anchor
cartesian cartesian

Cartesian Stream

Cartesian streams perform a cartesian join across all inputs. Although this can be useful in some cases, mostly it is easier and simpler to multiply output records with either an output multiplier - which can be configured for any stream type - or to use a multiplier pipe.

Anchor
byset byset

CalculateBySet Stream

Calculate by Set streams are like calculate streams in that an output record is produced for each input record. But in addition a grouping can be configured on the input pipe which allows, for each record processed, related rows to be included in calculations.

For details about when to use each type of streams, see Which Stream to Use and the linked pages.

Streams and Time Periods

In any given moment, a streams contains a set of records. This is the record-set. You can set the time period over which PhixFlow collects records. The period can be:

For non-transactional periods, PhixFlow checks for incomplete record-sets and reports an error if it finds them. However, pipes from transactional streams allow incomplete record-sets, as data is constantly changing.

Publishing Streams

Anchor
publish publish

When you make changes to a stream's properties or its attributes, PhixFlow publishes the changes to the PhixFlow database. This happens automatically in the background. Publishing many streams or streams with many attributes can take some time, and may slow performance.

If the stream properties are set incorrectly, PhixFlow will not be able to publish the stream to the database. If this happens, the

During the publishing process, PhixFlow may create temporary streams in its database. These are kept for a period, then automatically removed when a system task runs. For information about:

• the system task, see Using Tasks and Task Plans
• configuring the period that temporary streams are kept; see System Configuration → Delete Temp Streams after Days.