Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Overview

...

Overview

A pipe is a connector that links two elements in a PhixFlow model and sends data from the input to the output. Pipes allows you to control which attributes and which records from the input are delivered by to the output, although in most cases - with minimal configuration - you will get all columns and the records from the current run.

...

Country Is in ["England","France","Germany"] fx

Sort/Group

For lookup pipes this section is called Order/Index.

Use this section to group and sort data as it comes through the pipe. This section has:

  • a toolbar with standard buttons
  • a grid that lists the attributes that you want to sort or use to group
  • below the grid are the following options:

...

Note

You cannot aggregate data from attributes if the pipe's input is from:

If you need to aggregate data from a database collector, you can use an SQL query. 

This section has:

  • a toolbar with standard buttons
  • a grid that lists the attributes that you want to aggregate.

...

 means logging will be turned off for this pipe. This pipe may still be logged if Pipe Logging options are set in System Logging
FieldDescription
Data Expected

Insert excerpt
_check_box_untick
_check_box_untick
nopaneltrue
 means the pipe may receive no data from its input during an analysis run.

Insert excerpt
_check_box_ticked
_check_box_ticked
nopaneltrue
  means PhixFlow reports an error if the pipe receives no data from the input datasource, collector or stream during an analysis run.

Allow Incomplete Stream Sets

Insert excerpt
_check_box_untick
_check_box_untick
nopaneltrue
 to complete a stream set before passing the data to the output stream. During the analysis run, PhixFlow pulls data into the input stream until the stream set is complete. If it cannot complete the stream set, PhixFlow reports an error message.

PhixFlow cannot complete a stream set if:

  • either the input stream is set to be  
    Insert excerpt
    _static
    _static
    nopaneltrue
  • or all of the pipes reading from the stream are static.

Insert excerpt
_check_box_ticked
_check_box_ticked
nopaneltrue
  PhixFlow ignores incomplete stream sets in a static input stream and does not report an error.

Tip

You must tick this check box on all the pipes that will read from a static (or effectively static) input stream in the analysis run. PhixFlow will report an error if there is any pipe trying to complete the stream set during the analysis run.

Pipes that are not used in the analysis run do not try to complete a stream set, so will not report an error. (Unused pipes can occur if they lead to streams on branches of the model that are not being run.)

Buffer SizeEnter a number for the buffer size used to perform the stream calculation. If a large amount of data is being processed, then setting a large buffer size will give better performance.

Max Records To Read


Enter a number for the maximum number of records that should be read down this pipe. The pipe may read more than this number of records if it is configured to carry out multiple reads simultaneously. For example:

  • a pipe can be  connected to a File Collector that reads multiple files simultaneously
  • the pipe strategy is Directed with multiple workers.
Strategy

Select an option to specify how this pipe should be implemented. See the section on Directed Merge Strategy

Max Workers

This field is available when Strategy is Directed

Enter the maximum number of concurrent worker tasks. When no value is specified, this defaults to 1.

Worker Size

This field is available when Strategy is Directed

Enter the number of key values to read for a single worker task, which runs a single select statement.

When no value is specified, this defaults to 1000. This is the maximum value that can be used when reading from an Oracle database.

Log Traffic
Insert excerpt
_check_box_untick_check_box_untick
nopaneltrue

Insert excerpt
_checklog_box_tickedtraffic2
_checklog_box_tickedtraffic2
nopaneltrue
 means the

  • Pipe Logging : when ticked, PhixFlow always logs the number of records returned by this pipe
will be logged in the system log file. No logging will occur if Allow Logging is not ticked in System Logging.Altering the logging setting requires the Modify System Logging Configuration privilege.
  • , whatever is set here.

Insert excerpt
_log_traffic1
_log_traffic1
nopaneltrue

Custom Data to Read 
Anchor
custom
custom

...