Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: DEV-6763 Updated for change to read future data

...

Specify what input data to use. There are 6 options available:

  • Latest: supply data from the current run (the latest stream set). This is the mostly commonly used option.
  • Previous: supply data from the previous run (the previous stream set). This is used when you are comparing data for the current run with data from the previous run, for example, today's data with yesterday's.
  • All: supply data from all runs (all stream sets). Selecting All displays the Read Future Data field, which may be used with Transactional streams. 
  • All Previous: supply data from all runs except the current run (all stream sets except the latest stream set).
  • Same Run: this option should only be used where the input and output streams are set to Period: Transactional. The pipe will only collect data from inputs in the same analysis run. This configuration support several analysis runs going on at the same time without interfering with each other. 
  • Custom: Advanced fields are revealed by selecting select this option (see to display additional settings, described in Advanced Pipe Configuration), but you are recommended not to update these unless below. We recommend that you only use the custom settings when directed to by PhixFlow consultants or support.

...

Panel
Advanced considerations - future stream sets
bgColor#e6f0ff
titleBGColor#99c2ff
titleManaging Future Stream Sets

Anchor
future
future
In some circumstances the input Stream stream may have Stream Sets stream sets that have dates in the future relative to the Stream Set stream set being generated for the output Streamstream. This may happen, for example, if:

  1. you
have rolled back a number of Stream Sets
  1. roll-back some stream sets on the output
Stream
  1. stream
  2. but
have
  1. do not
rolled
  1. roll-back the corresponding
Stream Sets
  1. stream sets on the input
Stream, and have then requested
  1. stream
  2. and then request that the output
Stream
  1. stream is brought up to date.

Some of the Stream Sets stream sets on the input Stream stream will have dates in the future relative to some of the Stream Sets stream sets you are rebuilding.

By default, pipes will ignore any Stream Sets the Read Future Data checkbox is not ticked. This means pipes ignore any stream sets with dates in the future relative to the Stream Set stream set you are generating. This is so that if you are rebuilding an old Stream Set the pipe will You want to ignore future stream sets when you rebuild an old stream set, because you want the pipe to retrieve the same data on the rerun as it retrieved when the Stream Set stream set was first built.

Similarly, if you are running a Transactional StreamWhen you run analysis on a stream with a transactional period, it is possible that while as your analysis run is taking place, other analysis runs which started after yours may have completed before yours. These will have generated additional Stream Sets on the input Stream still running, a different run can start and complete. This run can generate additional stream sets on the input stream with a future data relative to the date of the Stream Set stream set you are generating. For Transactional input Streams it is possible to tell transactional input streams, you want the pipe not to ignore use these future Stream Sets by ticking streams. To do this, tick the Read Future Data tick box, which is available when Data To Read is All or Custom checkbox.

Static

Normally when a pipe requests data from a non-static input stream, that stream will first attempt to bring itself up to date, generating new stream sets as necessary, before supplying the data requested. However, if this field is ticked, the input stream will not run. Pipes from collectors cannot be marked as static.

...