PhixFlow Help

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Current »

About Master Lists

Lots of organisations do a lot of process work using some variation of what we call a master list. This is a bit like using an Excel spreadsheet that has a large number of rows and a large number of columns. The spreadsheet is passed around various people in the different departments so that people can:

  • either update data
  • or extract data to act on.

You can create a master list in PhixFlow

Manage a Small Master List

Scenario

Implement an approval process for credits on invoices in PhixFlow, using a master list.

Solution:

  • Create a merge stream to act as a master list, keyed off a unique identifier.
  • Create streams and/or file collectors to update the master list.
  • Create a pull pipe linking the master list Stream to itself, with its date offsets set to -1, -1.
  • Every time the stream is run, the master list stream is recreated from the previous version of itself and the updates that happen via other streams.

Manage a Large Master List

Scenario

A very large master list is periodically updated with a small create/update/delete feed. It is computationally expensive and time consuming to create a completely new master list stream, when only (for example) 220 out of 150,000,000 records are actually updated.

Implement a master list in PhixFlow where the master list is very large, and the number of updates is tiny by comparison.

Solution:

  • Create the Stream to act as a master list keyed on a unique identifier.
  • Create streams and/or file collectors to update the master list.
  • Create Secondary Streams to pull data from the master listStream, along Non-Historied pipes.
  • Group non-historied pipes by the unique identifier.
  • Construct the attribute logic in those secondary streams can distinguish the correct data from all the records that share the unique identifier.


  • No labels