Load a Subset of Data from a Database

Scenario

Data is stored in a database, sometimes in massive volumes. You may only need to process a tiny subset of the data. Instead of processing the entire database table, collect a small subset of the data only, based on a variable value.

Create a database collector to retrieve data from a database. This collector should have a where clause in it. The value of that where clause (the value being looked up) will be set automatically in PhixFlow.

Solution:

• Create a database collector to load some data into a stream in PhixFlow
• Create an input multiplier on that stream.
• Reference the input multiplier value(s) in the collector's SQL query using the {} or {=} notation .