Some systems deal with massive quantities of data. Where the processing bottleneck is found to be the database data extraction the data can, under certain circumstances, be extracted in parallel to speed up processing.
Use multiple pull pipes to extract data from a single database collector into a single stream.
Solution:
Create the database collector and the Stream to contain the data
Link the collector and stream with multiple pull pipes.
Each pipe has a unique name.
In the 'where' clause of the SQL statement in the collector, use the pipe name to limit the set of data being to be retrieved. E.g if the pipe is called db2, then have the SQL query only return those records where the id ends in 2. this can be does using the _pipeName internal variable.
