Versions Compared
compared with
Key
- This line was added.
- This line was removed.
- Formatting was changed.
Insert excerpt | ||||||||
---|---|---|---|---|---|---|---|---|
|
Scenario
Very large quantities of data are required to be processed from a set of files. It is useful to be able to process each file into a separate
...
table, such that if we are processing 10 files and there is a failure in in the 9th file, we will not have to re-process the first 8 files.
Load the data from 10 files into 10 different
...
tables in the same
...
recordset.
Solution:
- Create a
...
- table connected to a file collector
- Ensure the file collector will collect multiple files when it is activated (i.e. ensure there are) multiple files that match the file collector criteria.
- Create pipe connecting the
...
- table to itself.
- Set the pipe attribute : type to 'push'.
- Set the pipe attributes : From and To Date Offsets to 1.
...