How To: Read a Directory of Filenames

It is often useful to read a local or network directory and populate a PhixFlow stream with the names of the files. This can be used as a reference list for subsequent processing and can also form the basis of a duplicate detection pattern, ensuring that files are not processed mor than once. The examples below show how to read the entire folder contents,as well as just speciifc file types.

The model for these examples is 004 Read a List of Files in a Directory

Example 1: Read all files

Step-by-step guide

1. Add a File Collector to the model and confiure it as follows:
   
   1. Source Type = Specified Directory
   2. File Type = File Details Only
   3. File Location Strategy = All Files in Folder
   4. Input Directory Expression = Enter the name of the folder relative to the Import File Locaion in  System Configuration. Put the folder name in double quotes e.g "Data Files"
   5. File Pattern Expression = ".*." (This will read all file types)
2. Make sure the Enabled and Allow on-Schedued collection checkboxes are checked and then save and close the file colletor
3. Hover over the file collector and select Add a New Stream
4. Add an attribute to the stream to hold the fie name and enter the expression "in._fileName" without the quotes.
5. Run analysis on the stream and view the filenames

Copying multiple PhixFlow Objects

File Collector Configuration

Attribute Expression to Read Filename

 Example Folder and Results in PhixFlow

Example 2: Read all files with a given file extention

Step-by-step guide

This example shows how to modify the process to only read the csv files. This can be useful when there are multiple file types in a single folder and you want to have separate streams for each type, or just process them separately.

1. Add a File Collector to the model and confiure it as follows:
   
   1. Source Type = Specified Directory
   2. File Type = File Details Only
   3. File Location Strategy = All Files in Folder
   4. Input Directory Expression = Enter the name of the folder relative to the Import File Locaion in  System Configuration. Put the folder name in double quotes e.g "Data Files"
   5. File Pattern Expression = ".*." (This will read all file types)
2. Make sure the Enabled and Allow on-Schedued collection checkboxes are checked and then save and close the file colletor
3. Hover over the file collector and select Add a New Stream
4. Add an attribute to the stream to hold the fie name and enter the expression "in._fileName" without the quotes.
5. Run analysis on the stream and view the filenames 

Copying Multiple PhixFlow Objects

Modified Expression to read just the CSV files

Related articles