Inexact Match Streams

Scenario

Sometimes the data that you are trying to match is not exactly the same, but you still want to be able to align and merge data, or create a report on likely matches.

Create 2 streams for data. The goal is to match up those records from stream 1 where at least 3 of 5 fields match stream 2.

Solution:

• Create a stream for each input
• Create a (calculate) stream for the join.
• Make the pipe from one of the streams a pull pipe and the pipe from the other is a lookup pipe.
• In the join stream, for each record on the pull pipe, cycle thru all the records from the lookup pipe and flag those records where there is a match.