Number of input and output field
| Number of input field |
: |
One |
| Number of output field |
: |
One |
This transformation function is used to look up data in a reference file using a single reference field with a string-similarity operation. The reference entry with the highest similarity score is written to the output stream. This transformation is applicable when the reference data can fit into the memory of each worker.
The parameters of this transformation function are:
- Reference File
The name of the reference file.
- Index of Ref. Field
The index of the field in the reference file that will be matched against the input value. The default value is 0.
- Field Splitter
The delimiter used to split rows of the reference file into fields. The default delimiter is a comma (,).
- Index of Output Fields
The indices of the fields in the reference file whose values will be transferred to the output stream. For example, 1-2-4 means that fields 1, 2, and 4 will be included in the output.
- Similarity Method
The method used to calculate string similarity. All methods use default parameters.
The available algorithms are:
- LongestCommonSubsequence
- Cosine
- Jaccard
- JaroWinkler
- Levenshtein
- Metaphone
- NeedlemanWunsch
- QGram
- SmithWaterman
- SorensenDice
- Soundex
- Min Similarity
The minimum similarity score required for a reference entry to be considered.