Number of input and output field
| Number of input field |
: |
Two or more |
| Number of output field |
: |
One |
This transformation function is used to look up data in a reference file using multiple reference fields. It is applicable when the reference data can fit into the memory of each worker. If the reference data is too large, it is recommended to use a reduce-side join assembly.
The parameters of this transformation function are:
- File Reference
The name of the reference file (including the path).
- Field Splitter
The delimiter used to split each row of the reference file into fields. The default delimiter is a comma (,).
- Indexes of Ref. Fields
The indices of the fields in the reference file that will be matched against the input value. The default is 0. For example, 0-1 means the combination of the first and second fields is used as the lookup key.
- Index of Output Fields
The indices of the fields in the reference file whose values will be transferred to the output stream. For example, 1-2-4 means fields 1, 2, and 4 will be included in the output. If not specified, all fields except the key fields will be transferred.