File Based Custom Source

File Based Custom Source () reads input data using a custom Input Format class.
The specified Input Format class must be a subclass of:

        org.apache.hadoop.mapred.FileInputFormat<LongWritable, Writable>

This source is used when standard file readers are insufficient and specialized parsing or file handling logic is required.

Input Format
The fully qualified class name of the Input Format used to read files.
Configuration Name(s)
Configuration key name(s) passed to the Input Format.
Configuration Value
Value(s) corresponding to the configuration name(s).
Include Source File Name
Specifies whether the output record includes the source file name.
- This option is applied only for combined input formats.
Delimiter
Delimiter used between the source file name and the record content when Include Source File Name is enabled.
- Default value: tab (\t)
Input Path
Path to the input file(s).
- If omitted, the input path must be supplied as a command-line argument when running the job.
- For dynamic parameters, FROM_ARGUMENT(arg_name) can be used to pass the input path at runtime.
Default File System
The default file system used for the input path, unless the file system is explicitly specified in the path.
Field Declaration
Field name of the output record.
- Default field name: line

The following Input Format implementations are available by default:

Map Reduce	Spark 1.x RDD	Spark 2.x RDD	Spark 2.x Dataset	Apache Beam 2.29.0 Batch	Apache Beam 2.29.0 Streaming	Apache Flink 1.12.0 Batch	Apache Flink 1.12.0 Streaming	Apache Tez 0.9.1	PySpark