Custom Sink () is used to write output data using a custom OutputFormat class.
This class must be a subclass of:

    com.solusi247.mapred.BaseOutputFormat247<K,V>

The class writes data to HDFS or another Hadoop-compatible storage.

Parameters of Custom Sink

1. Output Format Class
   Specifies the Java class used to write output data.
   This class must extend:

   com.solusi247.mapred.BaseOutputFormat247<K,V>
   

   and override the method:

       public org.apache.hadoop.mapred.RecordWriter<K,V> getRecordWriter(
               org.apache.hadoop.fs.FileSystem fileSystem,
               org.apache.hadoop.mapred.JobConf job,
               String name,
               org.apache.hadoop.util.Progressable progress)
   

   This method defines how the data will be written to the output file.
2. Conf. Name(s)
   Defines configuration parameter names required by the Output Format.
   If there are multiple configuration names, separate them with a comma.
   Example:

           image.width,image.height
   

3. Conf. Value
   The values corresponding to the configuration names.
   Example:

       800,600    
   

4. Has Argument Path
   Specifies whether the output path is provided as a runtime argument.
   Values:
   • True → Output path must be passed as a command-line argument
   • False → Output path is not necessary
5. Output Path
   The path to the output directory.
   • If omitted, the input path must be supplied as a command-line argument when running the job.
   • For dynamic parameters, FROM_ARGUMENT(arg_name) can be used to pass the input path at runtime.
6. Arg. Order
   Defines the argument order of the output path when it is supplied through command line parameters.
7. Output Value Type
   Specifies the data type written to the output file.
   Possible values:
   • org.apache.hadoop.io.Text
   • cascading.tuple.Tuple
8. Sink Mode
   Defines the behavior when the output directory already exists.
   Options:
   • KEEP
     Keeps the existing file
   • REPLACE
     Replaces existing files
     Old output files will be overwritten

Built-in Output Format Classes

• com.solusi247.mapred.image.ImageOutputFormat
  It is used to write image data to HDFS.
• com.solusi247.mapred.image.SIFTDescriptorOutputFormat
  It is used to write SIFT Key Point Descriptor of image to HDFS.
• com.solusi247.mapred.image.BRIEFDescriptorOutputFormat
  It is used to write BRIEF Key Point Descriptor of Image to HDFS.
• com.solusi247.mapred.image.SURFDescriptorOutputFormat
  It is used to write SURF Key Point Descriptior of Image to HDFS.
• com.solusi247.mapred.video.Video2FramesOutputFormat
  It is used to extract and write video frames to HDFS."

Supported Engines

Map Reduce	Spark 1.x RDD	Spark 2.x RDD	Spark 2.x Dataset	Apache Beam 2.29.0 Batch	Apache Beam 2.29.0 Streaming	Apache Flink 1.12.x Batch	Apache Flink 1.12.x Streaming	Apache Tez 0.9.1	PySpark