Number of input and output link

Number of input link	:	One
Number of output link	:	One

Combiner Assembly () performs local aggregation (pre-aggregation) of data on each processing node before the shuffle phase.
Its primary goal is to reduce the volume of data transferred across the network, improving overall performance in distributed processing engines such as MapReduce and Spark.

Purpose

• Reduce network I/O during shuffle
• Improve performance for aggregation-heavy pipelines
• Perform early-stage aggregation on mapper/executor side
• Minimize memory and disk usage during data transfer

Execution Behavior

• Runs before the shuffle phase
• Executes locally on each node
• Produces partial aggregation results
• Final aggregation is completed by Aggregator Assembly
• Does not guarantee global correctness alone (must be followed by Group By + Aggregator)

Local Aggregate Types

1. Basic Aggregator
   Provides standard aggregation functions:
   1. Count
      Calculates the number of records
   2. Max
      Calculates the maximum value
   3. Min
      Calculates the minimum value
   4. Sum
      Calculates the sum of values
2. Condition Aggregator
   Performs aggregation only when a specified condition is satisfied:
   1. ConditionCount
      Calculates the count based on a condition
   2. ConditionMax
      Calculates the maximum value based on a condition
   3. ConditionMin
      Calculates the minimum value based on a condition
   4. ConditionSum
      Calculates the sum of values based on a condition
3. Calculation Aggregator
   Performs aggregation based on mathematical expressions applied to fields:
   1. CalculationMax
      Calculates the maximum value based on a math calculation
   2. CalculationMin
      Calculates the minimum value based on a math calculation
   3. CalculationSum
      Calculates the sum of values based on a math calculation

Performance Considerations

• Significantly reduces shuffle size
• Best suited for large datasets with heavy aggregation
• Ineffective if data is already small or highly unique
• Does not replace the Aggregator Assembly

Typical Use Cases

• Large-scale counting
• Summation of metrics
• Pre-aggregating logs or events
• Optimizing join and aggregation pipelines
• Reducing network bottlenecks in big data workflows

Supported Engines

Map Reduce	Spark 1.x RDD	Spark 2.x RDD	Spark 2.x Dataset	Apache Beam 2.29.0 Batch	Apache Beam 2.29.0 Streaming	Apache Flink 1.12.0 Batch	Apache Flink 1.12.0 Streaming	Apache Tez 0.9.1	PySpark