Number of input and output link

Number of input link	:	One
Number of output link	:	One

Top N Assembly () is used to select the top N records from each local partition based on a specified sorting criterion.

This assembly operates locally on each partition, meaning it does not perform a global shuffle across the entire dataset.

How It Works

• Input data is divided into multiple partitions.
• Within each partition, records are sorted based on one or more specified fields.
• Only the top N records from each partition are selected.
• The selected records are emitted to the output stream.

Key Characteristics

• Operates at the local partition level
• Does not guarantee global Top N results
• Improves performance by reducing data volume early
• Often used before shuffle-heavy operations (such as Group By)

Example

Input Data (2 Partitions)
Partition 1

    ID	Score
    ---------
    A	95
    B	80
    C	70

Partition 2

    ID	Score
    ---------
    D	90
    E	85
    F	60

Configuration

• Sort Field: Score (Descending)
• N = 2

Output

Partition 1

• A (95)
• B (80)

Partition 2

• D (90)
• E (85)

Common Use Cases

• Pre-filtering large datasets
• Reducing data size before aggregation or join
• Selecting top-performing records per partition
• Optimizing performance in distributed processing

Important Notes

• To compute a global Top N, the Top N Assembly must be followed by:
  A Group By & Buffer Assembly
• Sorting order (ascending or descending) must be defined explicitly.

Summary

The Top N Assembly selects the highest-ranked N records within each local partition, providing an efficient way to reduce data early in distributed processing pipelines, though it does not produce globally ranked results by itself.

Supported Engines

Map Reduce	Spark 1.x RDD	Spark 2.x RDD	Spark 2.x Dataset	Apache Beam 2.29.0 Batch	Apache Beam 2.29.0 Streaming	Apache Flink 1.12.0 Batch	Apache Flink 1.12.0 Streaming	Apache Tez 0.9.1	PySpark