Number of input and output link

Number of input link	:	Two
Number of output link	:	One

Join Assembly() (Reduce-Side Join) combines records from two datasets based on one or more common key fields. It functions similarly to database joins, but is designed for distributed processing across large datasets.

The first dataset connected to the Join Assembly is treated as the left-side dataset, while the second dataset is treated as the right-side dataset.

Recommendation:
For better performance, the right-side dataset should be smaller than the left-side dataset.

The shuffle process also occurs in this assembly.

Shuffle Process in Join

During a join operation, records with the same join key from both datasets must be brought together. This requires a shuffle process that includes:

• Partitioning
  Ensures records with the same join key are sent to the same reducer or task.
• Data Transfer
  Moves data between nodes across the cluster.
• Key-Based Matching
  Matches left-side and right-side records based on the join fields.

Note:
Joins can be expensive operations due to shuffle overhead, especially when datasets are large or unevenly distributed.

Parameters of Join Assembly

• Joiner
  Specifies the join method.
  Supported join types:
  • Inner Join
  • Left Join
  • Right Join
  • Outer Join
• Join Field
  Specifies the field(s) used to join the left-side and right-side datasets.
  Defined by drawing links between corresponding fields.
  Mandatory parameter
  Supports multiple join fields
• Partitioning Class
  Specifies the partitioning strategy used during the shuffle process.
  Applicable for Hadoop MapReduce and Spark RDD engines
  
  Available Partitioning Classes
  
  • Default Partitioning
    Uses the default hash-based partitioning.
  • cascading.solusi247.partition.AlphabetPartitioning
    Partitions data based on the first character of the group value, interpreted as an alphabet.
  • cascading.solusi247.partition.AlphaNumPartitioning
    Partitions data based on the first character of the group value, interpreted as alphanumeric.
  • cascading.solusi247.partition.RangeTextPartitioning
    Partitions data based on defined beginning text ranges.
    Example:

    A-Z,a-z
    

    • Uppercase values → partition 0
    • Lowercase values → partition 1
    • Values outside the range → partition 0
  • cascading.solusi247.partition.RangeNumberPartitioning
    Partitions data based on numeric ranges.
    Example:

    0-1000,1001-10000
    

    • 0–1000 → partition 0
    • 1001–10000 → partition 1
    • Values outside the range → partition 0
• Partition Parameter
  Specifies additional parameters for the selected partitioning class.
  Required when using:
  • RangeTextPartitioning
  • RangeNumberPartitioning
  Not required for:
  • Default Partitioning

Behavior

• Records from both datasets are matched using the specified Join Field(s)
• Join behavior depends on the selected Joiner type
• Shuffle is automatically triggered to align matching keys
• Output records include combined fields from both datasets
• Unmatched records are handled according to the join type

Typical Use Cases

• Enriching transactional data with reference data
• Combining fact and dimension datasets
• Correlating logs with metadata
• Implementing relational joins in distributed ETL pipelines

Performance Notes

• Keep the right-side dataset as small as possible
• Avoid data skew on join keys
• Choose an appropriate partitioning strategy to reduce shuffle overhead
• Consider pre-filtering datasets before joining

Supported Engines

Map Reduce	Spark 1.x RDD	Spark 2.x RDD	Spark 2.x Dataset	Apache Beam 2.29.0 Batch	Apache Beam 2.29.0 Streaming	Apache Flink 1.12.0 Batch	Apache Flink 1.12.0 Streaming	Apache Tez 0.9.1	PySpark