Number of input and output link

Number of input link	:	One
Number of output link	:	One or more

Pipe Splitter Assembly () is used to duplicate an input data stream into two or more identical output streams.
Each output stream receives the same records with the same structure and values as the input stream.

This assembly is commonly used when the same data needs to be processed by multiple downstream pipelines in parallel.

Functionality

• Accepts one input stream
• Produces multiple output streams
• Each output stream receives an exact copy of every input record
• No transformation, filtering, or aggregation is performed

How It Works

• A record arrives at the Pipe Splitter Assembly
• The record is copied (logically, not physically duplicated in memory)
• The same record is emitted to:
  • Output Stream 1
  • Output Stream 2
  • . . .
  • Output Stream N
• Each downstream assembly processes its own copy independently

Input and Output Structure

Input Record

[field1, field2, field3, ...]

Output Records

Stream 1: [field1, field2, field3, ...]
Stream 2: [field1, field2, field3, ...]
...
Stream N: [field1, field2, field3, ...]

Example

Input Stream

OrderID	Customer	Amount
O1001	Alice           1200
O1002	Bob             800

Output Streams
Stream A (for aggregation):

OrderID	Customer	Amount
O1001	Alice           1200
O1002	Bob             800

Stream B (for auditing):

OrderID	Customer	Amount
O1001	Alice           1200
O1002	Bob             800

Key Characteristics

• No data modification
• No filtering or sorting
• No shuffle process
• Stateless operation
• Low processing overhead

Common Use Cases

• Sending the same data to:
  • Aggregation and reporting
  • Validation and auditing
  • Real-time monitoring
  • Machine learning and feature extraction
• Branching a workflow without recomputation
• Reusing a single data source for multiple pipelines

Performance Considerations

Aspect              Description
Time Complexity     O(1) per record
Memory Overhead     Minimal
Network Overhead    None
Shuffle Phase       No

Summary

The Pipe Splitter Assembly enables parallel data processing by duplicating an input stream into multiple identical output streams. It is essential for building branching workflows where the same data must be consumed by different processing paths independently.

Supported Engines

Map Reduce	Spark 1.x RDD	Spark 2.x RDD	Spark 2.x Dataset	Apache Beam 2.29.0 Batch	Apache Beam 2.29.0 Streaming	Apache Flink 1.12.0 Batch	Apache Flink 1.12.0 Streaming	Apache Tez 0.9.1	PySpark