Number of input and output link

Number of input link	:	One
Number of output link	:	One

Debug Assembly () is used to capture and save the current stream of records into a file for debugging, validation, and troubleshooting purposes.
This assembly allows developers and data engineers to inspect intermediate data at any point in a processing pipeline without affecting the downstream data flow.

Functionality

• Receives a stream of records
• Writes records to a debug output file
• Forwards the same records unchanged to the next assembly
• Does not modify, filter, or aggregate data

How It Works

• Records arrive at the Debug Assembly
• Each record is written to a debug file (text or structured format)
• The same record is passed downstream
• Pipeline execution continues normally

Common Use Cases

• Verifying data correctness at intermediate stages
• Debugging transformation logic
• Inspecting results after filters, joins, or aggregations
• Comparing input vs output between assemblies
• Troubleshooting data quality issues

Best Practices

• Use Debug Assembly temporarily during development or testing
• Remove or disable it in production pipelines to reduce I/O overhead
• Place it:
  • After complex transformations
  • Before or after joins
  • Before sinks for output verification

Summary

The Debug Assembly provides a safe and effective way to observe and validate data at any stage of a pipeline. It records the stream to a file while allowing processing to continue uninterrupted, making it an essential tool for development, testing, and troubleshooting.

Supported Engines

Map Reduce	Spark 1.x RDD	Spark 2.x RDD	Spark 2.x Dataset	Apache Beam 2.29.0 Batch	Apache Beam 2.29.0 Streaming	Apache Flink 1.12.0 Batch	Apache Flink 1.12.0 Streaming	Apache Tez 0.9.1	PySpark