|
|
HGrid247-3.5 DE User Guide |
What is HGrid247-3.5 DE ?
HGrid247-3.5 DE is a multi-platform, drag-and-drop big data engineering ETL tool designed for both batch and stream processing.
HGrid247-3.5 DE enables users to design data engineering workflows easily using an intuitive graphical interface. It automatically generates optimized Java (and PySpark) code that can be executed on parallel and distributed processing frameworks. As a result, users do not need advanced programming skills to build scalable data processing pipelines.Key Capabilities
- Visual drag-and-drop workflow design
- Automatic code generation
- Support for batch and streaming workloads
- Multiple distributed processing engines as build targets
- Reduced dependency on manual coding
Supported Processing Engines and Build Outputs
HGrid247-3.5 DE supports ten processing engines as compilation and build targets:
- Hadoop MapReduce
- Output file:
<project-name>_mr.jar- Apache Spark 1.6 RDD
- Output file:
<project-name>_spark1.jar- Apache Spark 2.x RDD
- Output file:
<project-name>_spark2.jar- Apache Spark 2.x (Dataset)
- Output file:
<project-name>_spark2ds.jar- Apache Beam 2.29.0 (Batch)
- Output file:
<project-name>_beam.jar
- Apache Beam 2.29.0 (Streaming)
- Output file:
<project-name>_sbeam.jar- Apache Flink 1.12.x (Batch)
- Output file:
<project-name>_flink.jar- Apache Flink 1.12.x (Streaming)
- Output file:
<project-name>_sflink.jar- Apache Tez 0.9.1
- Output file:
<project-name>_tez.jar- Apache PySpark
- Output file:
<flow-name>.pySummary
HGrid247-3.5 DE provides a unified, low-code platform for building scalable data engineering pipelines across multiple big data engines, making it suitable for developers, data engineers, and analysts who want powerful processing capabilities without complex programming requirements.