Sinks of HGrid247-3.5 DE

Sinks of a HGrid247-3.5 DE

In HGrid247-3.5 Data Engineering (DE) , Sink is an operator that functions as the final destination of the data processing process in the workflow.

After the data is read by the Source and processed by the Assembly , the final result will be stored or sent to various destination systems via the Sink.

Sinks allow processed data to be stored on various types of systems, such as:

File storage
Database
Search engine
Messaging systems
Data warehouse

Following are the various types of Sink operators provided by HGrid247-3.5.

1. File-Based Sinks

This sink is used to save the processed data into a file .

Icon	Sink	Description
	Flat File Sink	Save data to a simple text file such as CSV or TXT
	XML File Format Sink	Saving data in XML format
	Parquet File Format Sink	Saving data in XML format
	Avro File Format Sink	Store data in Apache Avro format
	ORC File Format Sink	Store data in Optimized Row Columnar (ORC) format
	Row & Column Custom Separator File Format Sink	Save files with customizable row and column separators
	Custom Format Sink	Save data in a custom format configured by the user

2. Database Sinks

This sink is used to save data into the relational and non-relational database system .

Icon	Sink	Description
	RDBMS Sink	Save data to a relational database such as MySQL, PostgreSQL, Oracle, or other
	Hive Sink	Saving data to Hive tables in a Hadoop environment
	HBase Sink	Saving data to Apache HBase NoSQL database
	MongoDB Sink	Saving data to a MongoDB database
	InfluxDB Sink	Saving data to the InfluxDB time-series database

3. Search and Indexing Sinks

This sink is used to store data to the search or indexing system .

Icon	Sink	Description
	ElasticSearch Sink	Store data to Elasticsearch for search and analysis needs
	Solr Sink	Storing data into the Apache Solr search platform

4. Streaming and Messaging Sinks

This sink is used to send data to a messaging system or streaming platform .

Icon	Sink	Description
	Kafka Sink / Producer	Producer Sending data to Apache Kafka topic
	MQTT Sink	Sending data to MQTT broker

5. Utility Sinks

Additional sinks used for debugging or system operation purposes .

Icon	Sink	Description
	Standard Output	Display data results to console or log
	Execute DDL HiveQL	Execute HiveQL DDL commands such as CREATE TABLE or DROP TABLE