RDBMS Sink () is used to save pipeline result data to a relational database (RDBMS) using JDBC .

RDBMS Sink allows the system to write data directly to the database such as:

• MySQL
• PostgreSQL
• Oracle Database
• Microsoft SQL Server

JDBC Sink is typically used when the results of data processing need to be stored in a database table for:

• report
• dashboard
• business applications
• data analysis

Parameter RDBMS Sink

1. Driver ClassName
   Specifies the name of the JDBC driver class used to connect to the database.
   Driver examples:
   
   MySQL

       com.mysql.jdbc.Driver
           

   PostgreSQL

       org.postgresql.Driver
           

   Oracle

       oracle.jdbc.driver.OracleDriver
           

   Microsoft SQL Server

       com.microsoft.sqlserver.jdbc.SQLServerDriver. 
           

2. Connection Url
   Specifies the JDBC connection address to the database .
   General format:

       jdbc:database_type://host:port/database_name
   

   Example:
   
   MySQL

       jdbc:mysql://localhost:3306/telecomdb
   

   PostgreSQL

       jdbc:postgresql://localhost:5432/telecomdb
   

   Oracle

       jdbc:oracle:thin:@localhost:1521:ORCLCDB
           

   Microsoft SQL Server

       jdbc:sqlserver://localhost:1433;databaseName=MyDatabase;user=myUser;password=myPassword;
           

3. Username
   The username of the RDBMS account.
4. Password
   The password of the RDBMS account.
5. Batch Size
   Determines the amount of data written to the database in a single batch operation .
   Example:
   

       Batch Size = 1000
                   

   It means:
   • every 1000 records
   • the system performs batch insert/update
   Batch advantages:
   • faster
   • reduce the number of queries to the database
6. Table Name
   The name of the table where data is to be inserted or updated.
7. Sink Mode
   Determines how data is written to the database .
   There are three modes:
   • KEEP
     Data is only written if it does not already exist in the table .
     Existing data will not be changed.
   • INSERT
     Data is always added as a new row .
   • UPDATE
     If the data already exists (based on the primary key), then the data will be updated
8. Incoming Fields
   Specifies the fields from the previous assembly that will be written to the database.
   This field comes from the previous pipeline.
9. Table Columns
   Determine the database table columns to be filled .
   Mapping between pipeline fields and table columns is done here.
10. Columns Definition
    Used to define the column structure of a table if the table does not exist yet.
    This parameter is required if the table does not exist .
11. Primary Key
    Determine the primary key of the table .
    Primary key is used for:
    • ensure data uniqueness
    • supports UPDATE mode
    If a new table is created, a primary key must be defined.

Supported Engines

Map Reduce	Spark 1.x RDD	Spark 2.x RDD	Spark 2.x Dataset	Apache Beam 2.29.0 Batch	Apache Beam 2.29.0 Streaming	Apache Flink 1.12.0 Batch	Apache Flink 1.12.0 Streaming	Apache Tez 0.9.1	PySpark