MongoDB Sink () is used to store or insert data into a MongoDB database . MongoDB is a document-based NoSQL database that stores data in a JSON-like (BSON) format , making it ideal for Big Data applications, real-time systems, and applications that require data schema flexibility .

In HGrid247 pipeline , MongoDB Sink is usually used as the final destination (data storage) after data is processed or transformed from various sources such as Kafka, APIs, or streaming data.

Parameters of MongoDB Sink

1. Host Name (IP)
   MongoDB server hostname or IP address
   This parameter specifies the address of the MongoDB server that will receive the data.
2. Port Number
   Port server MongoDB
   Specifies the MongoDB server port .
3. Data Base Name
   Nama database MongoDB
   This parameter specifies the name of the MongoDB database where the data will be stored.
4. Collection Name
   The name of the collection where the data is stored
   A collection is a collection of documents in MongoDB , similar to a table in a relational database .
5. User Name
   MongoDB account username
   MongoDB account username used for authentication.
6. Password
   MongoDB account password
   Password of MongoDB account.
7. Connection Option
   Additional MongoDB connection options
   This parameter is used to add additional connection options to MongoDB.
   Example:

       Connection Option : retryWrites=true&w=majority
   

   This option is usually used for:
   • connection optimization
   • replication settings
   • security settings
8. Batch Size
   Number of documents entered in one batch
   Specifies the number of documents inserted into MongoDB in a single batch .
   Batch processing helps improve the performance of writing large amounts of data .
   Example:

       Batch Size : 1000
   

   This means that every 1000 documents will be sent at once to MongoDB .
9. SSL Enabled
   Enable or disable SSL connection
   This parameter determines whether the connection to MongoDB uses SSL (Secure Socket Layer) or not.
10. Field Names
    The name of the field to be stored in the MongoDB document
    This parameter specifies the name of the field in the MongoDB document to be saved.
    Example:

        Field Names : user_id,transaction_id,amount,timestamp
                

11. Field Types
    Data type of each field
    Determine the data type of each field .
    Example:

        Field Types : String,String,Double,Long
    

    Mapping example:

        Field               Type
        -------------------------------
        user_id             String
        transaction_id	String
        amount              Double
        timestamp           Long
    

Supported Engines

Map Reduce (Tested on hadoop 2.7.3 MongoDB 6.0)	Spark 1.x RDD	Spark 2.x RDD (Tested on spark 3.0.1 MongoDB 6.0)	Spark 2.x Dataset (Tested on spark 3.0.1 MongoDB 6.0)	Apache Beam 2.29.0 Batch (Tested on MongoDB 6.0)	Apache Beam 2.29.0 Streaming (Tested on MongoDB 6.0)	Apache Flink 1.12.x Batch (Tested on MongoDB 6.0)	Apache Flink 1.12.x Streaming (Tested on MongoDB 6.0)	Apache Tez 0.9.1	PySpark

---