HBase Sink () is used to write data from the pipeline into the HBase database .

Apache HBase is a column-family based NoSQL database that runs on top of the Apache Hadoop ecosystem . HBase is typically used for:

• large-scale data storage
• real-time read/write
• semi-structured data storage
• big data analytics

Unlike relational databases, HBase uses the concept of:

• Row Key
• Column Family
• Column Qualifier

Parameters of HBase Sink

1. Table Name
   Specifies the name of the table in HBase where the data will be stored.
2. Incoming Fields
   Specifies fields that come from the previous assembly in the pipeline .
   Example fields:

   MSISDN, LAC, CI, EVENT_DATE, DIRECTION
   

   This field is the data that will be entered into HBase .
3. Row ID From
   Determines the field that will be used as the Row Key (Row ID) .
   Row Key is a unique identifier for each row in HBase .
   Example:

       Row ID From = MSISDN
   

   This means that the MSISDN value will be the Row Key in the HBase table.
   Example structure:

       Row Key         Column Family	Column	Value
       -------------------------------------------------
       62812345678     telecom             lac	1126
       62812345678     telecom             ci	42313
   

4. Column Family
   Specifies the Column Family where the data is stored .
   In HBase, every column must be in a column family .
   If there are multiple column families, use the # sign as a separator.
   Example:

       Column Family = telecom#location
   

   This means there are two column families:
   • telecom
   • location
5. Column Qualifier
   Specifies the column names in the column family .
   If there are multiple columns, use a comma (,) as a separator.
   Example:

       Column Qualifier = lac,ci,event_date,direction
   

   The HBase structure that is formed:

       Row Key         Column Family	Column Qualifier	Value
       --------------------------------------------------------------------
       62812345678     telecom             lac                     1126
       62812345678     telecom             ci                      42313
       62812345678     telecom             event_date              2025-03-06
       62812345678     telecom             direction               ggsn
   

HBase Data Structure Example

For example configuration:

    Table Name = telecom_events
    Row ID From = MSISDN
    Column Family = telecom
    Column Qualifier = lac,ci,event_date,direction

So the table structure in HBase:

    RowKey: 62812345678
       telecom:lac = 1126
       telecom:ci = 42313
       telecom:event_date = 2025-03-06
       telecom:direction = ggsn

Supported Engines

Map Reduce	Spark 1.x RDD	Spark 2.x RDD	Spark 2.x Dataset	Apache Beam 2.29.0 Batch	Apache Beam 2.29.0 Streaming	Apache Flink 1.12.0 Batch	Apache Flink 1.12.0 Streaming	Apache Tez 0.9.1	PySpark