ElasticSearch Source () reads data from an Elasticsearch cluster using REST-based queries.
It allows batch or streaming ingestion of indexed documents into an HGrid247 workflow.

Parameters of ElasticSearch Source

• Protocol
  Protocol type used to connect to ElasticSearch
  • Default value: http
  • Common values: http, https
• Host/IP
  Host name or IP address of the ElasticSearch server
  Example: localhost, 10.10.10.5
• Port Number
  Port number on which the ElasticSearch service runs.
  Default: 9200
  Example: localhost, 10.10.10.5
• Index Name
  Name of the ElasticSearch index to be queried
  Example: logs_2025, sales_index
• Type Name
  Type name of ElasticSearch
  Used for older ElasticSearch versions
  For newer versions, this may be optional or set to _doc
• Query
  ElasticSearch query statement
  Can be a Query DSL JSON string
  Used to filter, search, or match documents
  Example:

  {
    "query": {
      "match": {
        "status": "ERROR"
      }
    }
  }

• Basic Authentication
  
  • User Name
    User name for HTTP basic authentication
  • Password
    Password for HTTP basic authentication
• SSL Authentication :
  
  • JKS Path
    Path to the Java Key Store (JKS) file used for SSL authentication
    Example: /opt/security/es-truststore.jks
  • SSL Password
    Password for the SSL Key Store

Supported Engines

Map Reduce	Spark 1.x RDD	Spark 2.x RDD	Spark 2.x Dataset	Apache Beam 2.29.0 Batch	Apache Beam 2.29.0 Streaming	Apache Flink 1.12.0 Batch	Apache Flink 1.12.0 Streaming	Apache Tez 0.9.1	PySpark