HGrid247-3.5 provides a plug-in feature that allows users to create custom operations or functions to expand the system's data processing capabilities.

Plug-ins are used when the standard operators available in HGrid247 are not sufficient to meet specific data processing needs.

Plug-ins on HGrid247 can be created using two approaches:

• Java-based plug-ins
• Python-based plug-ins

1. Java-Based Plug-ins

Java-based plug-ins are used to create more complex and high-performance data processing functions .

To create a Java plug-in, the following libraries are required:

    HGrid247-3.5/
        └──  libCommon/
               └──  HGrid247CommonLib-3.5.jar

This library provides an API used to develop plug-ins for HGrid247.

Types of Java Plug-ins

1. UDF Operation / Function
   
   Used to create custom functions that work on each data record .
   
   This plug-in is used in the assembly:

       UDF Function / Operation ()
   

   Usage example:
   • text normalization
   • special value calculation
   • data validation
2. UDF Buffer Operation
   
   Used to process grouped data .
   
   This plug-in is used in the assembly:

       UDF Buffer ()
   

   Usually used for:
   • data analysis in groups
   • batch processing in groups
3. Buffer Transformation Function
   
   Used to perform transformations on data that has been grouped and sorted .
   
   Assembly used:

       Buffer Transformation ()
   

4. Custom Transformation Function
   
   HGrid247 provides classes:

       UDFTransform247()
   

   which can be used to create custom transformation functions .
   
   This function can be used in assembly:

       Transformator ()
       Transformator2 ()
   

   Usage example:
   • string manipulation
   • data type conversion
   • custom data parsing
5. Custom Aggregation Function
   
   To create custom aggregation functions, HGrid247 provides the following classes:

       UDFAggregation247()
   

   This function is used in assembly:

       Aggregator ()
   

   Custom aggregation example:
   • median
   • weighted average
   • custom scoring
6. UDF Stream Source
   
   Used to create custom streaming data sources .
   
   Assembly used:

       UDFStreamSource ()
   

   Usage example:
   • reading data from a special system
   • read stream from a specific API
8. UDF Event Time Assigner
   
   Used to determine event time in streaming processing.
   
   Assembly used:

       EventTimeAssigner()
   

   Usually used in:
   • event processing
   • streaming analytics
9. SwitchCase Custom Function
   
   Users can also create custom functions in:

       SwitchCase Transformation Function ()
   

   used in assembly:

       Transformator ()
       Transformator2 ()
   

   This function is used to create complex conditional logic .

2. Python-Based Plug-ins

Besides Java, HGrid247 also supports Python -based plug-ins .

Python plug-ins use:

    Jython 2.7.3

Because it uses Jython, there are some limitations.

Python modules that can be used

• pure Python modules

Unusable Python modules

• modules implemented using the C language

Examples of modules that cannot be used:

• NumPy
• SciPy
• Pandas

Types of Python Plug-ins

1. UDF Python Transformation Function()
   
   Used to create transformation functions using Python.
   
   Used in assembly:

       Transformator ()
       Transformator2 ()
   

   Usage example:
   • text manipulation
   • parsing data
   • field transformation
2. UDF Python Operation / Function
   
   Used to create Python-based data processing functions .
   
   Assembly used:

       UDFPythonFunction / Operation ()
   

3. UDF Python Aggregation ()
   
   Used to create aggregation functions using Python .
   
   Assembly used:

       Aggregator ()
   

   Example:
   • custom metric
   • special statistical calculations
4. UDF Python Buffer Operation
   
   Used to process data in group form (grouped data) .
   
   Assembly used:

       UDFBuffer ()
   

5. UDF Python Buffer Transformation
   
   Used to perform transformations on data that has been grouped using Python.
   
   Assembly used:

       Buffer Transformation ()
   

Java vs Python Plug-ins

    Feature             Java Plug-in            Python Plug-in
    -------------------------------------------------------------
    Performance         Faster                  More flexible
    Library support	Java ecosystem          Python ecosystem
    Complexity          More complex            Easier to write
    Integration         Native with HGrid247	Via Jython