IPM Horizontal Scaling - Setup and Commands

This article covers the required setup steps and available commands for In-Place Masking (IPM) Horizontal Scaling.

Prerequisites

1. G-Repository Client 

• Must be created and configured on the IPM Server and all IPM Client machines.
• See G-Repository Overview for more information. 

2. KafkaConfig Folder 

This folder should be present in all IPM machines (Server & Clients). Complete these steps: 

• Download KafkaConfig.zip from this link.
• Extract the folder in your working directory.
• Set environment variable KAFKA_HOME with the path of the KafkaConfig folder.

Example: For Linux system, if you extract KafkaConfig.zip to /home/JohnDoe, update the .bashrc file as follows:

export KAFKA_HOME="/home/JohnDoe/KafkaConfig"

Make sure the GEN_ROCKET_HOME environment variable is defined with an export statement at the beginning.

Example:

export GEN_ROCKET_HOME="/home/JohnDoe/GenRocket"

3. GenRocket Runtime 

• The same version should be present on all IPM machines (Server & Clients).
• Recommended to use the latest version.
• See GenRocket Runtime Overview and How to Update GenRocket Runtime for more information.

4. Supported Java Version

• Supported Java Version for IPM are: Java 8, 11, 17 and 21.

5. Permission for Database User

• Permissions as mentioned in the Database Permissions article.

6. IPM Server Config.properties File

• Create IPM-server-config.properties file on IPM Server. 

Example: 

kafkaNodeId=1
serverIpAddress=localhost
kafkaPort=9092
kafkaControllerPort=9093
createCustomerOffsetTopic=true
logDir=<working-directory>/IPMLogs
kafkaLogDir=<working-directory>/IPM/KafkaLogDir

File Parameters

The following parameters should be included in the IPM-server-config.properties file: 

• kafkaNodeId - Defines a number that acts like a unique ID tag for each Kafka server in the cluster, so Kafka can recognize and distinguish one node from another.
• serverIpAddress - Defines the hostname or IP address of the Kafka server.
• kafkaPort - Defines the unique port number used by the Kafka service.
• kafkaControllerPort - Defines the unique port used by Kafka for internal controller communication.
• createCustomerOffsetTopic - Determines if a customer offset topic is created. The default value is true.
• logDir - Defines the directory for storing progress logs from IPM Clients. The generated log files are named with date format <MM-DD-YYYY>.log.  Example: logDir=/home/JohnDoe/IPMLogs
• kafkaLogDir - Defines the directory path for storing Kafka’s internal logs

Deploying IPM Clients on Multiple Different Machines

If there is a requirement to deploy the IPM Server on Machine-1 and run IPM Clients on multiple different machines (e.g., IPM Client 1 → Machine-2, IPM Client 2 → Machine-3, and so on), follow these steps:

• Update the IPM-server-config.properties file:
  • Set serverIpAddress = <IP / Host Name of Server where IPM Server running>.
• Ensure that the port numbers defined in kafkaPort and kafkaControllerPort in the IPM-server-config.properties file are open on the IPM Server so that all IPM Clients can connect successfully.
• From each IPM Client machine, verify connectivity to the IPM Server using:

telnet <IP / Host Name of Server where IPM Server running> <kafkaPort number>

Eg:

telnet 192.168.1.1 9092

Recommendations

1. Begin with masking one IPM table on a single IPM Client to resolve any setup and permission-related challenges.
2. Execute multiple IPM Clients on a single machine. Upon successful execution, the IPM Clients can then be scaled to run across multiple machines.
3. Ensure no database or machine-level changes are performed during IPM horizontal scaling validations.
4. Validate IPM Horizontal Scaling on small or medium sized tables and then move to larger ones.

Setup Instructions

Before initiating an IPM Server, verify that no Kafka process is currently running. If an active Kafka process is detected, terminate it before proceeding with the server startup.

Command to check if a Kafka process is running:

ps -eaf | grep -y kafka

If process is running, terminate it using its PID (Process ID):

kill <PID>

Then follow the steps below using the commands in the next section to launch the IPM Server, launch the IPM Client(s), monitor execution progress, etc. 

1. Launch IPM Server
2. Launch IPM Client
3. Monitor an IPM Client execution progress from IPM Server
4. Shutdown IPM Server
5. Ready for next batch execution

Commands

Launch IPM Server

genrocket -gipms <working-directory>/IPM-server-config.properties -groupName <working-directory>/IPMGroupTableConfig.json

The command will:

1. Start the IPM Server using IPM-server-config.properties and create the Kafka Topics required for communication between the IPM Server and IPM Client(s).
2. Process the IPMGroupTableConfig.json, which contains a list of config names and corresponding G-Repository client names to perform IPM operation
3. Schedule tasks to the available IPM Clients.

IPMGroupTableConfig.json
{
 "configs": [
    {
     "configName": "<CONFIG-NAME>",
     "grepo": "<REPOSITORY CLIENT NAME>"
    }
  ]
}

The CONFIG-NAME can be identified as shown in the screenshot below: 

Example:

genrocket -gipms /home/JohnDoe/config/IPM-server-config.properties -groupName /home/JohnDoe/config/IPMGroupTableConfig.json

IPMGroupTableConfig.json file

{
 "configs": [
    {
     "configName": "IPMConfig.User",
     "grepo": "IPMOracleRepo"
    },
    {
     "configName": "IPMConfig.Customer",
     "grepo": "IPMOracleRepo"
    },
    {
     "configName": "IPMConfig.CustomerAccountDetail",
     "grepo": "IPMOracleRepo"
    }
  ]
}

We have the IPMGroupTableConfig.json file with 3 configName entries to execute.

As per the command shown in the above screenshot:

• The IPM Server starts with PID 62671.
• The IPM Server is waiting to schedule the 3 configs to the available IPM Clients.

After the IPM Server has started successfully, as shown in the screenshot above, proceed to the next steps to launch the IPM Clients.

Launch IPM Client

The following command starts an IPM client and uses a Kafka Topic to communicate with IPM Server for IPM tasks.

genrocket -gipmc -configName <working-directory>/IPM-server-config.properties 

Example:

genrocket -gipmc -configName /home/JohnDoe/config/IPM-server-config.properties

1. IPM Client starts and becomes available for registration
2. IPM Client registers itself by communicating with the IPM Server
3. IPM Server assigns a unique client number to the IPM Client
4. IPM Client notifies the IPM Server that it is available and ready to receive IPM tasks
5. IPM Server assigns an IPM task to the IPM Client
6. IPM Client starts executing the assigned task
7. During execution, IPM Client continuously reports its progress/status back to the IPM Server
8. IPM Client connects to the gen_rocket_ipm_<index_number> schema containing IPM tables, stored procedures, and functions

As shown in the screenshot above, the IPM Client is now up and ready to consume requests from the IPM Server.

Add new Tasks In IPM Server Queue 

The following command adds a new IPM task to the IPM Server Queue:

genrocket -gipms <working-directory>/IPM-server-config.properties -gipmQueueAdd <working-directory>/IPMGroupTableConfig.json

Example:

genrocket -gipms /home/JohnDoe/config/IPM-server-config.properties -gipmQueueAdd /home/JohnDoe/config/IPMGroupTableConfig.json

IPMGroupTableConfig.json file
{
 "configs": [
    {
     "configName": "IPMConfig.User",
     "grepo": "IPMOracleRepo"
    },
    {
     "configName": "IPMConfig.Customer",
     "grepo": "IPMOracleRepo"
    },
    {
     "configName": "IPMConfig.CustomerAccountDetail",
     "grepo": "IPMOracleRepo"
    }
  ]
}.

Remove Tasks from IPM Server Queue

The following command removes an existing IPM task from the IPM Server Queue

genrocket -gipms <working-directory>/IPM-server-config.properties -gipmQueueRemove <IDs comma seperated>

Example:

genrocket -gipms /home/JohnDoe/config/IPM-server-config.properties -gipmQueueRemove 1,2,3

Shutdown IPM Server

The following command terminates the running IPM Server and Client processes.

Shut Down IPM Server and all Clients only:

genrocket -gipmShutdown <working-directory>/IPM-server-config.properties

Shutdown the IPM Server and all Clients, and delete all associated tasks.

genrocket -gipmShutdown <working-directory>/IPM-server-config.properties -clearTasks

Example:

genrocket -gipmShutdown /home/JohnDoe/config/IPM-server-config.properties

View IPM Server Task History 

The following command displays the execution history of the IPM tasks. This command is to be executed on the machine where IPM Server is running.

genrocket -gipmsHistory <working-directory>/IPM-server-config.properties

Example:

genrocket -gipmsHistory /home/JohnDoe/config/IPM-server-config.properties

The above command will display the IPM task execution history.

As shown in the screenshot above, the IPM task execution history is displayed.

Monitor an IPM Client execution progress from an IPM Server

The following command checks the logs for a specific IPM Client by filtering entries for its Kafka topic.

The logs are stored in the directory specified by the logDir property in the IPM-server-config.properties file.

grep -i 'Source\*\*\*IPM-Client-<N>'  <LogDir>/<MM-DD-YYYY>.log

Example:

grep -i 'Source\*\*\*IPM-Client-1' /home/JohnDoe/IPMLogs/07-25-2025.log

The above command displays the logs for IPM Client 1, as shown in the screenshot below.

Steps to increase the number of IPM Clients

Following the Database Permissions article: 

• Create additional schemas following the GenRocket defined convention: gen_rocket_ipm_N.
  • Each IPM client utilizes a dedicated database schema for storing temporary Stored Procedures, Functions, and the GenRocket Tables. Therefore, five running IPM clients require five separate schemas to execute five IPM tasks in parallel.
  • For Oracle, create additional users corresponding to the newly created schemas.
  • For Oracle, additional Database property files must be created using the convention gen_rocket_ipm_N.properties, each containing the user of the corresponding schema.
  • The new Database properties file should be saved in the same directory as the target table connection properties file.
• For SQL Server, the existing database user must have access to the new schemas in order to create temporary Stored Procedures, Functions, and the GenRocket Tables, as specified in the Database Permissions article.
• Launch IPM server & IPM clients.