Migrating Worker Nodes from Zookeeper to KRaft

Switching from Zookeeper to KRaft (Kafka Raft) simplifies your architecture by eliminating the need for an external Zookeeper cluster. With KRaft, you benefit from faster setup and easier maintenance. Additionally, Zookeeper may be deprecated soon, making KRaft a better choice.

Prerequisites

  • Ensure no scans are in progress on the worker node.

  • Confirm that the worker node is at least version 202503.1.0 and in a healthy state. If not upgrade the worker node to latest version.

  • Verify if the worker node is running Kafka in Zookeeper mode; both Kafka and Zookeeper should be running.

  • If the worker node has Kafka and a Kafka controller but is missing Zookeeper, it has already migrated to or is set up in KRaft mode. No action is needed.

About OneTrust 202504.1.0

Upgrading to version 202504.1.0, whether via the platform or terminal, enforces IS_KRAFT_MIGRATION_MODE to 'true'. This will automatically upgrade and migrate worker nodes during the worker node upgrade cycle from Kafka Zookeeper mode to KRaft mode. The migration has been tested and supports ongoing scans, meaning scans do not need to be paused or canceled for migration to occur.

If you experience upgrade errors or downed worker nodes due to migration errors, use the script, set_wn_kraft_mode, from the Administration tab in the worker node's details screen. This script will enforce KRaft mode, skip migration, delete Zookeeper, and reinstall the worker node in KRaft mode. For more details, see Script Management for Worker Nodes.

Important:

Prior to using the script, ensure there are no active scan jobs in progress or paused (intended to resume later), as they will not work during this process. It is recommended to cancel the ongoing scan or run the script regardless if the scan is minimal and/or can be lost.

To enable KRaft mode

  1. Navigate to Configurations tab in the worker node's details screen.
  2. Expand the Custom Options.
  3. Enable Feature Properties.
  4. For the Feature Variable field, select Migrate Kafka to KRaft Mode.
  5. Set the Value field to true.


  6. Click the Save button.
  7. Update the worker node to activate the change.

To check migration

Worker Node Logs

  1. Navigate to the Jobs tab in the worker node's details screen.
  2. Check the worker node upgrade job Pod logs for Kafka migration-related entries.
  3. The log should state Migration to KRAft is completed.


Audit Activity

  1. Navigate to the Activity tab in the worker node's details screen.
  2. Review the audit log after a successful migration.


Healthy Pod Check

  1. Navigate to the Services tab in the worker node's details screen.
  2. Ensure the Kafka and Kafka-controller Pods are running healthily. After a successful KRaft migration, the Zookeeper pod will be deleted and should no longer be present.

Post Migration Check

Run a Quick Catalog and Scan on any data source linked to the worker node. The scan should complete as expected.

Troubleshooting

If the migration fails to run from the OneTrust application, you can attempt the installation from the terminal.

  • Ensure 'Enable KRaft Mode' is enabled and the configuration is saved.

  • Obtain the latest curl package and run the installation from the terminal. If it still fails, follow these steps:

  1. Delete the Kafka and Zookeeper StatefulSet, PVC, and PV.

  2. Remove any Kafka controller-related pods and PVCs.

  3. Ensure no scans are in progress.

  4. Set IS_KRAFT_MODE_ENABLED to true and IS_KRAFT_MIGRATION_MODE to false in inventory/.guided-envs.

  5. Run ./deploy.sh install.

  6. This will reinstall the worker node's Kafka in KRaft mode. Contact OneTrust Support if you encounter further issues or have questions.