Adding Nodes to Onebox Deployments

To accommodate resource limitations, OneTrust recommends using the add node ability from MicroKubernetes to add more virtual machines (VM) or nodes to the cluster. For more generic information on Onebox deployments, see Onebox Deployment Guide.

Prerequisites

  • Supported:

    • Ubuntu 18.04 (Recommended)

    • RHEL 7.6-7.9; 8.1-8.5, 9.+

      • CIS RHEL 7 Level 1

      • CIS RHEL 8 Level 1

    • CentOS 7.7-7.9; 8.1-8.4,

      • CIS CentOS 7

      • CIS CentOS 8

      • CentOS Stream

  • Installed Microkubernetes, see Introduction to Microkubernetes for more information.

  • VM service account for sudo access with admin privileges

  • Latest version of kubectl, with a minimum version of 1.22

  • Firewall rule for server open inbound on port 8200. This will permit vault web UI access. You can restrict to needed source IPs.

To add a new node

Note:

For more information on this process from Microkubernetes, click here.

  1. Use the following command to create a cluster out of two or more already-running MicroKubernetes instances. The MicroKubernetes instance on which this command is run will be the master of the cluster and will host the Kubernetes control plane.
    CODE 
    microk8s add-node
  2. Use one of the following options on the MicroKubernetes instance that you wish to join to the cluster. This is NOT the node that you ran the add-node command from.
    CODE 
    From the node you wish to join to this cluster, run the following:
microk8s join 192.168.1.230:25000/92b2db237428470dc4fcfc4ebbd9dc81/2c0cb3284b05

Use the '--worker' flag to join a node as a worker not running the control plane, eg:
microk8s join 192.168.1.230:25000/92b2db237428470dc4fcfc4ebbd9dc81/2c0cb3284b05 --worker

If the node you are adding is not reachable through the default interface you can use one of the following:
microk8s join 192.168.1.230:25000/92b2db237428470dc4fcfc4ebbd9dc81/2c0cb3284b05
microk8s join 10.23.209.1:25000/92b2db237428470dc4fcfc4ebbd9dc81/2c0cb3284b05
microk8s join 172.17.0.1:25000/92b2db237428470dc4fcfc4ebbd9dc81/2c0cb3284b05
  3. Joining a node to the cluster should only take a few seconds. Afterwards you should be able to see the node has joined:
    CODE 
    microk8s kubectl get no

NAME               STATUS   ROLES    AGE   VERSION
10.22.254.79       Ready    <none>   27s   v1.15.3
ip-172-31-20-243   Ready    <none>   53s   v1.15.3
  4. When you run the get node command you will be able to get both the nodes.
    CODE 
    sudo microk8s join [internal-ip].7:25000/360f828e6b151c2c517898d5e5353531/2f2d86f558b8 --worker
Contacting cluster at 172.18.0.7

The node has joined the cluster and will appear in the nodes list in a few seconds.

This worker node gets automatically configured with the API server endpoints.
If the API servers are behind a loadbalancer please set the '--refresh-interval' to '0s' in:
    /var/snap/microk8s/current/args/apiserver-proxy
and replace the API server endpoints with the one provided by the loadbalancer in:
    /var/snap/microk8s/current/args/traefik/provider.yaml
  5. From the node you wish to join to this cluster, run the following:
    CODE 
    sudo microk8s add-node
microk8s join [internal-ip]:25000/360f828e6b151c2c517898d5e5353531/2f2d86f558b8
  6. Use the '--worker' flag to join a node as a worker not running the control plane, eg:
    CODE 
    microk8s join [internal-ip]:25000/360f828e6b151c2c517898d5e5353531/2f2d86f558b8 --worker
  7. If the node you are adding is not reachable through the default interface you can use one of the following:
    CODE 
    microk8s join [internal-ip]:25000/360f828e6b151c2c517898d5e5353531/2f2d86f558b8
sudo microk8s kubectl get nodes
NAME                  STATUS   ROLES    AGE   VERSION
autobots-multibox-1   Ready    <none>   33m   v1.28.1
autobots-multibox-2   Ready    <none>   76s   v1.28.1
  8. If you run get node command in the joined node, you will get below message:
    CODE 
    This Microkubernetes deployment is acting as a node in a cluster. Please use the microk8s kubectl on the master.

To remove a node

  1. On the node you want to remove, run the command microk8s leave.

    MicroKubernetes on the departing node will restart its own control plane and resume operations as a full single node cluster:

    CODE 
    sudo microk8s leave
Configuring services.
Generating new cluster certificates.
Waiting for node to start.
  2. After leaving the node if you run try to get nodes from master you will see a NotReady node in the master.
    CODE 
    sudo microk8s kubectl get nodes
NAME                  STATUS     ROLES    AGE   VERSION
autobots-multibox-1   Ready      <none>   46m   v1.28.1
autobots-multibox-2   NotReady   <none>   13m   v1.28.1
  3. To complete the node removal, call microk8s remove-node from the remaining nodes to indicate that the departing (unreachable now) node should be removed permanently:
    CODE 
    sudo microk8s remove-node
sudo microk8s kubectl get nodes
NAME                  STATUS   ROLES    AGE   VERSION
autobots-multibox-1   Ready    <none>   48m   v1.28.1