AWS Elastic Kubernetes Service (EKS) Deployment Guide

This document outlines the steps required to install and configure the OneTrust Data Discovery worker node for Amazon Elastic Kubernetes Service (EKS) environments. Currently, this installation is only supported on Mac-OS and Linux systems. Users may have to install a Linux instance on their Windows machine if they lack access to Mac-OS.

Prerequisites

  • AWS EKS 1.33.x with the following:

    • 5 x m5a.2xlarge (8 vCPU(s), 32 GB RAM) nodes

    • AWS subscription which can run the required cluster.

    • An owner/contributor user of the AWS subscription which will be used to create and manage the cluster.

    • Virtual Network and subnet in which the AWS cluster is getting deployed should have access to data sources.

    • Virtual Network and subnet in which the AWS cluster is getting deployed should have access to Internet to reach Docker Hub.

  • AWS EFS instance which can access by AWS EKS worker nodes.

  • A local (laptop) or remote (Jump box/bastion) machine which will be used to access the cluster master.

    • Minimum: 4 CPU, 8GB RAM, 250 GB HDD, Ubuntu 16+

    • Should have access to Internet to pull the OneTrust Docker Images from docker hub.

    • Docker >=19.x ce + pre-installed (OneTrust can also install this)

  • Since the parameter AllowVolumeExpansion is unset by default, you may need to set as true when upgrading or re-sizing storage. See Setting AllowVolumeExpansion for more information.

  • Latest version of kubectl, with a minimum version of 1.30

  • OIDC Provider, as your cluster has an associated OpenID Connect (OIDC) issuer URL. To use AWS Identity and Access Management (IAM) roles for service accounts, an IAM OIDC provider must exist for your cluster's OIDC issuer URL. See here for more information.

  • AWS EFS CSI Driver installed, if using AWS EFS filesystem. See here for more information and refer to AWS Worker Node Provisioning Migration Guide.

  • EBS CSI Driver installed, as GP2 storage class in Kubernetes uses EBS volume for provisioning during installation. See here for more information.

Adding EFS Storage

You'll need access to an EFS instance in the same VPC and all the subnets where the worker nodes can arise. Ensure that the worker security group (SG) is selected.

  1. Log in to the AWS Management Console.

  2. Search for EFS and select Create File System.



  3. Select the new EKS (not default) VPC from the drop-down list. Then complete the following:

    • Under Security Groups, select the same Security Group that is associated with your EKS cluster. You must also ensure that the inbound/outbound rules of this Security Group allow for traffic flow between the two services.

    • Enter a name. This can be any name.

    • Ignore step 3.

    • Click Create File System.



  4. Copy the following values for reference during deployment in step 12 of On-Premises Deployment.

    • File System ID

    • Region

    • DNS Name

      Note:

      This will actually be <file-system-ID>.efs.<region>.amazonaws.com.



On-Premises Deployment

Customer will need to provide the following during AWS deployment:

  • Access Key

  • Secret Key

  • Password

Note:

Customers can also provide their AWS instance profile information in lieu of their Access Key and Secret Key IDs. You will need to update the .envs-aws file with the following variable:

CODE 
IS_AWS_INSTANCE_PROFILE=”true”

Enter "false" to carry on with the default process for this deployment.

  1. Download the On-Premise installer file package ot-onprem-installer-dd-k8s-latest.zip. You will need to unzip and place it in a location on your Linux VM.

    Note:

    Contact your dedicated OneTrust contact or OneTrust support to acquire this file.

  2. Navigate to data-discovery/k8s-deployment

  3. Copy the folder inventory_template to cp -r inventory_template inventory

  4. Change the directory to data-discovery/k8s-deployment/inventory

  5. Run the command nano .envs to add the details.

  6. Enter the namespace of your choice in the Namespace configuration section.

    CODE 
    export OT_NAMESPACE_DD="<your-name-space>"

  7. Enter the Docker configuration details.

  8. Provide the storage name.

    CODE 
    export STORAGE_CLASS_NAME="gp2"

  9. Configure the scanner by adding the following:

    • URL

    • Worker Node ID

    • Client Credentials

  10. Provide the scanner credential information.

  11. If using shared storage configuration, provide:

    CODE 
    export EXTLIB_SHARED_STORAGE_NAME='efs-sc'
export CHOWN_INIT_ENABLED='false'

  12. Configure EFS with the following:

    CODE 
    export EFS_FS_ID=""
export EFS_REGION=""
export EFS_DNS_NAME=""  
    Note:

    For DNS name, use <EFS_FS_ID>.efs.<efsregion>.amazonaws.com

  13. Save and close the file.

  14. Run nano. env-aws and update the following fields:

    CODE 
    export CLOUD_AWS_REGION="ap-south-1"
export CLOUD_AWS_EKS_CLUSTER="devops-cluster"

    If using Instance Profile, provide the following:

    CODE 
    export IS_AWS_INSTANCE_PROFILE="true"

  15. Save and close the file.

  16. Open a terminal in data-discovery/k8s-deployment directory.

  17. Run the deployment script:

    CODE 
    ./deploy.sh install efs-new

  18. Confirm the configuration by pressing 'Y'.

  19. Access your AWS instance and provide the following:

    • AWS Access Key ID

    • AWS Secret Access Key

    • Default region name

    • Default output format

    Note:

    For instructions on using AWS instance profiles as credential information, refer to the beginning of this section.

  20. The installation will complete.

Setting AllowVolumeExpansion

  1. Run the following command to export the yml file of the gp2 storage class. The storage.yml file will be created in the current directory.

    CODE 
    kubectl get sc gp2 -o yaml > storage.yml

  2. Edit the storage.yml file and add the below value to the yml file.

    CODE 
    allowVolumeExpansion: true

  3. Save the yml file and run the below command:

    CODE 
    kubectl apply -f storage.yml

  4. Then verify the AllowVolumeExpansion as set to true by describing the storage class:

    CODE 
    kubectly describe sc gp2