HashiCorp Key Vault Configuration

This article includes guidance on using HashiCorp vault options as your preferred credential store during worker node creation. For additional information on this process, see the following: Guided Worker Node Installation Overview and Credential Vault Options for Data Sources.

Vault Configuration for HashiCorp Vault - Open Source (Internal)

Note:

Refer to official documentation from AWS Secret Manager, Azure Key Vault, CyberArk - Conjur, Google Cloud, and HashiCorp Vault - Enterprise for guidance on the respective practices for credential vault configuration.

  1. Port forward the installed vault port to your local machine port using the following command:

    CODE 
    microk8s kubectl port-forward --address <Private-IP-Address>  --namespace <namespace> service/vault 8200:8200 &

  2. You can then access the vault UI from your local machine via http://<Public-VMIP>.

  3. Log in using the Vault Token.

  4. After successful login, please click Enable new engine.

  5. Select KV, then click Next.

  6. Change the version to 1. You can keep the path as the default (e.g. /kv).

    Attention:

    If you do not change version value, you will encounter downstream impacts when attempting to deploy.

  7. Click Enable Engine, then click Create Secret.

  8. Provide the path to the secret (e.g. /ot/smb).

  9. Take one of the following actions:

    1. Add the key-value pair required for your Data Source authentication

    2. Toggle on JSON to enter the credential details in JSON format.

    Note:

    Note down the full path of the secret stored in vault, in this case it is /kv/ot/smb.

Vault Helper Script

The Vault Helper Script is an alternative to adding credentials in the OneTrust application in order to add credentials locally.

Note:

./vault_helper.sh is located in .../k8s../shared/scripts/lib/core/vault_helper.sh

Tip:

The following parameter options are also available:

  • ./vault_helper.SH --backup: backup existing secrets onto backup directory

  • ./vault_helper.SH --restore: restore secrets loaded from a directory

  • ./vault_helper.SH --mysql-encrypt-false: append mssql connection URL with ;encrypt=false;trustServerCertificate=false;hostNameInCertificate=*.database.windows.net;

  1. Once installation is complete, navigate to the shared/scripts directory.

  2. Set the namespace by executing the below command:

    CODE 
    kubectl config set-context --current --namespace=<namespacename>
    Note:

    The helper script makes use of kubectl commands to interact with the vault pod and perform operations. On the machine where you are running the script, you should have kubectl and also be able to interact with the kubernetes cluster.

  3. Run the following to get the help command notes:

    CODE 
    ./vault_helper.sh -h

  4. Run the following:

    CODE 
    ./vault_helper.sh -w=<SECRET_NAME>
    Note:

    Example: If you want to set credentials for for Amazon S3, then you could provide the secret name as ./vault_helper.sh -w=AmazonS3

  5. You will then be prompted to input a Secret Value.

  6. Input the proper JSON format for the credentials.

  7. Run the following to read and confirm the entered secret:

    CODE 
    ./vault_helper.sh -r=<SECRET>

  8. Run the following to check the credential configured, then copy the reference key to be configured for the OneTrust application:

    CODE 
    ./vault_helper.sh -l
    Note:

    For Onebox, add -o as extension to create or run any vault command.

    Example: ./vault_helper.sh -l -o

  9. If you want to use this script to automatically set up multiple credentials, you can use a text file:

    JSON 
    ./vault_helper.sh -w=AmazonS3 <<'EOF'
{
    "aws_key_id": "xyz",
    "aws_secret": "pdq",
    ...
}

EOF
# NOTE the "extra" empty blank line (no spaces on it either!)
./vault_helper.sh -w=Redshift <<'EOF'
{
    "aws_key_id": "xyz",
    "aws_secret": "pdq",
    ...
}

EOF
...

HashiCorp Enterprise AppRole Policy

The following policy needs to be configured if using AppRole authentication method for Hashicorp Enterprise.

  1. For example, if the path to the secrets is configured as follows:

    • kv/onetrust/secrets/mysql

    • kv/onetrust/secrets/oracle

  2. Set the policy on the approle to read them:

    CODE 
    path "kv/onetrust/secrets/*" {
  capabilities = ["read"]
}

Enabling Vault Rotation for HashiCorp Enterpise

HashiCorp vault uses many authentication methods. One of which is AppRole. In AppRole authentication, the user provides role-id and secret-id to login into the vault which in return issues a vault access token that is used in the header as X-Vault-Token for all future requests.

For more information, see AppRole-Auth Methods.

Prerequisites:

  • Credential Store: HashiCorp Vault - Enterprise

  • VAULT Authentication Type: AppRole

Guided Installation

  1. Add New Worker Node

  2. Set the credential store to HashiCorp Vault = Enterprise .

  3. Set vault authentication to AppRole.

  4. Enable Rotate Secret. By enabling this flag, the system will rotate the access token and secret Id.

  5. Set the buffer duration.

    Note:

    Enter the buffer duration at which the secret id will be rotated before the expiration date. Note the buffer duration should be less than the secret ID time-to-live (TTL). The default buffer time is 5 days.

  6. Add the following approle policy to allow for vault token rotation:

    JSON 
    If VAULT_ROLE_PATH is set to "approle/onetrust" and VAULT_ROLE is set to "myrole", then:

path "auth/approle/onetrust/role/myrole/*" {
  capabilities = ["update", "read"]
}
    Note:

    OneTrust recommends to set OT_CREDENTIAL_STORE_VAULT_KV_API_VERSION="2" in your .guided-envs if your KV secrets engine is version 2.

    Otherwise, it will be defaulted to 1.

Manual Installation

  1. Download latest installer package.

  2. Update the following variables when configuring your .envs:

    • export VAULT_ROTATION_BUFFER_DURATION="" # default is 5d (5 days), values can be changed to Xd -&gt; X days, Xs -&gt; X seconds, Xm -&gt; X minutes, Xh -&gt; X hours

      • Default is 5d (5 days).

      • Values can be changed to Xd -&gt; X days, Xs-8gt; X seconds, Xm-&gt; X minutes, Xh-&gt; X hours

    • export VAULT_ROTATION_ENABLED="" # default is false, values can be changed to 'true' or 'false'

Vault Troubleshooting FAQ

Why am I receiving a "Vault is sealed" error alert?

The vault was locked requiring manual recovery decoding.

Port forward to it and open the localhost url for vault and enter the key fetched from the below command at vault login page:

CODE 
kubectl -n <namespace> get secret vault-keys --template={{.data.VAULT_KEY}} | base64 -d

Or you can run the following command after you exec into the vault-0 pod

CODE 
vault operator unseal

How do I get the vault token in order to set up data sources in the vault?

Run:

CODE 
kubectl -n <namespace> get secret vault-keys --template={{.data.VAULT_TOKEN}} | base64 -d

Or use the vault_helper.sh in the utils folder:

CODE 
./vault_helper.sh