Amazon DynamoDB Connector

Overview

Used for OneTrust Data Discovery, the Amazon DynamoDB connector can be leveraged to connect with other NoSQL databases that contain data that needs to be scanned and retrieved for OneTrust Data Discovery.

Note:

This connector is currently under Public Preview. Contact OneTrust Support to request early access.

Pod Residency

nosql-scanner

Connection Method

Host and Master Key. For more information, see Amazon's developer documentation on DynamoDBhere.

Supported Scan Actions

Scan Type

Description

Catalog

This type can retrieve metadata from the list of keyspaces, tables, and items within DynamoDB and send back to OneTrust application.

Catalog & Scan

This will fetch the list of buckets and objects present in those buckets. OneTrust doesn't collect metadata in this scan type but will scan for content extraction and classification. It will return the PII-DEs and Terms as per the classification label identified.

Credential and System Permissions

OneTrust recommends to adhere to your organization's security and infrastructure policy and include system engineer resources or other qualified subject matter experts to set up credentials on data sources. For specific instructions for how to enable these credentials, refer to your data source's implementation documentation, knowledge base and/or support resources.

This connector relies on an AWS access key to use AWS SDKs. For more information, visit here to learn about setting up the DynamoDB web service.

IAM Permissions

Select AWS IAM permissions are required to scan DynamoDB. The following is a typical IAM Policy in JSON format with read permissions for a specific DynamoDB table:

Tip:

To avoid authentication errors, you will need to add permissions for the nodeIAM to assume the role on the account where the data is going to be scanned.

If you are using IAM roles versus account credentials to scan, you will need to configure cross-account scanning. For more information, see AWS Scanning with OneTrust Data Discovery.

JSON 
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "dynamodb:ListTables",
                "dynamodb:DescribeReservedCapacity*",
                "dynamodb:DescribeLimits",
                "dynamodb:DescribeTimeToLive"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "dynamodb:BatchGetItem",
                "dynamodb:GetShardIterator",
                "dynamodb:GetItem",
                "dynamodb:Scan",
                "dynamodb:Query",
                "dynamodb:GetRecords"
            ],
            "Resource": "arn:aws:dynamodb:<region>:<account_ID>:table/table_test"
        }
    ]
}

IAM Policy with Wildcards

In AWS IAM policies for DynamoDB, you can use wildcards to match multiple tables based on patterns in their names.

The following actions with wildcard resources must have permissions across all DynamoDB resources.

  • dynamodb:ListTables

  • dynamodb:DescribeReservedCapacity*

  • dynamodb:DescribeLimits

  • dynamodb:DescribeTimeToLive

Table Type

Description

ARN Pattern

All DynamoDB Tables in an AWS Account

This ARN matches all DynamoDB tables (* indicates any table name) in the specified AWS region and account.
CODE 
arn:aws:dynamodb:region:account-id:table/*
JSON 
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:Query",
        "dynamodb:Scan"
      ],
      "Resource": "arn:aws:dynamodb:region:account-id:table/*"
    }
  ]
}

Tables with a Specific Prefix

Replace your-table-prefix-* with the desired prefix.

This ARN matches DynamoDB tables whose names start with the specified prefix followed by any characters.
CODE 
arn:aws:dynamodb:region:account-id:table/your-table-prefix-*
JSON 
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:Query",
        "dynamodb:Scan"
      ],
      "Resource": "arn:aws:dynamodb:region:account-id:table/prefix_*"
    }
  ]
}

Specific DynamoDB Tables

Replace your-table-name with the name of the specific DynamoDB table.
CODE 
arn:aws:dynamodb:region:account-id:table/your-table-name
JSON 
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:Query",
        "dynamodb:Scan"
      ],
      "Resource": "arn:aws:dynamodb:region:account-id:table/table_test"
    }
  ]
}

Sample Connection JSON

JSON 
{
      "aws_access_key_id": "<access_key>",
      "aws_secret_access_key": "<secret_key>",
      "connectionUrl": "https://dynamodb.<region>.amazonaws.com",
      "region": "<region>",    
}

Scan Profile Support

Note:

For additional details on scan profile configuration and management, see Setting Up Scan Profiles.

Limits

  • Tables

  • Number of Items per Table

When applying includes paths in the scan profile, ensure that the tables referenced in the include are allowed by the IAM policies associated with your IAM role. The includes path need to be targeted to the same resource as your granular permissions. An example to help avoid failed tasks:

CODE 
"Resource": "arn:aws:dynamodb:region:account-id:table/prefix_*"


Collected Objects & Fields

Objects

Fields

Tables

Fields (Flattened json key value pairs)

JSON 
{ "root.parent.child.grandchild": "TYPE" }

  • Name

  • Path

  • Content Type (Fields)

Troubleshooting

AccessDeniedException - Resource

Check that your scan profile is targeted to the same resource as your granular actions.

AccessDeniedException - Account

Check that the actions dynamodb:ListTables, dynamodb:DescribeReservedCapacity*, dynamodb:DescribeLimits, dynamodb:DescribeTimeToLive have the resource “*” defined.

JSON 
{
    "Effect": "Allow",
    "Action": [
        "dynamodb:ListTables",
        "dynamodb:DescribeReservedCapacity*",
        "dynamodb:DescribeLimits",
        "dynamodb:DescribeTimeToLive"
    ],
    "Resource": "*"
 }

Status Code 405 - Incorrect URL

Check that the connectionURL used in the credential JSON is correctly formatted: "connectionUrl": "https://dynamodb.<region>.amazonaws.com"

CODE 
Code:[null (Service: AmazonDynamoDBv2; Status Code: 405; Error Code: null; Request ID: null; Proxy: null)] Source:[com.onetrust.discovery.scanner.dynamodb.channel.sources.processors.DynamoScanJobEventProcessor] Message:[null (Service: AmazonDynamoDBv2; Status Code: 405; Error Code: null; Request ID: null; Proxy: null)] Date:[2023-08-04T11:56:24]