Automated AI Discovery: Databricks Unity Catalog Integration for AI Governance Models

Overview

The Databricks Unity Catalog Integration allows you to discover and inventory AI models, notebooks, and experiments across your Databricks Unity Catalog environment while maintaining centralized governance controls. This seamless "Wizard" integration enables real-time visibility into your organization's AI landscape, mapping relationships between models, training data, and business processes to strengthen risk management and compliance. By eliminating manual tracking, teams can focus on innovation while maintaining comprehensive oversight of their AI systems through automated discovery and continuous monitoring.

Reduce manual data entry

The integration automates the synchronization of model attributes between Unity Catalog and OneTrust, eliminating the need for manual data entry. This can save you time and effort and help reduce errors.

Maintain up-to-date model information

The integration ensures that your model information is always up-to-date in OneTrust. This is important for compliance and risk management purposes.

Preserve the integrity of data

The integration allows you to configure a non-overwrite policy, which means that data from Unity Catalog will not overwrite data in OneTrust.

Prerequisites

  • Databricks workspace with Unity Catalog enabled

  • Service Principal created in Databricks with a generated OAuth secret and client ID

  • OneTrust OAuth credential created with Databricks workspace details and generated secret ID and client ID from Service Principal.

To create the integration

  1. In the Integrations menu, select Gallery.

    The Integration Gallery screen appears.

  2. Use the search function to locate and select the Databricks tile.

    The Gallery Details screen appears.

    Note:

    The Azure Databricks tile will be consolidated into the Databricks tile in a future release. Please use the Databricks tile to configure Unity Catalog AI Discovery.


    Integrations gallery - Databricks
  3. Click the Add button for the Unity Catalog AI Discovery workflow.

    The Add Workflow Name modal appears.


    Unity Catalog AI Discovery tile
  4. In the Name your new connection section, provide a unique name for the connection and a notification email. The connection name should reflect your Databricks workspace or project for easy identification.

    Unity Catalog AI Discovery Workflow Wizard
  5. In the Provide Login Credentials section, select existing OAuth Client Credentials or create a new one here with your details from the workspace and generated details from the Service Principal.
    Existing

    Provide Login Credentials section - Select existing credentials
    Create New

    Provide Login Credentials section - Create new credentials

    OneTrust Field

    Required from Databricks

    Access Token URL

    https://{hostname}/oidc/v1/token

    Example: https://adb-123456789.9.azuredatabricks.net/oidc/v1/token

    Client Secret

    Secret from Service Principal

    Client ID

    Client ID from Service Principal

    Scope

    Enter the desired scope(s) to control the level of access.
  6. Click the Next button.

    The Configure Trigger screen appears.

  7. Define how often this integration should trigger with the Frequency and Date and Time options.

    • Run Manually

    • Hourly

    • Daily

    • Weekly

    • Monthly

    • Yearly


    Workflow wizard - Configure trigger
  8. Click the Next button.

    The Specify workspace, catalog, and schema screen appears.

  9. Provide your given Workspace URL, Catalog Name, and Schema Name in the corresponding sections. These must be consistent with Databricks Unity Catalog structure.

    Specify workspace, catalog, and schema screen
  10. Click the Next button.

    The Integration Summary screen appears.

  11. Review your Databricks Unity Catalog integration settings. Confirm that these details are correct before activating the workflow.

    Integration Summary screen
  12. Click the Save button. The screen will populate with the last updated update and post an Inactive status.

    Integration Summary review screen
  13. Click the Activate button. A confirmation modal will appear for you click the Ok button.

    Confirmation modal
  14. The integration is now active and ready to trigger and sync your model information.