Skip to main content

Configure a Databricks Pattern Data Connection

Abstract

How to configure a Databricks Delta Share data connection using the hybrid pattern within Clean Room, including steps for setting up the connection, configuring permissions, and verifying the integration

LiveRamp Clean Room’s intelligence application on top of Databricks streamlines data collaboration, delivering end-to-end solutions that drive business outcomes for brands. This integration combines LiveRamp Clean Room’s flexible UI with Databricks Data Cloud to orchestrate clean room use cases through low-code data connectors in order to access data at the cloud source with zero data movement.

To configure a Databricks Pattern data connection, see the instructions below.

Overall Steps

Perform the following overall steps to configure a Databricks Pattern data connection in LiveRamp Clean Room:

  1. Create a data connection

  2. Map the fields

For information on performing these steps, see the sections below.

Prerequisites

Before proceeding, confirm that you've installed the Databricks CLI Framework (for instructions, see "Install Databricks CLI Framework").

Create a Data Connection

Procedure. To create a data connection:

  1. From the LiveRamp Clean Room navigation pane, select Data Management → Data Connections.

  2. From the Data Connections page, click New Data Connection.

    data_cxn_new.png

  3. From the New Data Connection screen, select "Databricks Data Connection".

    Screenshot 2024-03-28 at 10.57.38.png

  4. From the New Data Connection screen, select the Databricks credentials previously created or add new credentials as necessary.

    Screenshot 2024-03-28 at 10.59.31.png

  5. Configure the data connection:

    • Name: Enter a name of your choice.

    • Category: Enter a category of your choice.

    • Dataset Type: Select Generic.

    • Database Name: Enter the name of the database from your Databricks account.

    • Schema Name: Enter the name of the schema from your Databricks account.

    • Table/View Name: Enter the name of the table or view from your Databricks account.

  6. Review the data connection details and click Save Data Connection.

All configured data connections can be seen on the Data Connections page.

When a connection is initially configured, it will show "Verifying Access" as the configuration status. Once the connection is confirmed and the status has changed to "Mapping Required", map the table's fields.

Map the Fields

Once the connection is confirmed and the status has changed to "Mapping Required", map the table's fields and add metadata:

Procedure. To map the fields and add metadata:

  1. From the row for the newly-created data connection, click the More Options menu (the three dots) and then click Edit Mapping.

    The Map Fields screen opens and the file column names auto-populate.

    db_map_fields.png

  2. Click Next.

    The Add Metadata screen opens.

    db_add_metadata.png

  3. For any column that contains PII data, slide the PII toggle to the right.

  4. Select the data type for each column.

  5. If a column contains PII, slide the User Identifiers toggle to the right and then select the user identifier that defines the PII data.

  6. Click Save.

Your data connection configuration is now complete and the status changes to "Completed".

In this section: