Skip to main content

Configure a Databricks Delta Share Data Connection (Hybrid Pattern)

If you have data in Databricks and want to be able to use that data in questions in LiveRamp Clean Room, you can create a Databricks data connection.

A Databricks data connection for a Hybrid clean room can be used in the following clean room types:

  • Hybrid

  • Confidential Computing

Note

To configure a Databricks data connection to be used in a Databricks (native-pattern) clean room, see "Configure a Databricks Pattern Data Connection".

After you’ve created the data connection and Clean Room has validated the connection by connecting to the data in your cloud account, you will then need to map the fields before the data connection is ready to use. This is where you specify which fields can be queryable across any clean rooms, which fields contain identifiers to be used in matching, and any columns by which you wish to partition the dataset for questions.

After fields have been mapped, you’re ready to provision the resulting dataset to your desired clean rooms. Within each clean room, you’ll be able to set dataset analysis rules, exclude or include columns, filter for specific values, and set permission levels.

To configure a Databricks data connection to be used in either a Hybrid clean room or a Confidential Computing clean room, see the instructions below.

Overall Steps

Perform the following overall steps to configure a Databricks Delta Share data connection in LiveRamp Clean Room:

  1. Download credentials from Databricks

  2. Add the credentials

  3. Create a data connection

  4. Map the fields

For information on performing these steps, see the sections below.

Download Credentials from Databricks

Configure a Delta Share for use in enabling LiveRamp Clean Room to orchestrate access to your Databricks table in your Hybrid Clean Rooms:

  1. Go to your Catalog Explorer in Databricks, find the table you wish to share, and select "Share via Delta Sharing'.

    Screenshot 2024-06-18 at 13.10.26.png

  2. Select "Create a new share with the Table" and apply a new Share name.

  3. Click Share.

  4. Find the newly created share in your "Shares" screen and select the "Recipients" tab.

  5. Create a new recipient for use with clean rooms.

  6. Go to this recipient in your Catalog and capture the "Activation link" value. Click the link and you will be taken to download a credential file.

  7. Download and copy the JSON credential in the credential file.

Note

You may use these credentials for all Delta Table shares to LiveRamp Clean Room. Each table will have its own data connection, leveraging the same credentials.

Add the Credentials

After you've downloaded the credentials from Databricks, add the credentials to LiveRamp Clean Room:

  1. From the LiveRamp Clean Room navigation pane, select Data Management → Credentials.

  2. Click Add Credential.

    add_credential.png

  3. Enter a descriptive name for the credential and then select "Databricks Delta Credentials" for the Credentials Type.

  4. Enter the JSON credential you downloaded in the previous step to the Sharing ID field.

  5. Click Save Credential.

    Screenshot_2024-06-18_at_14_45_57.png

Create a Data Connection

After you've added the credentials to LiveRamp Clean Room, create the data connection:

Note

if your cloud security limits access to only approved IP addresses, talk to your LiveRamp representative before creating the data connection to coordinate any necessary allowlisting of LiveRamp IP addresses.

  1. From the LiveRamp Clean Room navigation pane, select Data Management → Data Connections.

  2. From the Data Connections page, click New Data Connection.

    data_cxn_new.png

  3. From the New Data Connection screen, select "Databricks Delta Table Connection".

  4. Complete the following fields in the Set up Data Connection section:

    Screenshot 2024-06-18 at 14.52.09.png

    • To use partitioning on the dataset associated with the data connection, slide the Uses Partitioning toggle to the right.

    • Category: Enter a category of your choice.

    • Dataset Type: Select Generic.

    • Delta Share Name: Enter the name of the share you created in the earlier step.

    • Delta Schema Name: Enter the name of the schema for the table you shared.

    • Delta Table Name: Enter the name of the table you shared.

  5. Review the data connection details and click Save Data Connection.

All configured data connections can be seen on the Data Connections page.

When a connection is initially configured, it will show "Verifying Access" as the configuration status. Once the connection is confirmed and the status has changed to "Mapping Required", map the table's fields.

Map the Fields

Once the above steps have been performed in Google Cloud Platform, perform the overall steps in the sections below in LiveRamp Clean Room.

Note

Before mapping the fields, we recommend confirming any expectations your partners might have for field types for any specific fields that will be used in questions.

  1. From the row for the newly created data connection, click the More Options menu (the three dots) and then click Edit Mapping.

    The Map Fields screen opens, and the file column names auto-populate.

    data_cxn_mapping_mapfields.png

  2. For any columns that you do not want to be queryable, slide the Include toggle to the left.

    Note

    Ignore the field delimiter fields because this was defined in a previous step.

  3. Click Next.

    The Add Metadata screen opens.

  4. For any column that contains PII data, slide the PII toggle to the right.

  5. Select the data type for each column.

  6. If a column contains PII, slide the User Identifiers toggle to the right and then select the user identifier that defines the PII data.

  7. Click Save.

Your data connection configuration is now complete and the status changes to "Completed".

You can now provision the resulting dataset to your desired Hybrid or Confidential Computing clean room.

In this section: