Skip to main content

Configure a Google Cloud Storage Data Connection (Customer-Hosted)

LiveRamp Clean Room’s application layer enables companies to securely connect distributed datasets with full control and flexibility while protecting the privacy of consumers and the rights of data owners.

To configure a customer-hosted Google Cloud Storage (GCS) data connection, see the instructions below.

Note

You can connect GCS to LiveRamp Clean Room using a LiveRamp-hosted GCS instance instead of using your own. For more information, see "Configure a Google Cloud Storage Data Connection (LiveRamp-Hosted)".

Overall Steps

Perform the following overall steps in Google Cloud Platform to configure a customer-hosted GCS data connection:

  1. Create a Google service account

  2. Create a Google service account key

  3. Grant permissions to access bucket objects to the service account

  4. Assign the custom role to the cloud storage service account

  5. Capture the data location (the GCS bucket file path)

Once the above steps have been performed in Google Cloud Platform, perform the following overall steps in LiveRamp Clean Room:

  1. Add credentials

  2. Create the data connection

  3. Map the fields

For information on performing these steps, see the sections below.

Perform Steps in Google Cloud Platform

Perform the steps in the sections below in Google Cloud Platform to configure a customer-hosted GCS data connection.

Create a Google Service Account

To create a Google service account in Google Cloud Platform:

  1. From the Google Cloud Platform main menu, select IAM & Admin → Service Accounts.

  2. Click CREATE SERVICE ACCOUNT. Save the service account email because you will need it in later steps.

    image idm3666

  3. Enter a name for the service account.

  4. Click Create & Continue.

  5. Configure roles and additional user access as needed.

    image idm3671

  6. Click DONE.

Create a Google Service Account Key

After you've created a Google service account, create a Google service account key:

  1. From the Google Cloud Platform main menu, select IAM & Admin → Service Accounts.

  2. Select the check box for the service account you created in the previous procedure.

  3. From the More Options menu for the service account's row, select Manage keys.

    image idm3676

  4. Click ADD KEY.

  5. For the key type, select JSON and then click CREATE.

    image idm3686

  6. The private key will be stored in your Download folder (it will look similar to the following example). Save this for use in the “Add the Credentials in LiveRamp Clean Room” section below.

    image idm3691

Grant the Service Account Permissions to Access Bucket Objects

To grant bucket objects permissions to the service account:

Note

If you haven’t already created a bucket, create a bucket by following these Google instructions.

  1. From the Google Cloud Platform main menu, select IAM & Admin → Service Accounts.

    image idm3696

  2. Select CREATE ROLE.

  3. Enter a title, description, and ID for the custom role.

  4. Select Add permissions.

  5. Enter "Storage Admin" in the filter.

  6. Add the following permissions:

    • storage.buckets.get

    • storage.objects.get

    • storage.objects.list

  7. Click CREATE.

Assign the Custom Role to the Cloud Storage Service Account

To assign the custom role to the cloud storage service account:

  1. From the Google Cloud Platform console, search for "Cloud Storage".

  2. From the navigation menu, select Buckets.

  3. Click the bucket name for the bucket you want to configure for access.

  4. Select the Permissions tab.

  5. Click Grant Access.

  6. Select SHOW INFO PANEL in the upper-right corner. The information panel for the bucket displays.

  7. From the information panel, click ADD PRINCIPAL.

    image idm3701

  8. In the Add members field, add the service account.

  9. From the Select a role dropdown, select Custom → role (where "role" is the custom Cloud Storage role you created in the previous procedure).

    assign_custom_role_to_bcket2.png

  10. Click SAVE.

  11. Confirm the custom role and service account are now associated with the bucket.

    saved_role2.png

Capture the Data Location

During the process of creating the data connection, you will need to enter the data location in the form of the GCS bucket file path.

  1. From the Google Cloud Platform console, search for "Cloud Storage".

  2. From the navigation menu, select Buckets.

  3. Select the Objects tab.

  4. From the More Options menu (the three dots) in the row for the bucket, select Copy gsutil URL.

    CR-GCS_Data_Connection-copy_file_path.png

  5. Save the bucket file path for use in the “Create the Data Connection” section below.

Perform Steps in LiveRamp Clean Room

Once the above steps have been performed in Google Cloud Platform, perform the overall steps in the sections below in LiveRamp Clean Room.

Add the Credentials in LiveRamp Clean Room

To add credentials:

  1. From the LiveRamp Clean Room navigation pane, select Data Management → Credentials.

  2. Click Add Credential.

    add_credential.png

  3. Enter a descriptive name for the credential.

  4. For the Credentials Type, select "Google Service Account".

  5. For the Project ID, enter the project ID.

  6. Enter the Credential JSON you stored in the "Create a Google Service Account Key" procedure above.

  7. Click Save Credential.

    mceclip0.png

Create the Data Connection

To create the data connection:

  1. From the LiveRamp Clean Room navigation pane, select Data Management → Data Connections.

  2. From the Data Connections page, click New Data Connection.

    data_cxn_new.png

  3. From the New Data Connection screen, select "Google Cloud Storage (with SA)".

    Screenshot 2024-03-28 at 19.51.21.png

  4. Select the credentials created in the previous procedure from the list.

  5. Configure the data connection:

    Screenshot 2024-06-17 at 11.43.12.png

    • Name: Enter a name of your choice.

    • Category: Enter a category of your choice.

    • Dataset Type: Select Generic.

    • File Format: Select CSV.

      Note

      • All files must have a header in the first row. Headers should not have any spaces or special characters and should not exceed 50 characters. An underscore can be used in place of a space.

      • If you are uploading a CSV file, avoid double quotes in your data (such as "First Name" or "Country").

    • Quote Character: If you are uploading CSV files, enter the quote character you'll be using (if any).

    • Field Delimiter: If you are uploading CSV files, select the delimiter to use (comma, semicolon, pipe, or tab).

    • Data Location: Enter the GCS bucket location captured in the “Capture the Data Location” section above, including the date macro and refresh type. For example, "gs://habu-client-org-123ab456-7d89-10e1-a234-567b891c0123/purchase_events/{yyyy-MM-dd}/incremental" (remove the brackets from the date shown in the example).

    • Sample File Path: Do not enter anything in this field. We will use the data location specified above to locate the file.

  6. Review the data connection details and click Save Data Connection.

    All configured data connections can be seen on the Data Connections page.

  7. Upload your data files to your specified location.

When a connection is initially configured, it will show "Verifying Access" as the configuration status. Once the connection is confirmed and the status has changed to "Mapping Required", map the table's fields.

You will receive file processing notifications via email.

Map the Fields

Once the connection is confirmed and the status has changed to "Mapping Required", map the table's fields and add metadata:

  1. From the row for the newly-created data connection, click the More Options menu (the three dots) and then click Edit Mapping.

    mceclip2.png

    The Map Fields screen opens and the file column names auto-populate.

    data_cxn_mapping_mapfields.png

  2. For any columns that you do not want to be queryable, slide the Include toggle to the left.

  3. If needed, update any column labels.

    Note

    Ignore the field delimiter fields because this was defined in a previous step.

  4. Click Next.

    The Add Metadata screen opens.

    image-20240612-162557.png

  5. For any column that contains PII data, slide the PII toggle to the right.

  6. Select the data type for each column.

  7. If a column contains PII, slide the User Identifiers toggle to the right and then select the user identifier that defines the PII data.

  8. Click Save.

Your data connection configuration is now complete and the status changes to "Completed".

In this section: