Perform Identity Resolution Through ADX

Identity resolution input data files should be formatted as CSV files. When creating input data files, follow these additional guidelines:

The standard PII resolution process passes the data through a privacy filter which removes the PII and reswizzles the table. Because of this, any attributes you need to keep associated with the identifier need to be included in the input table. For more information, see the "Privacy Filter" section below.

These column names cannot be used in the input file for PII resolution:

See the table below for a list of the suggested input file columns and descriptions for PII resolution.

The standard email-only resolution process operates similarly to PII resolution. Any attributes you need to keep associated with the identifier need to be included in the input table. For more information, see the "Privacy Filter" section below.

See the table below for a list of the suggested input table columns and descriptions for email-only resolution.

The device ID resolution operation can be used for the following purposes:

See the tables below for a list of the suggested input file columns and descriptions for these device ID resolution options.

See the table below for a list of the suggested input file columns and descriptions for translating device identifiers.

See the table below for a list of the suggested input table columns and descriptions for translating individual RampIDs into their associated household RampIDs.

This process enables the retrieval of an existing CID to RampID mapping, hosted by LiveRamp.

See the table below for a list of the suggested input file columns and descriptions for CID matching.

This process allows for the resolution of a universe dataset with a deconfliction configuration, minimizing conflicts across the entire dataset. The output from this process is a deconflicted hashed CID to RampID mapping.

When resolving a universe dataset, you can use one input data file or multiple input data files. Each input data file can include one identifier type or multiple identifier types (including PII, hashed email, or MAIDs). The examples below can be used for the specific situations listed but you can create an input data file that uses any combination of these identifiers.

The requirements for each identifier type (listed in the relevant sections above) apply here. For example, any PII-based records must include zip in order to match on address and hashed emails must be lowercased and UTF-8 encoded prior to hashing.

See the table below for a list of the suggested input data file columns for a job that will contain both hashed emails and MAIDs with deconfliction.

See the table below for a list of the suggested input data file columns for a job that will contain plaintext PII:

See below for the format of an AWS CLI call to initiate identity resolution (other than when calling to initiate resolution of a universe dataset with multiple files where each file contains different identifiers):

aws dataexchange send-api-asset \
  --data-set-id <data-set-id> \
  --revision-id <revision-id> \
  --asset-id <asset-id> \
  --method POST \
  --path "/adx/job/start" \
    --body '{
                "input_s3": "<Input S3 bucket>",
                "file_format": "csv",
                "file_pattern": "<Regex pattern for input files>[.]csv",
                "workflow_type": "resolution",
                "workflow_sub_type": "<Resolution sub type>",
                "target_column": "<Identifier column header>",
                "client_id": "<Client ID>",
                "client_secret": "<Client secret>",
                "input_columns": {<"Column name": "Column type">},
                "cross_region": "true"
            }'

See below for the format of an AWS CLI call to initiate the resolution of a universe dataset with multiple input files where each file contains different identifiers:

aws dataexchange send-api-asset \
  --data-set-id <data-set-id> \
  --revision-id <revision-id> \
  --asset-id <asset-id> \
  --method POST \
  --path "/adx/job/start" \
    --body '{

                "workflow_type": "resolution",
                "workflow_sub_type": "cid_mapping",
                "client_id": "<Client ID>",
                "client_secret": "<Client secret>",  
                "cross_region": "true",
		"config": {"deconflictionConfig": "<standard or maximized_1P_fidelity>"}
		"inputs": [{
              	    "input_s3": "<Input S3 bucket>",
       	            "file_format": "csv",
                    "file_pattern": "<Regex pattern for input files>[.]csv",
                    "input_columns": {<"Column name": "Column type">},
                    "target_columns": {<"key": "Column name">}
                  },
                  {
              	    "input_s3": "<Input S3 bucket>",
       	            "file_format": "csv",
             	    "file_pattern": "<Regex pattern for input files>[.]csv",
                    "input_columns": {<"Column name": "Column type">},
                    "target_columns": {<"key": "Column name">}
                  }
                ]
            }'

See below for examples of what a populated AWS CLI call to initiate translation might look like.

aws dataexchange send-api-asset \
    --data-set-id <data-set-id> \
    --revision-id <revision-id> \
    --asset-id <asset-id> \
    --method POST \
    --path "/adx/job/start" \
    --body '{
                "input_s3": "s3://my-input-bucket-name",
                "file_format": "csv",
                "file_pattern": "pii_input[.]csv",
                "workflow_type": "resolution",
                "workflow_sub_type" : "PII",
                "target_columns": {
                                    "name": ["FIRSTNAME", "LASTNAME"],
                                    "address": ["ADDRESSLINE", "ADDRESSLINE2"],
                                    "city": "CITY",
                                    "state": "STATE",
                                    "zip": "ZIPCODE",
                                    "email": "EMAIL",
                                    "phone": "PHONE",
                                    "hashedAttributes": ["CID"]
                                  },
                "client_id": "my-client-id",
                "client_secret": "my-client-secret",
                "input_columns": {
                                    "FIRSTNAME": "text",
                                    "LASTNAME": "text",
                                    "ADDRESSLINE": "text",
                                    "ADDRESSLINE2": "text",
                                    "CITY": "text",
                                    "STATE": "text",
                                    "ZIPCODE": "text",
                                    "EMAIL": "text",
                                    "PHONE": "text",
                                    "CID": "text",
                                    "LIKES_DOGS": "text"
                                 },
                "cross_region": "true"
            }'

aws dataexchange send-api-asset \
    --data-set-id <data-set-id> \
    --revision-id <revision-id> \
    --asset-id <asset-id> \
    --method POST \
    --path "/adx/job/start" \
    --body '{
                "input_s3": "s3://my-input-bucket-name",
                "file_format": "csv",
                "file_pattern": "resolution_input_2.*[.]csv",
                "workflow_type": "resolution",
                "workflow_sub_type": "EMAIL",
                "target_column": "hashed_email",
                "client_id": "my-client-id",
                "client_secret": "my-client-secret",
                "input_columns": {
                                    "hashed_email": "text",
                                    "gender": "text"
                                 },
                "cross_region": "true"
            }'

aws dataexchange send-api-asset \
    --data-set-id <data-set-id> \
    --revision-id <revision-id> \
    --asset-id <asset-id> \
    --method POST \
    --path "/adx/job/start" \
    --body '{
                "input_s3": "s3://my-input-bucket-name",
                "file_format": "csv",
                "file_pattern": "ctv_input[.]csv",
                "workflow_type": "resolution",
                "workflow_sub_type": "CTV",
                "target_column": "my_ctv_column",
                "client_id": "my-client-id",
                "client_secret": "my-client-secret",
                "cross_region": "true"
            }'

aws dataexchange send-api-asset \
--data-set-id <data-set-id> \
--revision-id <revision-id> \
--asset-id <asset-id> \
--method POST \
--path "/adx/job/start" \
--body '{
"input_s3": "s3://my-input-bucket-name",
"file_format": "csv",
"file_pattern": "cid_mapping_input[.]csv",
"workflow_type": "resolution",
"workflow_sub_type" : "CID_MAPPING",
"inputs": [{
"target_columns": {
"name": ["FIRSTNAME", "LASTNAME"],
"address": ["ADDRESSLINE", "ADDRESSLINE2"],
"city": "CITY",
"state": "STATE",
"zip": "ZIPCODE",
"email": "EMAIL",
"phone": "PHONE",
"cid": "CID"
},
"input_columns": {
"FIRSTNAME": "text",
"LASTNAME": "text",
"ADDRESSLINE": "text",
"ADDRESSLINE2": "text",
"CITY": "text",
"STATE": "text",
"ZIPCODE": "text",
"EMAIL": "text",
"PHONE": "text",
"LIKES_DOGS": "text",
"CID": "text"
},
}]
"client_id": "my-client-id",
"client_secret": "my-client-secret",
"cross_region": "true",
"config": {"deconflictionConfig": "STANDARD"}
}'

The following is an example of a response for a successful job submission for a call to initiate identity resolution:

{
    "ResponseHeaders": {
        "Content-Type": "application/json",
        "Content-Length": "97",
        ...
    },
    "Body": "{\"Job ID\": \"E660EC80F3BF4473A120D3CAC890CADC_AWS_US_EAST_1\", \"Status\": \"ADX Start job submitted\"}"
}

Use the Job ID in the poll job request to initiate the delivery of the output file (for more information, see the "Initiate Output File Delivery" section below).

See below for the format of an AWS CLI call used to initiate output file delivery:

aws dataexchange send-api-asset \
    --data-set-id <data-set-id> \
    --revision-id <revision-id> \
    --asset-id <asset-id> \
    --method POST \
    --path "/adx/job/poll" \
    --body '{
            "job_id": "<Job ID>",
            "output_s3": "<Output S3 bucket>",
            "file_format": "csv",
            "client_id": "<Client ID>",
            "client_secret": "<Client secret>",
            "cross_region": "true"
            }'

See below for an example of what a populated AWS CLI call used to initiate output file delivery might look like:

aws dataexchange send-api-asset \
    --data-set-id <data-set-id> \
    --revision-id <revision-id> \
    --asset-id <asset-id> \
    --method POST \
    --path "/adx/job/poll" \
    --body '{
            "job_id": "JOB_ID_123",
            "output_s3": "s3://my-output-bucket",
            "file_format": "csv",
            "client_id": "my-client-id",
            "client_secret": "my-client-secret",
            "cross_region": "true"
            }'

The following is an example of a response when processing is complete:

{
    "ResponseHeaders": {
        "Content-Type": "application/json",
        "Content-Length": "158",
        ...
    },
    "Body": "{\"Job ID\": \"E660EC80F3BF4473A120D3CAC890CADC_AWS_US_EAST_1\", \"Status\": \"ADX Poll job started for delivering output results. Re-poll later for updated status\"}"
}

In addition to the response received when processing is complete, you might get one of the following responses in the status parameter:

For information on finding the AWS-provided parameters, see this AWS article.

The standard PII resolution process passes the input table through a privacy filter which removes the PII and reswizzles the table (in addition to other operations). Because of this, any attributes you need to keep associated with the identifier need to be included in the input table. For more information, see the "Privacy Filter" section below.

Identity resolution of PII provides supplemental match metadata for additional insight into customer data that can provide powerful signals for making decisions based on RampIDs.

For PII resolution, the output table includes the fields shown in the table below.

The standard email-only resolution process operates similarly to PII resolution. Any attributes you need to keep associated with the identifier need to be included in the input table. For more information, see the "Privacy Filter" section below.

For email resolution, the output table includes the fields shown in the table below.

The output file for device ID resolution will follow the format shown in the table below.

The output file for CID matching will follow the format shown in the table below.

The output file for universe dataset resolution will follow the format shown in the table below.

To minimize the risk of re-identification (the ability to tie PII directly to a RampID), the service includes the following processes when resolving PII identifiers (PII resolution or email-only resolution):