Perform Identity Resolution Through ADX

Identity resolution input data files should be formatted as CSV files. When creating input data files, follow these additional guidelines:

The PII resolution process passes the data through a privacy filter which removes the PII and reswizzles the table. Because of this, any attributes you need to keep associated with the identifier need to be included in the input table. For more information, see the "Privacy Filter" section below.

These column names cannot be used in the input file for PII resolution:

See the table below for a list of the suggested input file columns and descriptions for PII resolution.

The email-only resolution process operates similarly to PII resolution. Any attributes you need to keep associated with the identifier need to be included in the input table. For more information, see the "Privacy Filter" section below.

See the table below for a list of the suggested input table columns and descriptions for email-only resolution.

The device ID resolution operation can be used for the following purposes:

See the tables below for a list of the suggested input file columns and descriptions for these device ID resolution options.

See the table below for a list of the suggested input file columns and descriptions for translating device identifiers.

See the table below for a list of the suggested input table columns and descriptions for translating individual RampIDs into their associated household RampIDs.

See the table below for a list of the suggested input file columns and descriptions for CID resolution.

See below for the format of an https curl call used to initiate identity resolution:

curl --location --request POST 
'https://<data-exchange-url>/adx/job/start' \
--header 'Content-Type: application/json' \
--header 'x-amzn-dataexchange-data-set-id: <data-set-id>' \
--header 'x-amzn-dataexchange-revision-id: <revision-id>' \
--header 'x-amzn-dataexchange-asset-id: <asset-id>' \
--header 'x-amzn-dataexchange-http-method: POST' \
--data-raw '{
        "httpMethod": "POST",
        "input_s3": "<Input S3 bucket>",
        "file_format": "csv",
        "file_pattern": "<Regex pattern for input files>",
        "workflow_type": "<Resolution type>",
        "workflow_sub_type": "<Resolution sub type>",
        "target_column": <Identifier column header>,
        "client_id": "<Client ID>",
        "client_secret": "<Client Secret>",
        "cross_region": “true”
        "match_limit": "<# of RampIDs returned>"

}'

curl -X POST "https://<data-exchange-url>/adx/job/start" \
     -H "Content-Type: application/json" \
     -d '{
         "input_s3": "s3://input_bucket",
         "aws_key_id": "",
         "aws_secret_key": "",
         "file_format": "csv",
         "file_pattern": "fake_pii_input.csv",
         "workflow_type": "resolution",
         "workflow_sub_type": "pii",
         "client_id": "<client-id>",
         "client_secret": "<client-secret>",
         "cross_region": "true",
         "input_columns": {
             "name": "text",
             "streetAddress": "text",
             "zipCode": "text",
             "phone": "text",
             "email": "text",
             "likes_dogs": "text"
         },
         "target_columns": {
             "name": ["name"],
             "streetAddress": ["address"],
             "zipCode": "zip",
             "phone": "phone",
             "email": "email"
         }
    }'

curl -X POST "https://<data-exchange-url>/adx/job/start" \
     -H "Content-Type: application/json" \
     -d '{
        "input_s3": "s3://input_bucket",
        "file_format": "csv",
        "file_pattern": "input/fake_email_input.csv",
        "workflow_type": "resolution",
        "workflow_sub_type": "email",
        "target_column": "HASHED_EMAIL",
        "client_id": "<client-id>",
        "client_secret": "<client-secret>",
        "cross_region": "true",
        "input_columns": {
            "hashed_email": "text",
            "likes_dogs": "text" 
        }
    }'

curl --location --request POST 
'https://<data-exchange-url>/adx/job/start' \
--header 'Content-Type: application/json' \
--header 'x-amzn-dataexchange-data-set-id: <data-set-id>' \
--header 'x-amzn-dataexchange-revision-id: <revision-id>' \
--header 'x-amzn-dataexchange-asset-id: <asset-id>' \
--header 'x-amzn-dataexchange-http-method: POST' \
--data-raw '{
        "httpMethod": "POST",
        "input_s3": "s3://my-input-bucket-name<Input S3 bucket>",
        "file_format": "csv",
        "file_pattern": "resolution_input_2.*[.]csv",
        "workflow_type": "resolution",
        "workflow_sub_type": "CTV",
        "target_column": "DEVICE_ID",
        "client_id": "my-client-id",
        "client_secret": "my-client-secret",
        "cross_region": “true”

}'

See below for the format of an AWS CLI call to initiate identity resolution:

aws dataexchange send-api-asset \
  --data-set-id <data-set-id> \
  --revision-id <revision-id> \
  --asset-id <asset-id> \
  --request-headers ‘x-api-key=XXXX-XXXX-XXXX-XXX-<client_id>’ \
  --method POST \
  --path "/adx/job/start" \
  --body "{\"input_s3\": \"<Input S3 bucket>", \"file_format\": \"csv\", \"file_pattern\": \"<Regex pattern for input files>*[.]csv\", \"workflow_type\": \"device_resolution\", \"workflow_sub_type\": \"<Resolution Sub type>\", \"target_column\": \"<Identifier column header>", \"client_id\": \"<Client ID>", \"client_secret\": \"<Client_sectret>", \”input_columns\”: {<Target Column and Passthroughs>} \"cross_region\": \"true\" }"

See below for an example of what a populated AWS CLI call to initiate identity resolution might look like:

aws dataexchange send-api-asset \
  --data-set-id <data-set-id> \
  --revision-id <revision-id> \
  --asset-id <asset-id> \
  --method POST \
 --request-headers ‘x-api-key=XXXX-XXXX-XXXX-XXX-<client_id>’ \  
--path "/adx/job/start" \
  --body "{\"input_s3\": \"s3://my-input-bucket-name\", \"file_format\": \"csv\", \"file_pattern\": \"resolution_input_2.*[.]csv\", \"workflow_type\": \"resolution\", \"workflow_sub_type\": \"email\", \"target_column\": \"hashed_email\", \"client_id\": \"my-client-id\", \"client_secret\": \"my-client-secret\", \”input_columns\”: {\”hashed_email\”: \”text\”, \”gender\”: \”text\”} \"cross_region\": \"true\", \"match_limit\": \"1\" }"

The following is an example of a response for a successful job submission for a call to initiate identity resolution:

{
   "Job ID": "9863C6588358503285051D4F0BC83_AWS_US_EAST_1",
   "Status": "ADX Start job submitted"
}

Use the Job ID in the poll job request to initiate the delivery of the output file (for more information, see the "Initiate Output File Delivery" section below).

See below for the format of an https curl call used to initiate output file delivery:

curl --location --request POST 
'https://<data-exchange-url>/adx/job/poll' \
--header 'Content-Type: application/json' \
--header 'x-amzn-dataexchange-data-set-id: <data-set-id>' \
--header 'x-amzn-dataexchange-revision-id: <revision-id>' \
--header 'x-amzn-dataexchange-asset-id: <asset-id>' \
--header 'x-amzn-dataexchange-http-method: POST' \
--data-raw '{
        "httpMethod": "POST",
        "job_id": "<Job ID>",
        "output_s3": "<Output S3 bucket>",
        "file_format": "csv",
        "client_id": "<Client ID>",
        "client_secret": "<Client Secret>"
}'

See below for an example of what a populated https curl call used to initiate output file delivery might look like:

curl --location --request POST 
'https://<data-exchange-url>/adx/job/poll' \
--header 'Content-Type: application/json' \
--header 'x-amzn-dataexchange-data-set-id: <data-set-id>' \
--header 'x-amzn-dataexchange-revision-id: <revision-id>' \
--header 'x-amzn-dataexchange-asset-id: <asset-id>' \
--header 'x-amzn-dataexchange-http-method: POST' \
--data-raw '{
        "httpMethod": "POST",
        "job_id": "JOB_ID_123",
        "output_s3": "s3://<my-output-bucket-name>\",
        "aws_key_id": "<AWS Key ID>",
        "aws_secret_key": "<AWS Secret Key>",
        "file_format": "csv",
        "client_id": "<my-client-id>",
        "client_secret": "<my-client-secret>"
}'

See below for the format of an AWS CLI call used to initiate output file delivery:

aws dataexchange send-api-asset \             
 --data-set-id <data-set-id> \
 --revision-id <revision-id> \
 --asset-id <asset-id> \
 --method POST \
 --request-headers ‘x-api-key=XXXX-XXXX-XXXX-XXX-<client_id>’ \
 --path "/adx/job/poll" \
 --body "{\"job_id\": \"<Job ID>", \"output_s3\": \"<Output S3 bucket", \"file_format\": \"csv\", \"client_id\": \"<Client ID\>", \"client_secret\": \"<Client Secret>", \"cross_region\": \"true\" }"

See below for an example of what a populated AWS CLI call used to initiate output file delivery might look like:

aws dataexchange send-api-asset \             
 --data-set-id <data-set-id> \
 --revision-id <revision-id> \
 --asset-id <asset-id> \
 --method POST \
--request-headers ‘x-api-key=XXXX-XXXX-XXXX-XXX-<client_id>’ \
 --path "/adx/job/poll" \
 --body "{\"job_id\": \"JOB_ID_123\", \"output_s3\": \"s3://<my-output-bucket-name>\", \"file_format\": \"csv\", \"client_id\": \"<my-client-id>\", \"client_secret\": \"<my-client-secret>\", \"cross_region\": \"true\" }"

The following is an example of a response when processing is complete:

{
   "Job ID": "9863C6588358503285051D4F0BC83_AWS_US_EAST_1",
   "Status": "Output results uploaded to AWS S3 bucket"
}

In addition to the response received when processing is complete, you might get one of the following responses in the status parameter:

For information on finding the AWS-provided parameters, see this AWS article.

The PII resolution process passes the input table through a privacy filter which removes the PII and reswizzles the table (in addition to other operations). Because of this, any attributes you need to keep associated with the identifier need to be included in the input table. For more information, see the "Privacy Filter" section below.

Identity resolution of PII provides supplemental match metadata for additional insight into customer data that can provide powerful signals for making decisions based on RampIDs.

For PII resolution, the output table includes the fields shown in the table below.

The email-only resolution process operates similarly to PII resolution. Any attributes you need to keep associated with the identifier need to be included in the input table. For more information, see the "Privacy Filter" section below.

For email-only resolution, the results end up in the output table in the same database, with the following fields (as shown below):

For email resolution, the output table includes the fields shown in the table below.

The output file for device ID resolution will follow the format shown in the table below.

The output file for CID resolution will follow the format shown in the table below.

To minimize the risk of re-identification (the ability to tie PII directly to a RampID), the service includes the following processes when resolving PII identifiers (PII resolution or email-only resolution):