Running LiveRamp’s Local Encoder In a Docker-Run Environment

From the command line, run the command “aws configure” with the following values:

AWS Access Key Id: [LR_VAULT_LR_AWS_ACCESS_KEY_ID]
AWS Secret Access Key: [LR_VAULT_LR_AWS_SECRET_ACCESS_KEY]
Default region name: eu-central-1
Default output format: json

To set your keys by editing the files manually (MAC OS example):

The Local Encoder application offers multiple output formats:

After you are approved for RampID or identity envelope output, update your configuration to add an additional variable using one of the methods listed below:

Optional encryption is available with Local Encoder (version 1.5 and greater). This functionality encrypts each row of data before it is sent to LiveRamp for processing.

To utilize encryption, use one of the following methods, depending on your implementation:

You can configure Local Encoder to return an error log to your desired location that provides details on any errors that were encountered during processing, including the row number for the row where the error occurred. You can then correct the errors and re-process the corrected data.

The error log includes the following information (see the example below):

To enable the error log file and set the file delivery location, see the sections below, depending on the method used.

docker run --rm --cap-add IPC_LOCK \
-v ~/configurationFileFolder:/config \
-v <localFolder>:/var/logs/vault-app \
vault-app \
-config-file test-configuration.yml

docker run --rm --cap-add IPC_LOCK \
-v ~/Documents/localencoder/input:/tmp/input \
-v ~/Documents/localencoder/output:/tmp/output \
-v ~/Documents/localencoder/log:/var/logs/vault-app \
--env LR_VAULT_ACCOUNT_ID=[LR_VAULT_ACCOUNT_ID] \
--env LR_VAULT_ACCOUNT_TYPE=[LR_VAULT_ACCOUNT_TYPE] \
--env LR_VAULT_LR_AWS_ACCESS_KEY_ID=[AWS_ACCESS_KEY_ID] \
--env LR_VAULT_LR_AWS_SECRET_ACCESS_KEY=[AWS_SECRET_ACCESS_KEY] \
--env LR_VAULT_INPUT=/tmp/input \
--env LR_VAULT_OUTPUT=/tmp/output \
--env LR_VAULT_LOCALE=eu-central-1 \
--env LR_VAULT_CUSTOMER_LOGGING_ENABLED=true
--name vault-app [Image Name]

To save processed files names permanently (after application shutdown), there are two options:

- LR_VAULT_METASTORE_DB_URL - datasource url, local file example: *h2:~/db/LOCAL_FILE_NAME*
- LR_VAULT_METASTORE_DB_DRIVER - driver class name, for H2: *org.h2.Driver*
- LR_VAULT_METASTORE_DB_USERNAME - username for H2 database
- LR_VAULT_METASTORE_DB_PASSWORD - password for H2 database
- LR_VAULT_METASTORE_DB_PLATFORM - SQL dialect of the underlying database, for H2: *org.hibernate.dialect.H2Dialect*

docker run --rm --cap-add IPC_LOCK \
-v ~/configurationFileFolder:/config \
-v ~/localencoderdb:/PATH_TO_DB vault-app \
-config-file [test-configuration].yml

- LR_VAULT_METASTORE_DB_URL - database url, local MySQL database example: *mysql://host.docker.internal:3306/DATABASE_NAME*; local PostgreSQL database example *postgresql://host.docker.internal:5432/DATABASE_NAME*
- LR_VAULT_METASTORE_DB_DRIVER - driver class name, for MySQL: *com.mysql.cj.jdbc.Driver*; for PostgreSQL: *org.postgresql.Driver*
- LR_VAULT_METASTORE_DB_USERNAME - username for database
- LR_VAULT_METASTORE_DB_PASSWORD - password for database
- LR_VAULT_METASTORE_DB_PALTFORM - SQL dialect of the underlying database, for MySQL: *org.hibernate.dialect.MySQL5InnoDBDialect*; for PostgreSQL: *org.hibernate.dialect.PostgreSQLDialect*

When using local folders for input and output sources, you have to mount the folder from our host machine:

docker run --rm --cap-add IPC_LOCK \
-v ~/configurationFileFolder:/config \
-v ~/inputFolderOnHostMachine:/LR_VAULT_INPUT \
-v ~/outputFolderOnHostMachine:/LR_VAULT_OUTPUT
vault-app \
-config-file test-configuration.yml

In this example, we are mounting input and output folders from the host machine root directory to LR_VAULT_INPUT and LR_VAULT_OUTPUT directories inside LocalEncoder docker containers. LR_VAULT_INPUT and LR_VAULT_OUTPUT are local input and output values that we provided either in a configuration file or by passing env var LR_VAULT_INPUT and LR_VAULT_OUTPUT. Folder names on the host machine are optional, they always have to be mounted to LR_VAULT_INPUT and LR_VAULT_OUTPUT.

By default, the application will output to the LiveRamp S3. If you would like to point output to a different location, set the LR_Local Encoder_OUTPUT variable:

The application outputs the data to a destination of your choice. Use the LR_VAULT_OUTPUT variable to set your destination, as shown below.  To onboard data to LiveRamp, set the output to the LiveRamp S3 bucket.

--env LR_VAULT_OUTPUT=[Your local directory, or S3 bucket]

A configuration file contains information that is used by the run command and allows for the use of run commands that are more streamlined.

When using a configuration file, you need to mount the folder of the configuration file on the host machine to the "/config" folder inside the docker container.

docker run --rm --cap-add IPC_LOCK \
-v ~/configurationFileFolder:/config \
vault-app \
-config-file test-configuration.yml

In this example, the "test-configuration.yml" file is located in "~/configurationFileFolder" on the host machine, and we are mounting it to the "/config" folder inside the docker container. Then passing "-config-file test-configuration.yml" as a command line argument to LocalEncoder app. Folder name on the host machine is optional, it always has to be mounted to "/config".

In order to use the configuration file, you need to mount a directory where the file is located to /config directory. Example: -v ~/Documents/localencoder:/config - the YAML config file should be located in the Local encoder folder. Then pass the file name as a command line argument. Docker Run example:

docker run --rm --cap-add IPC_LOCK \
-v ~/configurationFileFolder:/config \
[docker_image] \
-config-file [your_filename].yml

Create a yaml configuration file that follows this example:

liveramp:
  local_encoder:
    account_type: awsiam
    account_id: [LR_VAULT_ACCOUNT_ID]
    profile: PROD
    locale: [COUNTRY_CODE]
    output: [your_directory]
    input: [your_directory]
    filename_pattern: pattern
    header_mapping: custom_header=email1,custom_header2=email2
    country_code_column: [COUNTRY_CODE]
    public_key_encryption: true
    mode: default
    dry_run: false
    lr_aws:
      lr_id: 461694764112
      lr_access_key_id: [AWS_ACCESS_KEY_ID]
      lr_secret_access_key: [AWS_SECRET_ACCESS_KEY]
      lr_region: eu-central-1
    metastore:
      db:
        url:
        driver:
        username: 
        password: 
        platform: 

Include the following information in the file parameters:

You can include any of the following allowed identifier touchpoints for translation to RampIDs in both Onboarding and Addressability use cases:

See the header shown below for an example of what the header might look like when sending data in a pipe-separated file (psv) for an Onboarding use case, where segment data fields are included:

RID|EMAIL1|EMAIL2|EMAIL3|SHA256EMAIL1|SHA256EMAIL2|SHA256EMAIL3|MOBILE1|MOBILE2|SHA256MOBILE1|SHA256MOBILE2|LANDLINE1|SHA256LANDLINE1|FIRSTNAME|LASTNAME|POSTCODE|AAID|IDFA|IMEI|ATTRIBUTE_1|...|ATTRIBUTE_N

Replace ATTRIBUTE_1 … N in the example header with the name of your CRM attributes.

For more information on the format of the output file, see "Output Examples".