Create Test and Control Groups

Test and control groups (also sometimes known as "A/B testing") provide a way to measure the performance of your marketing efforts. By exposing one set of users (the test group) to a marketing campaign, while protecting a smaller set of users (the control group) from that campaign, you can see how well your campaign worked by comparing behaviors of the two groups. (See below for an example.)

Connect makes it easy to automatically split a segment into test and control groups.

Let's say you're head of marketing for The Costume Emporium. Your biggest sales holiday, Halloween, is fast approaching. You have a new line of hyper-realistic zombie costumes that you'd like to market to zombie horror enthusiasts. So you obtain a data set about fans of the TV show "The Walking Dead", upload it to LiveRamp, and now your Connect dashboard offers a segment called WALKING DEAD FAN.

Halloween's coming up, so you expect to see an increase in zombie costume sales, even without any extra marketing. How then do you measure the extra boost in sales that will come from your zombie ad-campaign?

With test and control segments, of course!

You can use Connect to automatically generate two new segments, WALKING DEAD TEST and WALKING DEAD CONTROL. We'll randomly assign each member of your original Walking Dead fan segment to exactly one of these two new segments.

Now, when you launch your zombie campaign, you can target only your test group customers, keeping the control group unexposed to your marketing push.

At the end of the holiday season, you can compare sales conversion rates between the two groups. Provided your marketing campaign worked, the test group should perform better than the control. Your marketing campaign's lift is the size of this improvement.

Tip

Control groups tend to be much smaller than test groups, so that most customers do get exposed to the marketing campaign under examination. By default, Connect creates test and control segments in a ratio of 90%/10% of users. You can adjust this ratio to whatever makes most sense for you.

Tip

Records are randomly assigned whenever you create test and control groups via Connect. Once a record is assigned to a test or control group, it's in that group for good. If you need to change the test/control group split for whatever reason, you should create new test and control groups.

Two types of segments can be split into test and control groups:

  • Segments in fields with enumerated values: For example, you can split on users with [AGE:25], but not simply on all users with a value in the [AGE] field.

  • Derived segments: Any derived segment created using the Segment Creator.

Fields that are considered non-enumerable, such as raw fields (those containing over 100 distinct values), cannot be split into test and control groups.

To create test and control groups for a field or segment:

  1. In the left navigation bar, hover over MANAGE and then select the audience that contains the segment you want to split into test and control groups.

    Derived Segment Manage menu.jpg
  2. Click + Create Control Group to open the Test/Control Split popup.

    Test and Control Create Control Group button.jpg
  3. . Click the Search by field name dropdown to open the search window.

    Test and Search by field name dropdown.jpg
  4. Begin typing the name of the field or segment that you want to split and then select your desired field from the options that appear.

    Test and Search by field name example.jpg
  5. If the field has segment values, select the desired segment values from the drop-down list.

    Test and Control Segment Choices.jpg
  6. If you want a different split that the default 90%/10% split, enter the desired value in the Test field (the Control field adjusts automatically). 

    Test and Control Test field entry.jpg
  7. Click Split.

    Test and Control Split button.jpg
  8. Click Close to close the popup.

    Test and Control Close button.jpg

The Control and Test segments are added to your segments table. 

Test and Control Results.jpg

The end of each segment name indicates whether the segment is a test or control segment, the percentage of the original segment it contains, and the date the segment was created.

Once you've created the test group and the control group, you can distribute the test group segment to your desired destinations as you would any other segment.

When you add new records to the underlying segment that a test or control group is based off of, the test and control segments will automatically grow in size to reflect the newly added records.

Any new records will be assigned to either the test or control group, based on the probabilities you specified when you originally created the test and control groups.