A B testing

Running an A/B test in manage

test creation

To create a new test;

• Create a new file, and the test definition using the information/template from the library, or check out the example test example-test.ts.

• In the abTests.ts file, import and add the new test to the tests array. This will make it available to the ab testing library and api.

• Add a switch for the test in the abSwitches.ts file, in the abSwitches object. The key should be ab + the id from the test definition. For example, if the id in the test definition is ExampleTest, then the switch key should be abExampleTest. The value should be a boolean, with true if the test is enabled, and false if the test is disabled.

The AB Testing Library has more information available to setup tests with.

Component integration

example component:

import { useAB } from '@guardian/ab-react';
import { tests } from '/experiments/abTests';
import { exampleTest } from '/experiments/tests/example-test';

export const ABTestDemo = () => {

  const ABTestAPI = useAB();
  // Using the API (https://github.com/guardian/ab-testing#the-api)
  
  // A) Example for getting and running the first possible runnable test:
  // Get the first possible runnable test
  const firstRunnableTest = ABTestAPI.firstRunnableTest(tests);
  // Get the variant information to run
  const variantFromRunnable = firstRunnableTest?.variantToRun;
  // Get the test method which should run
  const testToRun = variantFromRunnable?.test;
  // Run the test method
  console.log(
    'A) API - First Runnable Test - Outcome:',
    testToRun && testToRun({}),
  );

  // B) Example for checking if a user is in a particular test
  // then get the variant, and run the test method
  const runnableTest = ABTestAPI.runnableTest(exampleTest);
  console.log(
    'B) API - Check for Test - Outcome:',
    runnableTest?.variantToRun.test({}),
  );

  // C) Example for checking if a user is in a specific test and variant
  const isUserInVariant = ABTestAPI.isUserInVariant(
    exampleTest.id,
    exampleTest.variants[0].id,
  );
  console.log(
    'C) API - Check for test and variant boolean - Outcome:',
    isUserInVariant,
  );

  // you can use any of the above for conditional logic too
  // example using C)
  if (isUserInVariant) {
    console.log('C) User in a variant');
  } else {
    console.log('C) User not in a variant');
  }

  if (isUserInVariant) {
    return (
      <h1>
        AB TEST DEMO - In the {exampleTest.id} test and the 
        {exampleTest.variants[0].id} variant.
      </h1>
    );
  } else {
    return (
      <h1>
        AB TEST DEMO - NOT in the {exampleTest.id} test and the
        {exampleTest.variants[0].id} variant.
      </h1>
    );
  }

};

Forcing/Viewing a test

There are 2 ways of forcing yourself into a test:

1. Recommended: Use GU_mvt_id (or GU_mvt_id_local) cookie
2. Use URL parameters

Cookie (Recommended)

This method requires manually setting the GU_mvt_id cookie, or the GU_mvt_id_local cookie (if you're dev'ing locally).

Use this simple calculator to work out what value you need for a particular test. Simply add the AB Test Config information (audience + offset), and the variants in that test. Then modify the MVT ID value until Is user in test? is Yes and the variant you want is highlighted. Then copy this MVT ID value as the cookie value for the GU_mvt_id or GU_mvt_id_local cookie.

The library has useful documentation about the calculator.

At first glance this may seem more difficult that using the URL parameters, but the main advantages to this method are:

• Less chance of overlapping tests
  • URL params set the forcedTestVariants parameter in the framework, which may mean that it may overlap with any other tests in that bucket.
• You'll always be in that AB Test throughout the lifetime of the cookie
  • URL parameters are only valid for that single request, which means that tests that rely on multiple requests/routes would be required to add the parameters on every request manually
• More granular control for audience/offset testing
  • As the values are individual buckets, you can fine tune the audience and the offset as required.

URL Params

You can also force yourself into a test and variant using URL parameters, either in the query parameters (after the ?) or as a search parameter (after the #). This requires knowing the ab test id and the variant name. Also to note, you have to prefix the test id with ab-. For example, to force yourself into the ExampleTest and the variant variant. You could add the parameter onto the URL like this https://manage.the guardian.com#ab-ExampleTest=variant.

The advantages to this are that it's simple to do and test, however the parameters may not persist between requests, so might not be able to test a full flow relying on the AB test.