CitadexDocs

Incrementality Experiments: Measure the AI-Attributed Revenue Your Page Optimizations Add

Revenue Attribution → Experiments compares optimized pages with pages that were not optimized and shows the incremental AI-attributed revenue, leads or AI referral visits with a 95% range. When to use it, the recommended fix holdout, manual experiments and how to read the result.

Revenue Attribution4 min read

When to use it

After optimizing a set of pages (rewriting the pricing page, adding comparison pages), you want to know how much AI-attributed revenue the optimizations added. Comparing those pages before and after is not enough: AI traffic to the whole site may have risen or fallen in the same period. An experiment compares the optimized pages (test group) with pages that were not optimized (control group), removes the change both share, and what remains is the increment.

Where: Revenue Attribution → Experiments. Revenue attribution needs data first (at least the site script installed).

When you are about to optimize a batch of pages (for example the pages in the AI Traffic optimization plan), the most reliable measurement is a Fix holdout:

  1. 1

    Click Fix holdout

    It is the main button on the Experiments tab.

  2. 2

    Select the pages to optimize

    At least 4.

  3. 3

    Choose the metric and hold period

    AI-attributed revenue, AI leads or AI referral visits; the hold period is 28 days by default.

  4. 4

    Click Randomize and start

The pages are split at random into two groups: Test group (optimize now) is optimized as usual; Control group (optimize after …) stays unchanged until the hold period ends and is optimized afterwards. Because the split is random, the difference between the groups measures what the optimizations added, more reliably than a control group picked by hand. During the hold period, the AI Assistant and this week's tasks do not suggest changing the control pages; when it ends, the AI Assistant reports the result in the chat and reminds you to optimize the remaining pages. You can also ask the AI Assistant: "Set up a holdout for these pages."

Create an experiment manually

When the pages are already changed and can no longer be split at random, set the groups by hand: on the Experiments tab, click ⋯ → Create an experiment manually.

  1. 1

    Enter Name and Start date

    Use the day the change went live. It can be in the past.

  2. 2

    Fill in Test pages (optimized)

    Page paths separated by commas, for example /pricing, /compare/. matches anything, so /compare/* means every page under /compare/.

  3. 3

    Fill in Control pages

    Pages that were not optimized. Leave empty to use every landing page outside the test group.

  4. 4

    Choose a Metric

    AI-attributed revenue, AI leads or AI referral visits.

  5. 5

    Set Days to compare

    The same number of days before and after the start date: 30 by default, 7 to 120 allowed. Click Save.

Revenue Attribution → Experiments

Revenue Attribution → Experiments

Read the result

Each experiment is a card headed "Started {date} · N days before vs after".

  • Incremental: the additional AI-attributed revenue (or leads, visits) the optimizations brought, followed by Range a – b, the 95% range. The control group's before-to-after change gives what the test pages would have done without the optimizations; the actual value minus that is the increment.
  • Test group / Control group change: each group's percentage change after vs before.
  • Reliable / Not enough data: when both groups have at least 10 records before and after (won orders for revenue, leads for leads, visits for visits) the card shows Reliable and the incremental number. If any group has fewer than 10 in either period it shows Not enough data and no number.
  • Table: Pages, N days before, N days after and Change for each group. With an empty control group the pages column reads All other landing pages.

An experiment in progress shows Running · day N: before and after both use only the N days so far, and the result is fixed once the full period has passed. Before the start date it shows Not started yet.

FAQ

It says Not enough data. Wait for more days, create a new experiment with more days to compare, or use AI referral visits, which has larger counts, as the metric.

What does a range that crosses 0 mean? Within the 95% range the increment could be positive or negative, so the effect of this change cannot yet be distinguished from zero.

Which control pages should I use? Usually leave it empty. If you changed other pages in the same period, list only pages that remained unchanged.

How do I delete an experiment? Click ✕ in the card's top-right corner. Viewers can see experiments but cannot create or delete them.

Was this article helpful?