Skip to main content
Does the shorter opener book more? Does the warmer voice get fewer hang-ups? Guessing is free and wrong half the time. A split test turns one campaign into a controlled experiment: two to five variants of the agent, each new lead assigned to one, and every call tagged with the variant that spoke.

​
How a lead is assigned

Assignment is by weight, and it sticks.
  • Each live variant has a weight, a whole number from 1 to 100. Weights across the live variants total 100. Two variants at 50 and 50 is an even test.
  • A lead’s first contact picks the variant by hashing the campaign id and the lead’s phone number into a bucket from 1 to 100, and reading that bucket against the weights. Same lead, same bucket, every time.
  • A lead who has already spoken to a variant keeps it, for as long as that variant is taking calls. Follow-up calls, callbacks and cadence steps all go to the variant they already know.
  • If that variant is excluded from the cycle, a live one covers its calls until it is included again. The lead still belongs to the excluded variant throughout.
  • If the split’s configuration is invalid or the voice platform is out of sync, the lead falls back to the primary variant rather than failing.
Changing weights changes who new leads meet. It never permanently reassigns someone mid-conversation.

​
What differs and what is shared

A variant is a real agent. Only the settings that change how it sounds and what it says are its own. In the agent’s config those are the llm block (the prompt and begin message), the voice block, the interaction block, the language block and callScreening, plus the variant’s name. Everything else is shared by the whole split. Update the agent’s skills, its analysis block (outcomes and AI-generated fields) or its campaign settings once, and every variant gets it. Outcomes and fields are shared on purpose: if each variant recorded different fields, you could not compare them.

​
Drafts: variants that are not in the cycle yet

A variant does not have to take traffic to exist. Set a variant’s weight to 0 and it is excluded: the response comes back with splitDraft: true, it holds no share, and no call is sent to it. Send it any weight from 1 and it is back in the cycle. This is how you build the next challenger without paying for it in live calls. Clone a variant, rewrite its opener, dial it yourself until it holds up, then include it when you are ready to compare. An excluded variant is still a full member of the split:
  • Test calls reach it. A test call names the variant directly and is never weighted, so a draft answers exactly as it would in production.
  • Shared settings still reach it. Skills, the analysis block and campaign settings keep syncing, so a draft you include in three weeks is not carrying a stale outcome schema.
  • Names still have to be unique across the split, and a draft still takes its Roman numeral.
  • Its leads are covered, not lost. Excluding a variant stops it taking calls, including the follow-ups owed to leads it was already speaking to. Those calls still get made, by one live variant that stands in for it, the same one every time. Nothing about the lead is rewritten, so the moment you include the variant again it has those leads back. Calls made while covering do not count towards the covering variant’s own numbers.
The primary cannot be excluded. It is the campaign’s own agent and the fallback for every route, so it always takes traffic. At least one variant is always live: exclude everything else and the primary sits on 100 with the drafts beside it, which is a split you are still building rather than one that ended.

​
Names

Variants are named after the primary with a Roman numeral: Roofing Follow Up, Roofing Follow Up II, Roofing Follow Up III. The names show up as agent on every call record, so keep them readable. Rename a variant to give it a name of your own: the response then carries customName: true, a read-only flag, and the convention leaves that name alone. Send name: null to put it back on the pattern. The primary never carries the flag, because its name is the agent’s own. Normalise names re-derives only the names still on the pattern.

​
Reading the results

There is no separate results endpoint, and there does not need to be. Every variant has an agentVariantId, and the analytics, calls and contacts endpoints accept it as a filter, together with the campaignId. Call the analytics summary once per variant over the same window and compare bookingRate or conversationRate side by side. The walk-through is in Run a split test.

​
Changing the split

  • Add a variant. Weights are re-levelled to equal shares. The request accepts an Idempotency-Key header, and a replay returns the same result with idempotentReplay: true instead of a sixth variant.
  • Remove a variant. Its weight is redistributed across the live ones. The primary cannot be removed. Removing down to one member ends the split: status becomes ended and the campaign is an ordinary campaign again. Drafts count as members, so a split with one live variant and two drafts is still running. To pause a variant instead of ending it, exclude it.
  • Update weights, and decide who is in the cycle. Send the full set: every variant exactly once, 0 for the ones you are excluding, and the rest totalling 100. Leaving a variant out is an error rather than a shorthand for excluding it, so a dropped id never takes an agent off the air by accident. Every write carries revision, the optimistic concurrency token: send back the one you read, and a stale one gets 409.
  • Rename also renames the agent behind the variant, before the split is saved. If that fails you get 502 with SYNC_FAILED: nothing is committed, sync is untouched, and retrying is safe.

​
Boundaries

  • Two to five variants, drafts included. One variant is not a test.
  • At least one variant is live at all times, and it can never be the only thing you have: excluding is for the challengers, not for the primary.
  • A master agent can be split tested. Run the split from the parent campaign; every linked project applies the same weights to its own leads. Converting or adding a variant through a child campaign returns 409 (“Manage this Agent Split Test from its Master Agent.”), and so does a campaign whose master agent setup is incomplete.
  • The split lives on the campaign. Converting again after a split ended starts a new experiment, and leads are assigned afresh.
  • Weights are for new leads. To move an existing lead you would have to remove the variant they are on.

​
In the API