Bosca / Experiments

A/B Experiments

Measure the difference

An experiment doesn't invent its own variants — it watches a flag's targeting rule and measures the variations that rule already serves. So the thing you test is exactly the thing you ship, with nothing to reconcile between them.

A lens on a rule

The flag serves, the experiment measures

Attach an experiment to one of a flag's targeting rules and it starts measuring. The flag still decides which variation each person gets; the experiment simply records exposures and outcomes for the variations that rule rolls out.

  • The experiment's variants are the variations its attached rule already serves.
  • Point it at the default path instead to measure the flag's fallback.
  • No parallel variant list to keep in sync — one source of truth.
experiment · checkout-cta
Flag checkout-cta
Rule mobile-users — control / treatment
Experiment measures this rule

Sticky assignment

One person, one variation

Assignment is deterministic: the same person always lands in the same variation for the life of the experiment. It works for signed-in identities and anonymous installs alike, so results aren't muddied by someone flip-flopping between versions.

  • Keyed by security principal when signed in, or by installation id when anonymous.
  • Computed from a stable hash — consistent across sessions and devices, with no lookup required.
  • Stored as an assignment record so exposures line up cleanly with outcomes.
assignment
{
  "experimentId": "exp_9f3c…",
  "principalId": "usr_1a2b…",
  "variationKey": "treatment",
  "assignedAt": "2026-07-18T14:22Z"
}

Exclusion layers

Keep tests from colliding

Run many experiments at once without letting them contaminate each other. Put them in a shared exclusion layer and each person falls into at most one — once they're bucketed into an experiment, they're held out of the rest in that layer.

  • A layer partitions traffic so overlapping experiments never see the same user.
  • Layers are independent — enrollment in one says nothing about the others.
  • Leave an experiment out of any layer when it's safe to overlap.
layer · checkout
exp · button-colorenrolled
exp · cart-copyexcluded
exp · promo-bannerexcluded

one user → one experiment per layer

Lifecycle

From draft to done

An experiment moves through a clear lifecycle — drafted, running while it gathers exposures, paused if you need to hold, and completed or archived when the call is made.

DraftRunningPausedCompletedArchived

Keep exploring

More on Experiments

Experiments overview

Flags, A/B tests, rollouts, and results — one system.

Feature Flags

Typed flags with targeting rules and deterministic bucketing.

Rollout Policies

Ramp the winner automatically, with guardrails that halt on regression.

Results & Analysis

Real statistics from your analytics, with a ship / don't-ship call.