Rate effect vs mix effect

A rate effect is the part of a change in a ratio metric that comes from segments converting, refunding or delivering on time at a different rate. A mix effect is the part that comes from volume shifting between segments whose rates differ. Together they add up to the total change, and each points to a different fix.

What rate effect and mix effect mean

A ratio metric divides one count by another. Checkout conversion is orders over sessions. Refund rate is refunds over orders. On-time delivery is on-time parcels over all parcels.

The overall rate is a weighted average of each segment’s rate, weighted by that segment’s share of the denominator. So the total can move for two reasons.

• Rate effect. One or more segments now convert at a different rate. The mix holds still and the rates move.
• Mix effect. The rates hold still and each segment’s share of volume moves. If volume shifts toward a segment with a lower rate, the total falls even though no segment got worse.

Mix shift analysis splits a change into these two parts. A standard method, the Bennet (shift-share) decomposition, uses the average of the before and after values for each segment.

rate effect = sum over segments of (average share × change in rate)
mix effect  = sum over segments of (average rate × change in share)

Using averages makes the two parts add up to the total change exactly.

A worked example (illustrative numbers)

Say an online store runs a paid social campaign that sends a wave of mobile traffic. These numbers are made up to show the arithmetic.

Segment	Sessions before	Conversion before	Sessions after	Conversion after
Desktop	40,000 (67%)	5.0%	30,000 (40%)	5.2%
Mobile	20,000 (33%)	2.0%	45,000 (60%)	2.2%
Total	60,000	4.0%	75,000	3.4%

Both segments improved. Desktop went from 5.0% to 5.2%, and mobile went from 2.0% to 2.2%. Total conversion still fell from 4.0% to 3.4%, a drop of 0.6 points. Over the same period, orders rose from 2,400 to 2,550.

The decomposition shows why.

Effect	Contribution to the change
Rate effect	+0.20 points
Mix effect	-0.80 points
Total change	-0.60 points

The rate effect is small and positive, because both segments converted slightly better. The mix effect is large and negative, because the share of sessions moved from desktop, which converts at about 5%, to mobile, which converts at about 2%. All of the drop, and then some, comes from a change in who arrived.

Simpson’s paradox in metrics

That example is a case of Simpson’s paradox. Every segment moves one way while the total moves the other, because the weights changed underneath. It turns up in business metrics more often than people expect, as in these examples.

• Conversion rate falls after a campaign brings in lower-intent traffic, while each channel’s own rate holds steady.
• On-time delivery falls at a hub because more parcels go to its slowest carrier, while no carrier gets slower.
• Refund rate rises because sales shift toward a category that is always returned more often, while each category’s own refund rate stays flat.
• Activation rate falls because a launch brings a burst of signups from a channel that activates less often.

The paradox also runs the other way. A healthy total can hide a real drop in one segment if volume shifts toward a segment with a higher rate. That’s why you check both effects even when the total looks fine.

Why you fix them differently

Each effect points to a different owner, and mixing them up wastes days. Treat a mix effect as a rate effect and engineers hunt for a bug that doesn’t exist. Treat a rate effect as a mix effect and a real break gets written off as “traffic changed.”

	Rate effect	Mix effect
What changed	How a segment behaves	Which segments the volume comes from
Where to look	Releases, bugs, pricing, a funnel step, a supplier or carrier	Campaigns, channel budgets, partners, routing rules, seasonal audiences
Who usually acts	Product, engineering or operations for that segment	Marketing, growth or whoever sets the volume split
Example (illustrative)	iOS checkout conversion falls from 4.8% to 3.9% after an app release	A campaign doubles mobile’s share of sessions

A mix effect can also mean the metric needs a second look. If conversion fell only because you bought more mobile traffic, and orders rose, the business may be fine.

How Metron reports which one happened

When you define a ratio metric in Metron, you give it a numerator and a denominator from the same table, plus the dimensions to slice by. When Metron investigates a change in that metric, it works in three steps.

1. It splits the change into a rate effect and a mix effect, using a decomposition where the two always add up to the total change.
2. It searches the segments of each dimension for each effect separately. One search looks for segments whose rate moved unusually against their own history. The other looks for segments whose share of volume moved unusually.
3. It accounts for sample size in both searches, so a small segment’s ordinary noise isn’t read as a change.

The write-up then says which effect drove the change and where it sits. An example mix finding reads, “71% of the drop is the North-East hub. No carrier got slower there. A larger share of parcels now goes to the slowest one.”

As with every Metron finding, the where (attribution) and the when (timing of nearby events) are scored separately, and cause is never claimed. The four steps of a Metron investigation show where this fits. For the manual routine this replaces, see how to find out why a metric dropped. For a ratio metric in context, see investigating SaaS activation rate.

Metron is in private beta for teams on PostgreSQL or BigQuery. If you have a ratio metric that moves and nobody can say whether the rate or the mix changed, request beta access.