Why threshold alerts fire every Monday, and how to stop the false positives

Weekly cycles, annual swings and trend make static thresholds fire on normal variance. Seasonal decomposition gives each metric its own normal instead.

Updated · Zello Labs

Threshold alerts throw false positives because most metrics follow a weekly cycle and a trend. A fixed line fires on every quiet day of the week and stays silent through a real drop on a busy one. Remove the seasonal pattern and the trend first, then judge what remains against the metric’s own normal.

The patterns inside a business metric

Most business metrics combine a few named parts.

Component What it is Example (illustrative)
Weekly seasonality Repeats every seven days B2B signups fall at the weekend and recover on Monday
Daily seasonality Repeats every 24 hours, visible in hourly data Checkout volume bottoms out overnight and peaks in the evening
Annual seasonality Repeats once a year Retail revenue climbs through November and dips in January
Trend Slow movement in the level over months Revenue grows as the customer base grows
Residual What’s left after the patterns and trend are removed Day-to-day noise, plus any real change

Real changes show up in the residual.

Why do threshold alerts fire every Monday?

A static threshold compares every day with the same line. When a metric has a weekly cycle, the low days of the cycle cross that line every week.

Say a B2B product’s daily signups run about 400 on weekdays and 150 on weekends (example numbers). A threshold of 250 fires on Saturday and Sunday. If the alert summarizes the weekend, or the data for those days lands on Monday morning, the alert channel fills up every Monday.

Nothing is wrong, and everyone learns that. After a few weeks people stop reading the alerts. After a few months someone mutes them. A real problem that shows up later goes unnoticed.

Percentage-change rules fail the same way. Day-over-day compares Monday with Sunday, two different points in the cycle, and week-over-week fires on every holiday week.

Why static thresholds also miss real changes

Lowering the threshold to stop the weekend alerts makes the line too loose to catch a drop on a busy day.

Take an online store whose revenue runs about $30,000 on Sundays and $50,000 on Thursdays (example numbers). A threshold at $28,000 stays quiet on Sundays. A Thursday that falls to $40,000, a 20% drop from its usual level, never gets near it.

Trend makes it worse. A line set when revenue was half its current size is far too loose today, so someone has to keep moving every line by hand, for every metric.

Situation (example) Static threshold Seasonal baseline
Normal Sunday low Fires Stays quiet, because Sunday lows are expected
Thursday 20% below its usual level Stays quiet Flags it, because the residual is unusual
Metric doubles over a year Line drifts out of date The trend component moves with the metric

How seasonal decomposition sets a normal for each metric

Seasonal decomposition splits a time series into the seasonal pattern, the trend and the residual. STL (seasonal-trend decomposition using Loess) handles one seasonal period, such as the seven-day cycle in daily data. MSTL extends it to several periods at once, which hourly data needs, because it has a daily cycle inside a weekly one. STL can also down-weight outliers, so last month’s outage doesn’t bend the pattern.

Then you only ask whether today’s residual is unusual for this metric, so each metric gets its own expected range and nobody sets a line by hand.

Here is how Metron applies this when it checks a metric.

  1. It reads the metric’s history from your warehouse and decomposes it in a way that resists outliers, using STL with a weekly period for daily metrics and MSTL for hourly metrics.
  2. It measures how large the residual usually is, using past residuals only. The point being judged is left out, so a large anomaly can’t widen the range used to judge it.
  3. It scores the latest residual against that typical size, which gives the expected value and the normal range for that metric.
  4. When it checks many metrics in one run, it applies a false discovery rate correction. A metric checked alongside many others needs stronger evidence, which keeps the alert list readable.

Two more safeguards keep false alarms down. Data quality checks run before detection, so a late or partial load never reaches the detector. Low-volume count metrics, such as a handful of chargebacks a day, use a count-based (Poisson) test, because a normal curve gets the shape of small counts wrong.

Annual patterns are harder. Estimating a yearly cycle takes at least two full years of history, and holidays move around the calendar. A slow swing such as a summer lull tends to show up in the trend. A one-off day such as Black Friday can still stand out as a change, so log days like that as events in Metron. When one falls near the start of a change, the write-up lists it next to the change.

What happens with a brand-new metric?

A decomposition needs several full weeks of history to tell the weekly pattern apart from noise.

A metric that’s new to Metron often isn’t new at all. When Metron checks a metric, it reads the history already in your warehouse, so a table with a year of rows behind it starts with a year of history.

For a metric that really is new, Metron steps up in stages.

History available What Metron does
Very little No detection yet. Metron labels detection as not available.
Some, short of the full model A simpler model using the recent median and spread, with no seasonal component, labeled low confidence because a seasonal change can slip past it.
Enough to separate the weekly pattern and the trend The full seasonal decomposition described above.

If you’re stuck with threshold alerts for now

You can cut most Monday noise without new tooling.

  1. Compare each day with the same weekday over the last several weeks, never with the day before.
  2. Set separate bands for each day of the week, or for each hour on hourly metrics.
  3. Express bands as a percentage of the recent level so they move with the trend.
  4. Retire alerts that nobody has acted on in a quarter.

Once an alert points at a real change, our guide on how to find out why a metric dropped covers what to do next. Our comparison of anomaly detection and root-cause analysis explains the difference between flagging a change and explaining it.

Metron is in private beta for teams on PostgreSQL or BigQuery. If threshold alerts have trained your team to ignore a metric that matters, request beta access.

Common questions

Why do my metric alerts fire every Monday?

Usually because the metric has a weekly cycle and the alert uses a fixed threshold. Low weekend values cross the line every week, and the alert often lands on Monday morning when the weekend data arrives. Nothing is wrong, so people learn to ignore the alert, and eventually someone mutes it.

How do you reduce false positives from seasonality in metric alerts?

Remove the predictable parts first. A seasonal decomposition such as STL or MSTL separates the weekly pattern and the trend from the residual. Then judge only the residual against its own typical size. Each metric gets its own expected range, and normal weekly swings stop triggering alerts.

What is STL decomposition?

STL stands for seasonal-trend decomposition using Loess. It splits a time series into a repeating seasonal pattern, a slowly moving trend and a residual, and it can down-weight outliers. MSTL extends the method to several seasonal periods at once, such as the daily and weekly cycles in hourly data.

How does Metron handle a metric with little history?

It reads the history already in your warehouse, so many metrics start with plenty. For a truly new metric, Metron runs no detection until there is enough data, then uses a simpler model labeled low confidence, then switches to full seasonal decomposition once it can separate the weekly pattern and the trend.