Anomaly detection and root cause analysis for BigQuery metrics

Metron runs anomaly detection and root cause analysis on metrics stored in Google BigQuery. You connect a read-only service account, browse your datasets, and define a metric by pointing at real columns. When you run a check, Metron judges each change against the metric’s own normal, finds the segment that holds most of it, and lists nearby events.

What you need before you connect

• A Google Cloud project with the BigQuery datasets that hold the rows behind your metric.
• A service account with read-only access to those datasets, and its JSON key. Metron never writes to your warehouse.
• A timestamp column on the table, so each row has a date and time.
• A column, or an expression over columns, that gives the value you want to track.
• The columns you want to slice the metric by, such as region, plan or platform.
• A language model endpoint you supply, for the plain-language write-up.

How do you connect BigQuery to Metron?

1. Open Connections and choose BigQuery as the warehouse type.
2. Enter your project ID and paste the service account’s JSON key.
3. Save. Metron tests the connection before it stores anything, and a failed test saves nothing.
4. Metron puts the key in a HashiCorp Vault secrets backend, outside its own database. API responses never include it, and deleting the connection deletes the stored key.

Each saved connection has a Test button, so you can confirm later that the key still works.

Browse your datasets and column types

The schema browser walks from datasets to tables to columns. Each column shows its BigQuery type next to the type Metron detected, such as timestamp, numeric, categorical or boolean. You can see at a glance which column can date a metric, which can carry its value, and which can slice it.

Define a metric by pointing at real columns

The metric builder starts from the same schema. You pick a dataset and a table from dropdowns filled from your warehouse, then:

1. Choose the timestamp column from the table’s real columns.
2. Write the value expression. It can be a single column or a CASE WHEN over columns.
3. For a rate, add a denominator expression. Metron then treats the metric as a ratio.
4. Tick the columns to slice by. Each one shows its detected type.
5. Name the metric and save.

Metron validates the expression when you save it. It checks every column name against the table’s real columns and allows arithmetic, CASE WHEN and a short list of functions. A column that doesn’t exist, a second statement or a subquery is rejected, and the error says what Metron found. The timestamp column and slice columns go through the same check.

What does an investigation return?

An investigation answers three questions in order. It starts when you run a check or when someone asks about a change.

Question	What Metron reports
Is the change real?	Expected value, actual value and the gap, judged after the weekly pattern and the trend are removed
Where does it sit?	The segment that holds most of the change, its share, and whether the rate changed or the mix shifted
What happened near it?	Deploys and logged events close to the start of the change, with the time gap

Because Metron removes the weekly pattern first, a normal Monday dip doesn’t count as a change. Read more about how removing the weekly pattern cuts false alarms. A new metric gets a simpler model until it has enough history.

For a rate such as conversion, the split between a rate effect and a mix effect tells you whether a segment’s own rate fell or its share of the volume changed. Deploys and releases arrive from GitHub by webhook. You log other events by hand.

A language model you supply words the findings in a few plain sentences. It sees the findings only, never your tables. Metron scores the “where” and the “when” claims separately and never states a cause. Follow-up questions are answered from the same evidence, and each write-up can go to a webhook or an email address.

Your data stays in BigQuery

Metron runs in your environment and reads BigQuery in place, with no pipeline copying your tables. The page on self-hosted root cause analysis covers what stays in your environment and what leaves.

Metron is in private beta, and we set up each team by hand. If your metrics live in BigQuery, tell us which metric you want to start with.