Metrics

Quoin needs two things from your app to compute anything. Quoin must know who a unit is. Quoin must also know what happened. Both come from you. Neither requires a warehouse first.

Source 1 — events

Send events from your app or backend with a small client:

quoin.identify('user_412', { tier: 'starter', signedUpAt: Date.now() });

quoin.track('upgraded', { unit: 'user_412' });

identify(unit, traits) attaches traits to a unit. It carries only these traits: tier, signedUpAt, client and a pre-hashed emailHash. For the population of a metric, "active" means any event in the window that you measure. You rarely need to call identify() on its own schedule. It is enough to call it once when traits change. track(event, { unit, at?, idempotencyKey? }) records that a named event happened for a unit. It records nothing more. An event carries no properties, so track() cannot send a value such as an order amount. A metric that is built on an event names the event and a kind. The kind is one of these:

• A count of how often the event happened in the window.
• A proportion, which shows whether the event happened at all.

Events gives both shapes and the queue behind them in detail.

Source 2 — SQL templates against your own database

If the number is already in your own Postgres, you do not need to copy it as an event. A metric can be a SQL template that returns one number for one unit. Quoin runs the template read-only against a replica. You connect the replica once for each workspace. See Database connections for the role, TLS and network rules:

select count(*)
from orders
where user_id = :unit
	and created_at >= :from
	and created_at < :to
	and status = 'paid'

A template can use only the parameters :unit, :from and :to. All three are required. Quoin fills them for each unit. :unit is the unit id as text. :from is the time when the unit was first exposed. :to is the end of the window of the unit. Both times are timestamptz in UTC. You write the expression for each unit, and Quoin wraps it. Because of this, the template never names the unit-id column and never returns more than one column. If the result of a template is NULL for a unit, Quoin reads it as 0 for that unit. A binomial metric needs 0 or 1 (or true/false) for each unit. A count needs a number that is never negative. If a value does not follow these rules, Quoin refuses the read. Quoin does not guess.

Quoin runs the wrapped template over the exposed units in statements of at most 20,000 units each. Every statement runs in its own read-only transaction with a 5-second statement timeout. If the statements return more rows than units, or fewer rows, Quoin refuses the read.

Quoin refuses to save a template, and gives a sentence that says why, when the template meets one of these conditions:

• It uses a parameter other than :unit, :from and :to, misses one of them, or uses a positional $1.
• It holds more than one statement, or starts with anything other than select or with.
• It contains a data-changing or session keyword (insert, update, delete, create, set, copy and the other such keywords), or a set operation (union, intersect, except).
• It calls a function that runs SQL from a string, sleeps, signals, touches files, takes locks, moves a sequence or reaches another server.
• It fails to parse, or it returns something that is not a number. The Postgres error shows under the field.

These checks catch mistakes early. The read-only role, the read-only transaction and the timeout keep your database safe. Quoin supports only Postgres as a metric source. If your own data is in SQLite or D1, Quoin does not support it as a source. Send the number as an event instead.

A workspace that reads from its database can also give Quoin a population template. The template returns unitId, tier, createdAt and lastActiveAt, and it has the one parameter :since. With it, the reach estimate of an experiment counts eligible users from your own tables. For now, you set it only through the outcomeSource.setPopulationTemplate procedure. No screen draws it. The quoin command that sets it from a file is the work of QN-03a-ii.

Defining a metric

A metric names its source and its aggregation. For anything other than a plain count, it also names its direction. The direction shows if a higher value means better or worse. Because of the direction, you can use a metric as a guardrail and as a primary metric. If you create a metric that names an event that Quoin has never seen, Quoin refuses by default. The message is: "No event named "upgraded" has been received yet. If it's about to ship, check "declare anyway" — the metric will read 0 for everyone until it arrives." The event name is inside plain quotation marks in the message, exactly as written here.

Metrics freeze once an experiment starts using them

When the pre-registration of an experiment locks in a metric, the definition of that metric is frozen for the life of the experiment. Quoin refuses an edit and gives the reason: "Used by 1 running experiment (email.digest_send_time). Wait until the experiment ends, or create a new metric." The key of the experiment is in plain parentheses in the message. This rule makes the metricHashes fingerprint of a metric trustworthy. A report can always show if the metric it is about to read is the same metric that the experiment pre-registered against.

As of QN-06, this refusal is real. edit and delete call the metricInUseBy seam. The seam scans the running experiments of a workspace for one that names this metric as its primaryMetric or a guardrail. It returns the first match. The sentence above is the exact string that the code throws.

Next