Datus extensions to Apache Ossie (vendor spec)

• Status: draft, in use by Datus (dosi-engine + the Datus generation agent).
• Spec version: datus-ext/1 (the major series)
• Current version: 1.10 — see §2.1 for what each minor added and how a version bump is decided. Any engine can report the version it implements: dosi info, GET /v1/capabilities, or dosi_engine.DATUS_EXT_VERSION.
• Scope: semantics that Apache Ossie (formerly OSI) core spec 0.2.0.dev0 cannot yet express, carried inside the Ossie-sanctioned custom_extensions field so a document stays fully Ossie-valid and every non-Datus consumer ignores them.

For a task-oriented walkthrough with before/after examples, see the Datus extensions guide. This page is the normative contract.

Separately from model semantics, dosi-engine's connection configuration also speaks Datus: profiles use the agent.yml datasources: vocabulary and a full agent.yml can be passed as --connections directly — see connectors.md.

This is a private vendor extension: it does not need to be accepted by the Ossie working group to be usable. Where an extension generalizes a gap that belongs in the core spec (time granularity, cumulative windows), the long-term home is an Ossie RFC; this document is the interim vehicle Datus ships against today.

1. Why this is safe against Ossie

Ossie's schema is strict (additionalProperties: false everywhere), so we cannot add bare fields like relationship.join_type. But the spec provides a first-class escape hatch on every major object:

custom_extensions:
  - vendor_name: <string>
    data: <JSON string>

custom_extensions is present on SemanticModel, Dataset, Field, Relationship, and Metric (see crates/dosi-model/src/spec.rs). The upstream ossie validator accepts it, so a document carrying Datus extensions passes the validate-osi gate unchanged. A consumer that doesn't understand vendor_name: DATUS simply skips it — the model still means exactly what pure Ossie says it means. The Datus extensions only refine engine behavior at points Ossie leaves undefined; they never change what a metric or dimension is.

2. The envelope

A Datus extension is one custom_extensions entry:

• vendor_name: canonically the uppercase constant "DATUS" (matching the Ossie spec's uppercase enum style, e.g. ANSI_SQL). Matching is case-insensitive; the historical lowercase datus keeps working.
• data: a JSON string (per the Ossie spec data is a string, not an object) decoding to a single JSON object — the payload. At most one DATUS entry per object. Two envelope keys are reserved: v and requires.

Consumption rules (dosi-engine):

• Absent → the documented Ossie default (below). Extensions are purely additive; a model with none behaves exactly as today.
• Unknown payload keys are ignored, not rejected — so a newer Datus agent can emit keys an older engine doesn't read without breaking it (forward-compat). The exception is requires, below.
• Unknown vendor_name (anything but DATUS, compared case-insensitively) is ignored.
• A malformed DATUS payload (not JSON, or a known key with the wrong type) is a structured error (invalid_datus_extension), not a silent default — a Datus-authored document is held to the Datus contract. Compilation still continues with the default, so one pass reports every problem in the model.

v — the declared version

v is MAJOR.MINOR. The string form is canonical ("v": "1.1"); a JSON number is accepted for convenience ("v": 1.1, and the bare integer "v": 1 meaning 1.0).

A JSON number can only carry minors 0–9. 1.10 and 1.1 are the same f64, so a number cannot express minor 10; a number with two or more fractional digits is rejected outright with a message telling you to use the string form. Once the current minor reaches 10, write "v" as a string always.

v is optional and omitting it is always safe — an unversioned payload behaves exactly as it did before the version gate existed, and produces no diagnostic. The version is declared per custom_extensions entry, not per document: a model-level v does not become a default for its datasets, fields, relationships, or metrics.

What stamping a version buys you, given the engine's own version:

Situation	Engine's response	Code
v absent or null	silent — behaves as before the gate	—
v malformed (bad string, wrong JSON type, number with ≥2 decimals)	error	invalid_datus_extension
v below 1.0, or a major newer than the engine's	error	unsupported_datus_ext_version
minor newer than the engine's	warning naming the keys that were dropped; everything the engine knows still applies	datus_ext_version_ahead
a key newer than the v you declared	warning; the key is honored anyway	datus_ext_key_newer_than_declared
minor older than the engine's, keys all within it	silent	—

A future major is the one blanket rejection: a major bump means at least one key changed meaning, and an engine that predates the change has no way to know which — reading the payload anyway would produce quietly wrong numbers. Minors are additive by construction, so they only ever warn.

requires — keys that must not be dropped

data: '{"v": "1.4", "requires": ["semi_additive"], "semi_additive": {"reduce": "last"}}'

requires is an array of key names the producer declares must not be silently ignored. A consumer that does not implement a listed key raises datus_ext_key_required naming it, instead of dropping it.

This exists because a consumer cannot judge the danger of a key it has never heard of. "Unknown keys are ignored" is the right default for a key like join_type, where ignoring it falls back to a documented behavior — but the wrong default for a key like semi_additive, where ignoring it means summing an end-of-period balance and returning a number that is simply wrong. The producer knows which is which; requires is how it says so.

Emit requires for every key whose ignore-impact is silent (see §2.1) and for nothing else: letting an older engine degrade is better than refusing to serve it. dosi info / GET /v1/capabilities report each key's impact.

In Datus mode, a non-null silent key omitted from requires emits datus_ext_requires_missing, even when v is absent. The model still compiles; add the named key to requires to protect older consumers.

2.1 Versioning policy

Every key this engine reads is registered with the version that introduced it and what it costs a consumer to ignore it. The registry lives in crates/dosi-compiler/src/ext.rs and is the source of truth: the version gate, the basic-mode warning text, dosi info and GET /v1/capabilities all read it at runtime, so none of them can drift from it. The change log in §7 is written by hand against that registry.

Ignore-impact levels, ordered by increasing danger — note that failing loudly ranks safer than quietly returning different numbers:

Impact	Meaning
inert	Presentation metadata the engine never consumed. Results are identical.
degraded	No wrong numbers — the request fails loudly instead (a structured error, or a query name that stops resolving).
documented	Numbers change, but predictably and in a documented direction (LEFT vs INNER, NULL vs 0).
silent	Numbers are wrong and the caller cannot tell. Must be listed under requires.

Two shipped keys are silent: measure (D-MEASURE, 1.4) and time (D-FORMAT, 1.6) — see the change log in §7. Every other key is degraded or documented, so basic mode remains coherent for models that use neither. A model that uses one must list it under requires, which is what stops a datus-mode engine that does not implement the key (an older Dosi, another vendor's reader of the envelope) from silently computing a different number: it fails at the version gate with datus_ext_key_required instead. Basic mode is a different case — it never reads the payload at all, requires included, and only warns ignored_vendor_extension; a model with a silent key should not be run in basic mode.

When to bump:

• MINOR — purely additive: a new key, or a new accepted value for an existing key that no old payload could have used. Old payloads keep working unchanged; a new payload on an old engine gets datus_ext_version_ahead and loses only the new key. A new key whose impact is silent is still a minor bump, but the producer must list it under requires.
• MAJOR — only when an existing key changes meaning, is renamed, or is withdrawn. Every payload declaring an older v is then rejected per changed key, with an instruction for updating the YAML.
• Neither — engine-internal work that changes no key's contract. A version with no key behind it would make the registry's invariants vacuous.

3. Extension catalog

D-JOIN — relationship join type

Where: a Relationship's custom_extensions. Payload key: join_type, enum "left" | "inner". Default (absent): "left".

Controls how a fan-out branch follows this relationship (many→one) when it joins the "one" side in to reach a dimension or an off-base measure:

• "left" (default): keep rows on the many side that have no match — orphan fact rows survive and land in the NULL group of any dimension drawn from the missing side. Reconciliation / audit semantics.
• "inner": drop unmatched many-side rows. Attribution semantics — a measure is only counted where the entity it attributes to actually exists.

This makes the INNER-vs-LEFT choice an explicit property of the relationship instead of a hard-coded generator default (see docs/semantics.md S-JOIN).

relationships:
  - name: sales_to_regions
    from: sales
    to: regions
    from_columns: [region_id]
    to_columns: [region_id]
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.0", "join_type": "inner"}'

D-CONFORM — conformed relationships between facts

Where: a Relationship's custom_extensions. Payload key: cardinality, enum "many_to_one" | "many_to_many". Default (absent): "many_to_one" — the ordinary foreign-key edge the join walk follows. Impact if ignored: silent — list it in requires.

Two fact tables often share dimensions — a date, a platform — without either being the many side of the other: a login row does not belong to a payment row. Strict OSI can relate them only through a shared dimension table, so a cross-fact metric like payers per active player by date has no date to group by (unconformed_dimension). many_to_many declares the pairing directly: each from_columns / to_columns pair is the same dimension spelled on each side.

relationships:
  - name: login_payment_conformed
    from: netbar_login
    to: netbar_payment
    from_columns: [stat_date, platform_id]
    to_columns:   [pay_date,  platform_id]
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.8", "requires": ["cardinality"], "cardinality": "many_to_many"}'

A conformed edge is never joined row to row — that would fan every SUM / COUNT out. Each fact aggregates alone, projects its own spelling of the paired column under the query's output name, and the branches merge on that name exactly as S-FAN-3 merges any two branches:

{"metrics": ["pay_rate"], "group_by": ["netbar_login.stat_date"]}

WITH m0 AS (SELECT netbar_payment.pay_date AS stat_date, COUNT(DISTINCT netbar_payment.player_id) AS payers
            FROM main.netbar_payment GROUP BY netbar_payment.pay_date),
     m1 AS (SELECT netbar_login.stat_date AS stat_date, COUNT(DISTINCT netbar_login.player_id) AS users
            FROM main.netbar_login GROUP BY netbar_login.stat_date)
SELECT COALESCE(m0.stat_date, m1.stat_date) AS stat_date, CAST(m0.payers AS DOUBLE) / m1.users AS pay_rate
FROM m0 FULL OUTER JOIN m1 ON m0.stat_date = m1.stat_date OR m0.stat_date IS NULL AND m1.stat_date IS NULL

What the pairing makes possible (fixture fixtures/datus_conform/):

• Either spelling works in group_by, where and time_range.dimension, for metrics on either fact — netbar_payment.pay_date groups a login-only metric too; the output column keeps the spelling the query used.
• metric_time resolves per branch when the facts' primary time dimensions are paired: each branch filters and groups its own date.
• A grain rolls up per branch (stat_date:month): each fact truncates its own column and the merge key is the truncated value. Facts stored at different grains may be paired; the coarser side's D-GRAIN floor applies.
• Transitive: payment ↔ login and login ↔ register pair payment with register too — three facts, one date dimension.
• Only the paired columns are shared. Grouping a cross-fact metric by an unpaired column is unconformed_dimension; filtering on one is no_join_path, with a hint that the filter cannot be applied to one side alone (make the filtered member a D-DERIVE filter metric instead). A conformed edge cannot be named in a relationship path (invalid_relationship_path) — there is no row on the other side to reach.
• S-FAN-6 still holds: a count inside a ratio reads NULL for a date that has rows on one side only; only a bare count metric fills 0.
• A pair may be stored differently on each side — a native DATE on one fact, a D-FORMAT %Y%m%d string on the other. A pairing says the columns are one dimension, not one encoding, so every bound the planner moves across is folded into the target column's encoding: the single time range, each tagged range's CASE gate, and the scan predicate those gates OR together. The one thing that cannot cross is a where_sql comparison — D-FORMAT refuses an encoded column there (the literal would not be in its encoding), and naming the pair's native side is refused the same way rather than silently rewritten (unsupported_filter, naming both spellings). Use time_range with that dimension instead.
• A window metric may be grouped by either spelling as long as its axis is one column. A window over one fact asked for under the other fact's name is that fact's own series, ordered by its own column, under the name the query used. A window metric whose own members span two paired facts has no single column to order by and stays not_implemented.

Declared once, checked once: paired columns must agree on dimension.is_time and, when both declare a datatype, on the type; a dataset contributes at most one column to a conformed class; join_type is invalid on a many_to_many edge (invalid_datus_extension). Model validation still warns relationship_target_not_unique on the edge — correct under strict OSI, and the one signal a consumer that ignores the key gets.

Why silent: an engine that drops the key sees an ordinary many→one edge and joins the facts row to row. A COUNT DISTINCT compose then returns plausible numbers (every login without a same-day payment falls into the NULL bucket); a SUM across the join is refused as fan_out_risk only when the dimension sits on the other fact, and double-counts otherwise. Listing cardinality in requires makes a datus-mode engine without it fail at the version gate instead. Basic mode (§5) is that degradation by design.

Design: ../design/d-conformed-edges.md; cases: ../design/d-conform-e2e-cases.md, and ../design/d-lod.md (4.5, the D cases in 9.3) for alignment on a level of detail.

D-FILL — metric null-fill for missing merge groups

Where: a Metric's custom_extensions. Payload key: fill_nulls_with, a JSON number. Default (absent): engine default — a bare COUNT metric fills 0 (docs/semantics.md S-FAN-6); every other metric keeps NULL.

An explicit fill_nulls_with takes precedence over the S-FAN-6 count default and applies to any metric kind (SUM, ratio, expression), COALESCE-ing the metric's output to the given number in both merge and single-branch plans.

After a multi-branch merge, a group present in only some branches projects NULL for the metrics evaluated in the missing branch. fill_nulls_with: <n> COALESCEs this metric's value to <n> (typically 0) so a group with no contributing rows reports a concrete number instead of a blank.

Scope guard (unchanged from S-FAN-6): fill is applied to the metric's output, never to a measure embedded in a ratio/expression — a count denominator must not become a literal 0 divisor. For a ratio, fill the ratio, and the engine still evaluates it null-safely.

Under a window stage the count default lands one layer lower — on the merged measure column itself, since the stage and everything above it read columns rather than branch-qualified expressions. Same answer either way: adding a window metric to a query never moves a count from 0 to NULL.

metrics:
  - name: sales_total
    expression:
      dialects:
        - dialect: ANSI_SQL
          expression: SUM(sales.amount)
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.0", "fill_nulls_with": 0}'

D-TIME — primary (aggregation) time dimension

Where: a Dataset's or a Metric's custom_extensions. Payload key: time_dimension, a string name reference — on a dataset, the name of one of its own is_time fields; on a metric, "field" or "dataset.field". Default (absent): a dataset with exactly one is_time field gets it as implicit primary time (this inference reads no extension, so it applies in basic mode too); zero or several is_time fields → no primary time.

Declares which time column is a table's/metric's business time axis — the column the engine uses when a query asks for a time range or grain without naming a column. Resolution per metric (S-TIME-5):

1. the metric's own time_dimension (highest precedence);
2. else the unique primary time among the metric's datasets (explicit dataset time_dimension, else the single-is_time inference).

A metric-level bare "field" resolves against the metric's own datasets first (time-field names like etl_dt recur across datasets), widening to the whole model only if none of them has the field.

Queries consume this through the reserved name metric_time (--group-by metric_time:month, --time-dimension metric_time), and through the time-range fallback: a range with no named dimension and no time item in the group-by filters each metric's primary time — per aggregation branch, on that branch's own column. A metric with no resolvable primary time yields no_primary_time_dimension. Metrics on one base with different primary times are not an error: each time axis gets its own branch over that base, and the branches merge on the group keys, so each metric returns what it returns alone.

The reference must exist and have an effective is_time: true (explicit or from the core datatype default), else invalid_datus_extension.

datasets:
  - name: orders
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.1", "time_dimension": "order_date"}'
    fields:
      - name: order_date
        dimension: { is_time: true }
      - name: ship_date
        dimension: { is_time: true }
metrics:
  - name: shipped_revenue
    expression:
      dialects: [{ dialect: ANSI_SQL, expression: SUM(orders.amount) }]
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.1", "time_dimension": "ship_date"}'

D-GRAIN — native time granularity

Where: a Field's custom_extensions (the field must be dimension.is_time: true, explicitly or from the core datatype default). Payload key: time_granularity, enum "day" | "week" | "month" | "quarter" | "year". Default (absent): inferred from time.format or core datatype: Date (day); otherwise unknown — any query-time grain may be requested.

Declares the grain the column is stored at (a monthly snapshot's etl_dt, say). Requesting a strictly finer grain (:day on a month column, via an explicit group-by item or via metric_time) is a structured grain_too_fine error instead of silently wrong data. Truncation to the native grain or coarser is unchanged. Interim vehicle for the upstream RFC's dimension.time.granularity.

fields:
  - name: etl_dt
    dimension: { is_time: true }
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.1", "time_granularity": "month"}'

D-FORMAT — time storage format

When to configure it

Use D-FORMAT when a time field stores dates as strings or integers, such as '20260907' or 202609. Native DATE/TIMESTAMP fields normally need no configuration: that is the default when time.format is absent.

time.format describes the stored encoding, not the output display format. Declare it in a field's DATUS custom_extensions. Its effective is_time must be true, either explicitly or from the core datatype default. The declaration describes the result of the field's expression, which may be a calculation rather than a source column.

Quick start

Within a dataset's fields, declare a string date and an integer month as follows. storage is inferred from datatype: String gives "string", Integer gives "int". Without datatype, it still defaults to "string". Include "time" in requires so older Datus-mode engines that do not support it reject the model instead of ignoring it.

fields:
  - name: dt
    expression:
      dialects: [{dialect: ANSI_SQL, expression: dt}]
    datatype: String
    dimension: {is_time: true}
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.6", "requires": ["time"], "time": {"format": "%Y%m%d"}}'
  - name: etl_month
    expression:
      dialects: [{dialect: ANSI_SQL, expression: etl_month}]
    datatype: Integer
    dimension: {is_time: true}
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.6", "requires": ["time"], "time": {"format": "%Y%m", "storage": "int"}}'

For a dataset named events, a query's time-range fragment still uses YYYY-MM-DD, regardless of the stored encoding. The start is inclusive and the end exclusive:

time_range:
  dimension: events.dt
  start: "2026-01-01"
  end: "2026-04-01"

Writing filters

Use time_range to filter D-FORMAT encoded fields. Its bounds always use YYYY-MM-DD; the engine converts them to the declared storage encoding.

where_sql rejects any reference to a D-FORMAT encoded field with unsupported_filter, even when the literal uses the correct encoding. This also applies to computed dataset fields, references inside functions or CAST, IS NULL, IN, BETWEEN, and comparisons between fields. Remove the reference from where_sql and use time_range instead. Conditions that cannot be expressed by time_range, such as selecting isolated dates or NULL values, are not supported through where_sql for these fields.

Native temporal declarations (date, datetime, timestamp, timestamp_tz) remain allowed in where_sql, as do fields without a D-FORMAT encoding.

This restriction applies to query where_sql. D-DERIVE filter and filters inside metric expressions still require literals in the stored encoding; they do not convert ISO date literals automatically. For example, a %Y%m%d string field requires '20260401', not '2026-04-01'. Use dosi list dimensions to inspect time_format and time_storage.

Configuration reference

The time object accepts format, storage, and granularity. Besides the native type names below, format accepts a continuous, ordered, fixed-width prefix of %Y%m%d%H%M%S. %Y is a four-digit year; %m, %d, %H, %M, and %S are two-digit, zero-padded month, day, hour (00–23), minute, and second. Stop after any component, but do not skip, repeat, or reorder them.

Any fixed literal text may appear between components or as a prefix/suffix: %Y-%m, %Y年%m月%d日, %Y.%m.%dT%H:%M, and %Y%m%d%H%M%S are valid. Write %% for a literal percent sign. Unknown directives, variable-width modifiers, fractional seconds, and timezone directives are not supported. For example, %Y%d, %d/%m/%Y, %-m, and %Y%m%d%z are rejected.

These combinations extend the existing 1.6 contract without a version bump. Engines predating this enhancement reject layouts they do not recognize:

custom_extensions:
  - vendor_name: DATUS
    data: '{"v":"1.6","requires":["time"],"time":{"format":"%Y年%m月%d日"}}'

Every stored value must follow the declared layout, including literal text and padding. Bare-column range comparisons require a column collation that orders these fixed-width values chronologically; use an appropriate binary/code-point ordering when necessary. The declaration does not inspect or change collation.

The original six encodings remain supported:

format	Stored value	Inferred grain	Allowed storage
%Y%m%d	20260907	day	string, int
%Y-%m-%d	2026-09-07	day	string
%Y/%m/%d	2026/09/07	day	string
%Y%m	202609	month	string, int
%Y	2026	year	string, int
%Y-%m-%d %H:%M:%S	2026-09-07 12:30:00	Not inferred	string

Native temporal declarations do not parse or re-encode the field:

format	Meaning	Inferred grain
date	Native date	day
datetime, timestamp, timestamp_tz	Native date and time	Not inferred

Prefer datatype: Date, DateTime, or DateTimeTz for new native temporal fields; no time.format is needed. The legacy spellings remain supported: date must agree with Date, datetime and timestamp with DateTime, and timestamp_tz with DateTimeTz. Native temporal datatypes reject stored patterns such as %Y%m%d. Time has no supported D-FORMAT encoding.

Native declarations do not accept storage. For encoded fields, an explicit storage must agree with datatype (String → "string", Integer → "int"); other declared datatypes cannot use patterns. Omitted storage is inferred from the datatype, or defaults to "string" when datatype is absent. "int" accepts only uninterrupted component prefixes without literal text (including literal digits), such as %Y%m%d%H or %Y%m%d%H%M%S. The longest encoding has 14 digits: choose a sufficiently wide integer-valued column, such as BIGINT or DECIMAL(n,0), for a compact timestamp. A physical DECIMAL(n,0) column holding an integer date key may declare logical Integer; logical Decimal is not an integer encoding. storage cannot be declared without format. These checks use the field expression's result type, not the source column type. Basic mode ignores the Datus block with a warning, while retaining the core datatype time-role default.

granularity accepts day, week, month, quarter, or year. It may be omitted when the format implies it. time.granularity and time_granularity are equally supported spellings: neither is deprecated, but they must agree when both are present. A declared grain cannot be finer than the encoding. A time object may declare only granularity, but cannot be empty. Year/month/day prefixes infer year/month/day regardless of their literal text; prefixes including a clock component do not infer a grain. Hours, minutes, and seconds in storage do not enable sub-day grouping or date-time time_range bounds: those query features remain unsupported. Parsed output uses January/day one for omitted calendar components and zero for omitted clock components; for example, %Y-%m denotes the first day of that month, and %Y%m%d%H denotes the start of that hour.

Grouping and range boundaries

A %Y%m field can be grouped by month, quarter, or year, but not by day or week: its stored values contain no such detail. Likewise, %Y supports year grouping. Requesting a finer grain produces grain_too_fine.

Range boundaries must align with the stored periods. For %Y%m, end: "2026-03-15" produces grain_too_fine: use "2026-03-01" to exclude March or "2026-04-01" to include it. The error offers both aligned bounds; the engine does not choose whether to include a partial month. Year encodings require year boundaries. These checks apply to the query's bounds; scan expansion for window calculations is handled by the engine.

Selected time dimensions are parsed into date/time values, including when no grain is requested. Do not rely on results retaining the stored text layout. D-FORMAT does not provide an output display-format setting.

Validation and known limits

The declaration must match the data. Dosi checks supported formats and configuration combinations, and — where the field declares core Field.datatype — that the declared storage and native format agree with it. It does not inspect warehouse column types or values, so dosi validate cannot establish that %Y%m%d matches the actual field expression's output.

A wrong format or storage type can cause a warehouse error, NULL groups, or incorrect filtering without an error. For example, declaring %Y%m%d for values stored as '2026-09-07' also generates filter literals in the wrong layout. Check both the expression's type and representative stored values.

ClickHouse uses parseDateTime, whose DateTime result is limited to roughly 1970-01-01 through 2106-02-07. Earlier or later dates, including an SCD2 '99991231' end-date sentinel, are not supported reliably for grouping or parsed projections. Verified on ClickHouse 24.8.14: parseDateTime rejects 1960, 2107, and 9999 dates with CANNOT_PARSE_DATETIME. parseDateTime64 is unavailable in that version. The BestEffort alternatives are not a safe replacement: parseDateTimeBestEffort clamps out-of-range values, and parseDateTime64BestEffort changes 9999-12-31 to 2299-12-31. Bare encoded time_range comparisons do not use this parser, but that does not make an out-of-range grouping safe.

In basic mode, DATUS extensions are ignored with a warning, including requires. Use Datus mode for fields that depend on encoded-date handling. Epoch encodings such as epoch_seconds and epoch_millis are not supported.

How it affects SQL

Grouping parses an encoded field into a date/time value and truncates it when a coarser grain is requested. At the stored grain, or with no requested grain, the GROUP BY key can remain the stored column and the SELECT expression parses it.

For time_range, the engine instead converts the bounds into the stored encoding. The examples above generate predicates of these forms:

WHERE dt >= '20260101' AND dt < '20260401'
WHERE etl_month >= 202601 AND etl_month < 202604

Keeping parsing functions off the filter column helps warehouses use partition pruning. Actual pruning depends on the warehouse, field expression, and partition definition.

D-DATASET — metric home dataset

Where: a Metric's custom_extensions. Payload key: dataset, a string naming a dataset of the model. Default (absent): attribution is derived from the metric SQL alone.

A COUNT(*) names no column, so in a multi-dataset model with no other aggregate pinning a dataset, Ossie leaves its attribution undefined and the engine errors (count_star_needs_dataset). The hint resolves exactly that tie. It never overrides SQL-derived attribution — a metric whose aggregates name columns keeps their datasets regardless of the hint (extensions never redefine what the SQL says). An unknown dataset name is invalid_datus_extension.

metrics:
  - name: chat_message_count
    expression:
      dialects: [{ dialect: ANSI_SQL, expression: "COUNT(*)" }]
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.1", "dataset": "chat_record"}'

D-WINDOW — derived window metrics (period-over-period / rolling / cumulative)

Where: a Metric's custom_extensions. Payload key: window, a JSON object — full schema in window-extension.md. Default (absent): the metric is its plain base aggregate — no derivation.

Declares a window derivation over the metric's own expression (which must infer as a single plain aggregate): the three sugars pop / rolling / cumulative, or the general offset / frame form they desugar to. The time axis is not declared here — D-WINDOW composes with D-TIME (metric time_dimension > dataset primary; a missing axis fails at query time with no_primary_time_dimension). Offsets lower to a calendar-correct shifted-key self-join with automatic scan expansion + output trim; frames lower to agg(v) OVER (… ROWS …), with reset adding a DATE_TRUNC partition key and its own expansion + post-window trim. The remaining query restrictions (one shared time axis among the metrics that need one; a no-reset running metric or a partition-global one beside scan-expanding metrics under a time_range) are structured not_implemented errors; a reset finer than the queried grain is window_reset_too_fine. D-FILL never applies to derived outputs (NULL means "no comparable prior period"). Raw window SQL in metric expressions stays rejected (window_in_metric) — this extension is the only window path.

metrics:
  - name: revenue_mom_growth
    expression:
      dialects: [{ dialect: ANSI_SQL, expression: "SUM(orders.amount)" }]
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": 1, "window": {"type": "pop", "offset": "1 month"}}'

D-DERIVE — derived metrics (filter / compose / lod)

Where: a Metric's custom_extensions. Payload key: derive, a JSON object with a type discriminant — plus the window key's base field (M2, below). Default (absent): the metric is self-contained, no structure metadata. Supported forms: M1 ships filter + compose; M2 adds window.base and compose-with-window-members; 1.10 adds lod (D-LOD, documented with the rest of D-LOD). reagg and cross-dataset filter predicates are reserved and reject not_implemented with a hint naming their replacement or milestone.

Declares how a metric derives from other metrics of the same model — structure Ossie expressions cannot carry. Two degradation tiers: on the flatten tier (filter, compose over leaves/filter members) the metric's expression is a legal, semantically equivalent Ossie aggregate — basic mode computes it and gets the same numbers, and the engine enforces the equivalence at compile time (derive_expression_mismatch); on the stage tier (window.base, compose with window members) no flat expression can express the semantics, so the expression is only the nearest approximation and basic mode legitimately diverges (with the ignored-extension warning). Either way, treat the expression as tool-maintained output, not something to edit by hand.

• filter — {"type":"filter","base":<metric>,"where":<predicate>}: a dimensional-filter variant of base. The predicate folds into the aggregate argument (CASE WHEN pred THEN arg END; COUNT bases count THEN 1), never a WHERE — non-matching time buckets keep existing (value NULL), which is what makes the variant safe under ROWS-framed windows. M1 limits the predicate to the base's own dataset (a cross-dataset predicate is not_implemented until M4) and base to a plain non-derived single aggregate. The engine records the subset relation (subset_of) in metric metadata.
• compose — {"type":"compose","expr":<arithmetic>,"fill":<n>}: scalar arithmetic over metric names ("revenue - total_refunds"). Identifiers resolve strictly to metric names — a qualified or unknown name is an error with candidates, never a silent column read. Aggregate calls are rejected (composition never re-aggregates). fill replaces a member's NULL after branch alignment, before the arithmetic (the cross-fact "missing bucket poisons the sum" fix). The engine detects linear combinations (Σ cᵢ·metricᵢ), records members + coefficients + the leaf-measure decomposition, and fixes the conformed dimension set (dimensions every member branch reaches, under the unique-join-path rule) at compile time — dosi list exposes it, and a query grouping outside it gets unconformed_dimension naming the member that misses the dimension. Members sharing no dimension warn compose_no_conformed_dimensions at compile time. Attribution metadata is labeled exact only for linear composes over SUM/COUNT-tier measures (a filtered COUNT(DISTINCT) overlaps its complement — never exact).
• lod — {"type":"lod","expr":<level of detail>}: the view-relative level-of-detail keywords, avg(include(base, dims…)) and exclude(base, dims…), and any compose over one. Unlike the other two families the metric's expression is not an equivalent fallback — it is the nested aggregate the declaration describes, which no engine can execute, so basic mode fails closed rather than computing one level of it. Documented in full with the field-hosted half: D-LOD.

M2 additions:

• window.base — the window key (D-WINDOW) accepts "base": <metric>: the window computes over the named metric's series instead of this metric's own expression. The base may be a leaf, a filter metric, a ratio (1.9), a compose — whether its measures sit in one dataset or several — or a compose over another window's output (1.9). A cross-fact base's series only becomes a value after the branch merge, so the stage sits on the merge instead of on one aggregate. Rejected: naming a window metric directly (the inner window would have no stage of its own to sit on — wrap it in a compose, which gives it one). Metadata records derive_family: "window", the derive_base chain and the inherited subset_of. Windows over a filter base inherit the CASE-WHEN bucket preservation: buckets with no matching rows still exist, so ROWS frames stay calendar-correct and cumulative sums skip the NULLs.

A ratio or compose base is what makes "rank the markets by close rate", "this rate versus last month's" and "am I above the average rate" expressible — none of which a single aggregate can state.

Nested windows (1.9) go one step further: a window reading a compose over another window's output is a second level, lowered as one more SQL stage. The scorecard shape needs it — rank the markets within a brand, then weight the scores within a market — because the two comparisons partition in different directions and one OVER has one PARTITION BY. This is not the re-aggregation I-1 bans: the row count is unchanged, each level recomputes with the query, and the weighted value lands on every row rather than folding them away. Levels are capped at 3, and past the first one partition must be declared explicitly — the query_dimensions default reads clearly on a single stage, but stacked, "which dims, minus whose exclusions" is no longer answerable from one declaration. Note what the window does and does not average: a whole-partition avg over a ratio base is the unweighted mean of the per-row rates, which is not the pooled ratio of sums. Both are legitimate and they differ; the window is the one that says "mean of the rates". - compose with window members — a compose whose expr references window metrics runs the arithmetic over the window output ("cumulative AOV" = revenue_cum / order_count_cum: numerator and denominator each accumulate, then divide). Members may mix window metrics with leaves; all members share one window stage, so they must share a time axis (cross-fact window members are rejected). Attribution is always approximate and there is no leaf-measure decomposition — window output cannot be served from materialized aggregates. - filter members in compose — a compose member may be a filter metric ("new_order_count / order_count"); it composes on the flatten tier exactly like a leaf.

Compose members that are themselves composes are expanded inline at compile time: the compiled artifact is flat, exactly as if the inner arithmetic had been written out. That is what lets a formula be stated once and reused — and, because the outer's expression is checked against the expansion of the inner's declaration, editing the inner and leaving a stale copy in a consumer is a derive_expression_mismatch rather than a number nobody questions.

Four consequences worth knowing before you nest:

• The expression is the full expansion. Write out the whole flattened arithmetic, including the parentheses the nesting implies and the CAST(… AS DOUBLE) an inner bare-ratio member compiles to. It is tool-maintained output — regenerate it from the declaration rather than editing it by hand.
• fill pushes down to the leaves. Each leaf reference takes the innermost fill declared on its path, so an outer fill: 0 still repairs a missing branch inside a member that declared none. The decision is per reference: one leaf reached through two members that fill it differently keeps both.
• derive_members is the flattened leaf list on the flatten tier, with coefficients folded across the levels (quadrupled = doubled × 2 over doubled = revenue × 2 reports revenue at 4). Conformed dimensions intersect along the DAG, and unconformed_dimension therefore names the leaf that misses the dimension, not the intermediate compose. dosi lineage still draws the edge to the member you wrote.
• On the stage tier a compose member stays a member. A window anywhere in the tree puts the metric on the stage tier, and there inlining would be wrong: each member is its own series out of the window stage, so dissolving an intermediate compose into leaves would lose arithmetic the stage never projects (cum_aov / revenue would lose the division and become two window members over a leaf). derive_members is therefore the DIRECT members there, the planner computes each nested compose as a hidden stage output, and substitution runs innermost-first at lowering. The tier decision reads the whole tree, so a window on an intermediate compose stages everything above it too.
• Nesting is capped at 256 member references in one expansion — an inlined member is substituted at every occurrence, so referencing one twice doubles the expansion per level.

Cycles among metric references are metric_reference_cycle with the full path. Metadata surfaces on every host (dosi list, REST, MCP, Python): derive_family, derive_base, derive_expr, subset_of, derive_members, leaf_measures, attribution, conformed_dimensions, and required_dimensions — the dims a query has to group by, gathered over the whole chain (window-extension.md#frozen-degrees).

metrics:
  - name: net_revenue
    expression:
      dialects: [{ dialect: ANSI_SQL, expression: "COALESCE(SUM(orders.amount), 0) - COALESCE(SUM(refunds.ref_amount), 0)" }]
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.4", "derive": {"type": "compose", "expr": "revenue - total_refunds", "fill": 0}}'
  # Reuses net_revenue; `expression` is the expansion, so a change to
  # net_revenue's declaration breaks this metric loudly.
  - name: net_revenue_per_order
    expression:
      dialects: [{ dialect: ANSI_SQL, expression: "(COALESCE(SUM(orders.amount), 0) - COALESCE(SUM(refunds.ref_amount), 0)) / COUNT(orders.order_id)" }]
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.4", "derive": {"type": "compose", "expr": "net_revenue / order_count"}}'

D-MEASURE — explicit measure name

Where: a Metric's custom_extensions. Payload key: measure, {"name": <identifier>}. Default (absent): the sanitizer-generated stem (S-MEASURE-1).

Names the metric's single synthesized measure explicitly. Measure names are otherwise derived by sanitizing the aggregate argument ASCII-only, so two aggregates differing only in non-ASCII content (CASE literals such as the Greek 'νέο' / 'παλιό' — "new" / "old") collapse to one stem and fail measure_name_collision — the model cannot compile at all. The explicit name is a label: dedup still keys on the aggregate signature (an undeclared textual twin shares the declared name), the name enters the same collision checks (two explicit names on one signature, or one name on two signatures, still collide loudly), and it applies only to single-aggregate metrics. Because measure names surface in result columns and metadata, ignoring this key silently renames them — emit it under requires so older engines refuse instead (the first silent-impact key since requires shipped).

metrics:
  - name: new_product_revenue
    expression:
      dialects: [{ dialect: ANSI_SQL, expression: "SUM(CASE WHEN orders.category = 'νέο' THEN orders.amount END)" }]
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.4", "requires": ["measure"], "measure": {"name": "orders_new_product_amount_sum"}}'

D-PARAM — query-time parameters

Where: a Metric's custom_extensions. Payload key: params, a list of {name, type, default, allowed | min/max, description}. Default (absent): no parameters — the metric is exactly what it declares.

A metric's window width, offset count, rank bucket count, navigation position, or a literal in its filter predicate becomes a typed query-time parameter: one moving_avg(n) instead of r7_revenue / r30_revenue / r90_revenue, one ltv(n) instead of eight ltvN columns. Parameters fill typed holes only — never SQL text — so a model validates once for every legal binding, and a binding can never inject anything.

metrics:
  - name: moving_avg
    expression:
      dialects: [{ dialect: ANSI_SQL, expression: "SUM(sales.amount)" }]
    custom_extensions:
      - vendor_name: DATUS
        data: |
          {"v": "1.5",
           "params": [{"name": "n", "type": "int", "default": 3, "min": 1, "max": 12,
                       "description": "trailing window width in buckets of the queried grain"}],
           "window": {"type": "rolling", "function": "avg", "periods": {"param": "n"}}}
  - name: paid_within_n
    expression:
      dialects: [{ dialect: ANSI_SQL, expression: "SUM(CASE WHEN base.datediff_pay < 7 THEN base.sum_pay_amount END)" }]
    custom_extensions:
      - vendor_name: DATUS
        data: |
          {"v": "1.5",
           "params": [{"name": "n", "type": "int", "default": 7, "allowed": [1, 3, 7, 14, 30, 60, 90, 180]}],
           "derive": {"type": "filter", "base": "pay_total", "where": "datediff_pay < :n"}}

Declaration. Every parameter has a name ([a-z][a-z0-9_]*, unique within the metric), a type (int | float | string), a mandatory default, and a domain — allowed (an explicit value list) or min/max (a closed range, either side optional); the two are mutually exclusive. The domain is the governance boundary: it says which definitions this metric is allowed to take. A string parameter must declare allowed (an unbounded string parameter would be an arbitrary dimension value) and takes no min/max; an int/float parameter without any domain compiles with the datus_param_unbounded warning. default must lie in the domain.

Slots. A parameter is referenced from exactly these places:

Slot	Where	Syntax	Parameter type
offset.count	window.offset.count (object form; the "1 month" sugar takes no parameter)	{"param": "n"}	int, domain ≥ 1
frame.preceding	window.frame.preceding	{"param": "n"}	int, domain ≥ 0
rolling.periods	window.periods under type: rolling	{"param": "n"}	int, domain ≥ 1
rank.buckets	window.rank.buckets (ntile)	{"param": "k"}	int, domain ≥ 1
value.n	window.value.n (nth_value)	{"param": "k"}	int, domain ≥ 1
filter.literal	a literal inside a D-DERIVE filter metric's where	:n (bind syntax)	int / float / string

The window slots are integer counts, so they take int parameters whose domain has a declared lower bound at or above the slot's own minimum; a filter.literal hole takes any of the three types (region = :r, amount > :t). One parameter may fill several slots, and one predicate may hold several parameters (datediff_pay < :n AND platid = :plat). A parameter-only predicate (:n < 7) is rejected — a filter must still test a column. :name anywhere else — a metric expression, a query-time where_sql — is an error, never a silent literal. A parameter no slot references compiles with datus_param_unused.

The compiled model is instantiated at every parameter's default, so a model that declares parameters plans exactly like one that does not until a query binds something. For a filter slot the authored expression must be that default instantiation (< 7 above): the D-DERIVE equivalence gate holds the two together, and a bound value re-materializes the same measure a hand-written literal would produce — paid_within_n at n = 30 and a hand-written SUM(CASE WHEN datediff_pay < 30 …) share one column of the base CTE.

Binding. A query binds by name, for every queried metric that declares the parameter — directly or through its derive members (a compose inherits its members' parameters, so ltv = paid_within_n / new_user_cnt takes n):

{"metrics": ["new_user_cnt", "ltv"], "group_by": [{"field": "base.register_date"}],
 "params": {"n": 30}}                          // one binding
{"metrics": ["new_user_cnt", "ltv"], "group_by": [{"field": "base.register_date"}],
 "params": {"n": [1, 3, 7, 14, 30, 60, 90, 180]}}   // a list: one column per value

Omitted parameters take their defaults; a binding that no queried metric declares is unknown_metric_param (with the declared names as candidates); a value outside the type or domain — including 7.5 or "7" for an int, or a list where the parameter is used by attribution — is param_out_of_domain with a ready-to-send suggested_retry. The one implicit conversion: a float parameter accepts a JSON integer. Lists expand the metric into one column per value (several lists take the cartesian product, first declared parameter outermost), capped at 64 columns per query (param_expansion_too_large).

Output. A default binding leaves the column named after the metric; a non-default binding names it {metric}__{param}_{value} (moving_avg__n_6, ltv__n_30; float 1.5 → _1_5, strings sanitized: o'neil → o_neil), several parameters in declaration order, and under a list binding every column is suffixed. The rule exists so two dashboards can never show the same metric name over different windows — and every response carries the machine-readable form regardless: outputs, one entry per metric column with its metric and resolved params (defaults included), in the CLI JSON, the REST and MCP responses, the Python dicts, explain, and as Arrow field metadata (dosi.metric, dosi.params).

The sanitizer is ASCII-only, so it cannot always keep a string domain's values apart: an all-CJK allowed list collapses onto one suffix, and so does us-east next to us_east. Such a value takes an opaque token instead — h followed by 8 hex digits of a frozen FNV-1a hash of the value (by_channel__c_h0898d4b7), widened to 16 on the astronomically unlikely collision. The token is a function of the value and its declared domain and nothing else, so it does not move when the query binds a different set, when allowed is reordered, or when a value is added to it. A value keeps its stem whenever that stem is unique in the domain and is not itself of the reserved h<hex> shape, so a readable ASCII domain is named exactly as before. The model is told at compile time which values went opaque, and with what token, by the datus_param_opaque_token warning; outputs[].params carries the value, and dosi query --format text prints a legend under the table.

An order key names a metric (by its model name) or a group-by item (by its output name, or by the spelling the query used — sales.sale_date:month as well as sale_date__month), never a column the engine assigned. A metric key resolves while the metric is one column, suffix and all; a metric expanded by a list binding is unknown_order_key, because there is no spelling that picks one of its columns. To rank by one binding, bind that one value — the retry on the error is the params fragment that does it.

Attribution binds one value per parameter (a list is refused with a single-value retry) and echoes the resolved bindings in comparison_metadata.params.

Catalog. dosi list metrics, GET /v1/metrics and MCP list_metrics carry each metric's params (declaration plus the slots it fills); describe_metric adds param_schema, a JSON Schema for the params map (allowed → enum, min/max → minimum/maximum) — an agent reads it, then binds.

Compatibility. Under --osi-basic the D-WINDOW / D-DERIVE hosts are ignored, so params is inert with them (one ignored_vendor_extension warning) and any query-time params is unknown_metric_param. An engine older than 1.5 rejects {"param": …} in a window slot structurally (the field is an integer there) and the "v": "1.5" envelope at the version gate — which is the guard that matters for :name in a filter, since a 1.4 engine would otherwise render the bind marker into SQL. Note that MetricQuery tolerates unknown fields: a 1.4 server silently drops a client's params and answers with the defaults. Check GET /v1/capabilities (or dosi info) for params before relying on it.

D-DIM — which fields are offered as dimensions

Where: a Field's custom_extensions. Payload key: is_dimension, a boolean. Default (absent): the engine infers the field's role (see below).

Ossie types every non-time field as a regular dimension, so a model cannot say that a column is a measurement (unit_cost), an identifier (c_customer_id), free text, or PII. Grouping by such a column is not a structural error — the SQL runs, the numbers are right, and the analysis is meaningless. is_dimension is how the model says so once, at modelling time, instead of every caller re-deciding it at every call.

It is a listing signal, not a permission:

Written	Effect
absent	the engine infers (below)
false	the field is not recommended: list_dimensions marks it is_dimension: false, and attribution does not offer it as a candidate. Naming it in group_by still works.
true	the field is recommended even where inference would exclude it — the way to group a numeric column into buckets

Because it never reaches the planner, ignoring the key changes no number; it only costs a caller a worse shortlist (documented impact).

datasets:
  - name: store_sales
    fields:
      - name: ss_quantity            # a measurement no metric aggregates
        expression: { dialects: [{ dialect: ANSI_SQL, expression: ss_quantity }] }
        custom_extensions:
          - vendor_name: DATUS
            data: '{"v": "1.7", "is_dimension": false}'
      - name: i_current_price        # numeric, but analyzed in price bands
        expression: { dialects: [{ dialect: ANSI_SQL, expression: i_current_price }] }
        custom_extensions:
          - vendor_name: DATUS
            data: '{"v": "1.7", "is_dimension": true}'

Inference when the key is absent. The engine uses only model structure — it never probes the warehouse (a COUNT(DISTINCT …) on a high-cardinality column is not a cost the engine may impose). A field is not offered when it is a primary-key member, a single-column unique key of the metric's own home dataset, a column named in a relationship's join pair, or a column some metric measures — the value read by a SUM/AVG/STDDEV*/VAR*. Everything else is offered, and list_dimensions reports which rule applied, as source.

Three aggregates that mention a column without measuring it, and therefore leave it a dimension:

• COUNT — COUNT(DISTINCT region) ("how many active regions") is an ordinary metric over an ordinary dimension. This holds for the metric being listed too: the answer for a field does not depend on which metric you ask about.
• MIN / MAX — they order strings and dates perfectly well; MIN(activity_code) picks one identifier, it does not measure it.
• A condition inside an aggregate — SUM(CASE WHEN is_instant THEN 1 ELSE 0 END) counts rows where a flag holds; the flag stays a dimension. Only the THEN/ELSE values are measured. A D-DERIVE filter folds into exactly this shape, so it behaves the same way.

Inference cannot see a business key that is neither a declared key nor a relationship column, a numeric column no metric reads, free text, or PII — those are exactly the fields worth declaring.

D-LOD — fixed level-of-detail fields

Where: a Field's custom_extensions. Payload key: lod, an object {"expr": "fixed(base, dims…)"} — one level-of-detail expression, the same grammar a metric's derive: {type: "lod"} uses. base names a metric that is a single plain aggregate; the dimensions are fields of the field's own dataset, and naming none is the table-scoped FIXED — Tableau's {MAX([date])}, one value broadcast onto every row. Default (absent): the field is an ordinary row-level attribute.

A lod field's value is not a stored column: it is base evaluated over the dataset grouped by dims — Tableau's {FIXED [dims]: base}. That makes the result of an aggregation usable as an axis: how many users were active exactly N days, where N is itself COUNT(DISTINCT date) per user. Queries carry no new syntax; the field is groupable, filterable and aggregatable like any other, and the engine plants the rollup stage — a two-stage plan when the view and its filters are functions of the rollup keys, the broadcast below otherwise.

fields:
  - name: active_day
    # the aggregate verbatim — see "the expression is not a sentinel" below
    expression:
      dialects: [{ dialect: ANSI_SQL, expression: "COUNT(DISTINCT event_date)" }]
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.10", "requires": ["lod"],
                "lod": {"expr": "fixed(active_days, sessions.user_id)"}}'

One grammar, two carriers. fixed is the same word here and in a metric's derive, parsed by the same parser; what differs is what each carrier accepts. A field takes only fixed, and no outer aggregate — its value is one per key, so there is nothing to collapse. The view-relative keywords are the metric's, because their grain is a function of the query.

Form	Field	Metric
fixed(base, dims…)	yes — the value is a column of the row	yes — the value attaches to the view's rows
include(…) / exclude(…)	no — a field has no "view"	yes
outer aggregate agg(…)	no	include requires one; exclude and fixed forbid one

// how many users seen on channel 2 were active exactly N days in January
{"metrics": ["user_num"],
 "group_by": ["sessions.active_day"],
 "where": "sessions.channel_id = 2",
 "time_range": {"start": "2025-01-01", "end": "2025-02-01"}}

SELECT f.active_day, COUNT(DISTINCT sessions.user_id) AS user_num
FROM main.lod_session AS sessions
JOIN (SELECT user_id, COUNT(DISTINCT event_date) AS active_day
      FROM main.lod_session
      WHERE event_date >= DATE '2025-01-01' AND event_date < DATE '2025-02-01'  -- time_range: context
      GROUP BY user_id) AS f
  ON sessions.user_id IS NOT DISTINCT FROM f.user_id
WHERE sessions.channel_id = 2                                                    -- where: dimension filter
  AND sessions.event_date >= DATE '2025-01-01' AND sessions.event_date < DATE '2025-02-01'
GROUP BY f.active_day

The January window scopes the per-user count; the channel filter selects rows after it, so a user active on two channels in January is a 2-day user on channel 2 too — the order Tableau evaluates FIXED in (see filter placement).

Re-aggregating a pinned value is spelled exactly once: an ordinary metric that aggregates the field (AVG(sessions.daily_users) over a daily_users field keyed by event_date is the average daily actives; group it by metric_time:month for the monthly series). There is no separate keyword for it.

Where a filter applies: Tableau's order of operations

active_day is a function of the rows it is computed over, so where a predicate applies decides the number, not just its coarseness. The layers follow Tableau's order of operations — context filters, then FIXED, then the filter shelf:

• time_range (and time_ranges) and context_filter are the context layer. The analysis window passes into every rollup's input, so the query above counts each user's days in January. This is what the window means in every other Datus stage — metric_time, D-WINDOW, attribution — and it is what a Tableau author gets by adding the date filter to context. context_filter does the same for any other condition: with context_filter: "sessions.channel_id = 2" the days are counted on channel 2 only. A context condition cannot read a D-LOD field, since it feeds the rollup that computes it.
• The where clause is a dimension filter. It selects the rows the view aggregates and never reaches a fixed rollup's input: the query above counts each user's January days across every channel, then keeps the channel-2 rows. A filter on a joined dimension (channel.channel_name = 'pc') is a dimension filter too. A conjunct that names only the rollup's own keys (sessions.user_id <> 'A') is applied to the rollup's output instead — the same rows, through the cheaper two-stage plan; any other stored-column conjunct switches the branch to the broadcast, where it filters the detail rows.
• A predicate over a D-LOD field applies to rollup values — between the two stages, or after the broadcast joins — and never feeds back into any rollup's input. That is what makes cohorts work: where: "sessions.first_day BETWEEN …" selects a population by acquisition date while still rolling its members up over all history. It holds across key families too: where: "sessions.daily_users >= 4" on an active_day distribution keeps the session rows that fall on busy days, while active_day itself is still counted over every date.
• Relationships split the same way. A fixed rollup joins only the datasets its own expression, keys and context layer read — Tableau's relationships (the logical layer), where a FIXED value is computed on the tables it references. A relationship the view joins for a dimension, the where clause or another metric reaches the detail rows only. So adding a dimension from another dataset never changes a fixed value, even over an INNER relationship: with sessions → promo_day declared inner and only one promotion day, grouping by promo_day.promo_name keeps the view to that day's sessions while active_day still counts each user's whole history. To compute the rollup over matched rows only, name the far side in the context layer (context_filter: "promo_day.promo_name = 'launch'"): the join then enters the rollup, and with it the rows it drops.
• Every conjunct is placed on its own, so an AND of a stored-column condition and a D-LOD condition needs no care. A predicate that names both layers in one expression — orders.event_date = orders.first_order_day (first-day revenue), or an OR / NOT across the two — is evaluated after the broadcast, as a whole.

The where clause therefore has no spelling for the context reading of a condition ("days active on channel 2 only"); context_filter is that spelling, per query. A condition that belongs to the dataset itself, as Tableau's data source filter does, is an inline query source (source: "SELECT * FROM t WHERE channel_id = 2") today; a dataset-level filter extension is tracked as issue #197.

Broadcast: reading the value next to the row

The two-stage plan reads the rollup and nothing else, which works as long as everything the view selects is a rollup key or a D-LOD field of that rollup. When it is not — a dimension outside the keys, a stored column aggregated per LOD group, a row-level expression that reads the value, two key families in one query — the engine broadcasts instead: the fact is read as detail rows, and each row is joined with its entity's rollup row, one derived table per key family, all computed over the same filtered scan. Many rows meet one rollup row, so the join never fans out. Tableau's FIXED does exactly this.

// average daily actives by channel — channel_id is not a key of the per-date rollup
{"metrics": ["avg_daily_users"], "group_by": ["sessions.channel_id"]}

SELECT g.channel_id, AVG(f.daily_users) AS avg_daily_users
FROM (SELECT channel_id, event_date                    -- one row per (channel_id, event_date)
      FROM main.lod_session GROUP BY channel_id, event_date) AS g
JOIN (SELECT event_date, COUNT(DISTINCT user_id) AS daily_users
      FROM main.lod_session GROUP BY event_date) AS f
  ON g.event_date IS NOT DISTINCT FROM f.event_date
GROUP BY g.channel_id

A rollup key counts as "a stored column" here when the aggregate counts rows. The rollup has one row per key, so COUNT(DISTINCT user_id), MIN and MAX over the key read the same there as on the detail rows and keep the two-stage plan; COUNT(user_id), COUNT(*), SUM and AVG count the fact's rows and broadcast — grouped by active_day, two-day users have 8 sessions, not 4.

Two consequences worth knowing:

• Weighting: each key counts once per group. A pinned value is a property of its key, not of the rows that carry it, so aggregating it counts each key once per group — the fact is read as one row per (view dimension × rollup key), the GROUP BY channel_id, event_date derived table above. Channel 2 appears on three dates carrying 4, 4, 1 → 3.0, not the 3.5 its six session rows would give. Tableau computes the same question the same way. What this is not is a weighted average: if you want the detail rows to weigh, aggregate a stored column instead.

The same holds for a row-level field whose expression reads only D-LOD values and constants — CASE WHEN active_day > 1 THEN 1 ELSE 0 END is one value per user however many session rows carry it — so SUM of it counts each user once per group, exactly as summing active_day would, and exactly as Tableau evaluates a calculation over FIXED values at the FIXED grain. A field that also reads a stored column (event_date = first_order_day) is a property of the row and weighs by rows. The grid is keyed by the view dimensions and the keys of the families being aggregated; a family the query only groups by — a D-LOD field, or a field computed from one, as the view dimension — is joined inside the grid to compute that dimension and contributes no key of its own. - One grain per query. A metric that aggregates a pinned value and one that aggregates a stored column are grained differently — once per key versus once per row — and a query asking for both is refused by name rather than answered at one of the two grains. So is a query whose metrics aggregate values pinned at different key families (AVG(active_day) per user beside AVG(daily_users) per date): the grid is deduplicated by the keys being aggregated, and a grid keyed by both would repeat each value once per key of the other — neither metric would match its answer alone. Query them separately. - A NULL key still matches. GROUP BY gives a NULL key its own rollup row, so the join is null-safe (IS NOT DISTINCT FROM, or the equivalent OR form where a dialect needs it) — otherwise those detail rows would leave the aggregate silently, and the same question would read differently depending on which shape the planner picked. On a dialect that cannot hash the OR form this is a slower join; declare the column NOT NULL upstream if that matters. - A row-level field may read a D-LOD field. CASE WHEN event_date = first_order_day THEN 'new' ELSE 'returning' END is a legal field expression: naming a D-LOD field reads the broadcast value, and grouping by the flag gives new-vs-returning revenue with no new syntax. What such a field cannot do is feed the value back into a rollup — as a dims key or a base argument (nested LOD), as a relationship column, or as a time dimension — all refused at compile time.

Which shape a query gets is decided per branch and is visible in --explain (LodRollup vs LodBroadcast). The two-stage shape is kept whenever it suffices, so a query that planned before the broadcast existed plans byte-for-byte as it did.

The rollup keeps the storage encoding

A rollup key is projected as the stored value — no grain truncation, no D-FORMAT parse. The outer stage owns the view: it parses a %Y%m%d key and truncates it to the requested grain exactly once (DATE_TRUNC('MONTH', STRPTIME(f.event_ymd, '%Y%m%d'))). Truncating inside as well would silently coarsen the rollup itself, changing what the field means.

The expression is not a sentinel

The field's expression carries the real aggregate, and the engine checks it agrees with base (derive_expression_mismatch otherwise). Two reasons: it is what a consumer that ignores the extension reads, and an aggregate used as a row-level column fails loudly at the warehouse — strictly better than a sentinel, which would silently group by a wrong row set. That is also how basic mode degrades (§5).

View-relative: include and exclude on a metric

Where: a Metric's custom_extensions. Payload key: derive, family {"type": "lod", "expr": …}. Default (absent): the metric is whatever its expression says.

A fixed field pins one grain for every query, which is why it can be a field: its value is a property of the row. include and exclude pin a grain relative to the query's own group-by, so they cannot be — they are hosted on metrics, where "the value depends on the view" is already the contract.

- name: revenue_per_user
  # the nested aggregate verbatim — see "the expression is not a sentinel"
  expression:
    dialects: [{ dialect: ANSI_SQL, expression: "AVG(SUM(orders.amount))" }]
  custom_extensions:
    - vendor_name: DATUS
      data: '{"v": "1.10", "derive": {"type": "lod",
              "expr": "avg(include(revenue, orders.user_id))"}}'

The expr is authored as SQL so ordinary tooling can lint it, and reads as a tiny grammar:

Form	Inner grouping	Outer aggregate
agg(include(base, dims…))	the view's dimensions plus dims — finer	required: something must collapse the extra rows
exclude(base, dims…)	the view's dimensions minus dims — coarser	forbidden: each view row already reads one inner row

{"metrics": ["revenue_per_user"], "group_by": ["orders.channel_id"]}

SELECT f.channel_id, AVG(f.revenue) AS revenue_per_user
FROM (SELECT orders.channel_id, orders.user_id, SUM(orders.amount) AS revenue
      FROM main.lod_order GROUP BY orders.channel_id, orders.user_id) AS f
GROUP BY f.channel_id

Which one you want is a real question, and the model answers it. Averaging "revenue per user" by channel has two defensible readings, and they differ whenever a user touched more than one channel:

Spelling	Reading	By channel
avg(include(revenue, orders.user_id))	per user within the channel	pc 12.5, mobile 20.5
AVG(orders.user_revenue) over a fixed field	each user's lifetime total, once per channel they ordered on	pc 17.5, mobile 23

Neither is wrong. The engine never picks: each is its own metric, spelled once.

The filter rules are the same as for a field. A query filter and time_range scope the inner grouping, so the level of detail is computed over the rows the query asked about. Two differences from a fixed rollup follow from "view-relative":

• The grain is applied inside. Grouping by metric_time:month groups the inner stage by month and user, then averages — not by day and user. A fixed rollup does the opposite (see above), because there the key is a stored column and the view belongs to the outer stage.
• The outer stage reads the rollup and nothing else, so every metric in the query must be a level of detail of the same grain. A plain metric alongside one is refused by name rather than answered at the wrong grain.

exclude attaches, it does not replace. Where include groups finer and collapses back, exclude groups coarser and carries the result onto every view row: the fact is aggregated twice and the two are joined on the dimensions they share (a LEFT JOIN), or on none at all when the declaration names every view dimension (a CROSS JOIN — the grand total). Each view row meets exactly one of the coarser rows, so the attach cannot fan out.

- name: revenue_grand   # exclude every view dimension: the grand total
  expression:
    dialects: [{ dialect: ANSI_SQL, expression: "SUM(SUM(orders.amount))" }]
  custom_extensions:
    - vendor_name: DATUS
      data: '{"v": "1.10", "derive": {"type": "lod", "expr": "exclude(revenue)"}}'
- name: revenue_share   # what it is for: a divisor the view does not group by
  expression:
    dialects: [{ dialect: ANSI_SQL, expression: "SUM(orders.amount) / SUM(SUM(orders.amount))" }]
  custom_extensions:
    - vendor_name: DATUS
      data: '{"v": "1.10", "derive": {"type": "compose", "expr": "revenue / revenue_grand"}}'

WITH m0 AS (SELECT orders.channel_id, SUM(orders.amount) AS revenue
            FROM main.lod_order GROUP BY orders.channel_id),
     m1 AS (SELECT SUM(orders.amount) AS revenue_grand FROM main.lod_order)
SELECT m0.channel_id, m0.revenue, m0.revenue / m1.revenue_grand AS revenue_share
FROM m0 CROSS JOIN m1

Two things worth knowing:

• A metric composed over a level of detail is one too. revenue_share's own expression is a nested aggregate, because its divisor's value is not expressible as a flat aggregate — so basic mode fails closed on the share column exactly as it does on its member. The engine finds the derivation through the compose: missing it would not be an error, it would silently compute the divisor at the view's own grain and make every share 1.
• The coarser value is recomputed, never re-aggregated. A window function (SUM(SUM(x)) OVER (PARTITION BY …)) would get the same answer in one pass for an additive base, and over-count a COUNT(DISTINCT) one. The side runs the base's own aggregate at the coarser grain, which is right for any aggregate.
• With no group-by at all there is nothing to exclude from, so the metric is simply its base and the plan is one flat aggregation. That is exclude only: a keyless fixed metric still skips the dimension filter with no group-by, so it is still its own side.

fixed on a metric: a value the view does not group by

Where: a Metric's custom_extensions. Payload key: derive, family {"type": "lod", "expr": "fixed(base[, dims…])"}.

The same declaration a field makes, hosted on a metric so that it can attach to the view instead of being a column of it. With no dimensions it is Tableau's table-scoped {SUM([Sales])} — one row, CROSS JOIN'd onto every view row. With dimensions, they must be dimensions the view groups by, and the side attaches through a null-safe LEFT JOIN: each view row meets exactly one of its rows, so the attach cannot fan out.

- name: revenue_fixed_all       # Tableau {SUM([amount])}
  expression:
    dialects: [{ dialect: ANSI_SQL, expression: "SUM(SUM(orders.amount))" }]
  custom_extensions:
    - vendor_name: DATUS
      data: '{"v": "1.10", "derive": {"type": "lod", "expr": "fixed(revenue)"}}'
- name: revenue_share_of_all
  expression:
    dialects: [{ dialect: ANSI_SQL, expression: "SUM(orders.amount) / SUM(SUM(orders.amount))" }]
  custom_extensions:
    - vendor_name: DATUS
      data: '{"v": "1.10", "derive": {"type": "compose", "expr": "revenue / revenue_fixed_all"}}'

fixed and exclude are not two spellings of one thing. Both can be keyless, both attach a total to every view row — and under a filter they give different answers, because fixed is computed before the dimension filter and exclude is the view's own grouping one dimension coarser:

{"metrics": ["revenue", "revenue_share_of_all", "revenue_share"],
 "group_by": ["orders.channel_id"],
 "where": "orders.user_id <> 'A'"}

WITH m0 AS (SELECT orders.channel_id, SUM(orders.amount) AS revenue
            FROM main.lod_order AS orders WHERE orders.user_id <> 'A'   -- the view
            GROUP BY orders.channel_id),
     m1 AS (SELECT SUM(orders.amount) AS revenue_fixed_all
            FROM main.lod_order AS orders),                            -- fixed: no WHERE
     m2 AS (SELECT SUM(orders.amount) AS revenue_grand
            FROM main.lod_order AS orders WHERE orders.user_id <> 'A')  -- exclude: scoped
SELECT m0.channel_id, m0.revenue,
       m0.revenue / m1.revenue_fixed_all AS revenue_share_of_all,
       m0.revenue / m2.revenue_grand     AS revenue_share
FROM m0 CROSS JOIN m1 CROSS JOIN m2

Neither denominator is wrong. "Share of everything we sold" and "share of what this filter selected" are different questions, and Tableau has both keywords for exactly that reason. Which one a metric means is written in the model, once.

A keyed fixed reads the same way — the side groups by the declared dimension and skips the filter, so the filter scopes the numerator only:

{"metrics": ["revenue", "revenue_fixed_by_channel"],   // fixed(revenue, orders.channel_id)
 "group_by": ["orders.channel_id"], "where": "orders.user_id <> 'A'"}
// channel 1  : revenue 25, fixed 25
// channel 2: revenue 35, fixed 82  — the filter removed 47 from the numerator only

Its keys must be the view's. A fixed metric keyed by something the query does not group by would meet many view rows at once, and collapsing them means re-aggregating the fixed values across keys — a grid over (view dimension × key), which is what a fixed field does when an ordinary metric aggregates it. The refusal names both ways out. A query with no group-by is no exception: its view has none of the keys either. For the same reason an outer aggregate over fixed(…) is refused rather than computed.

Cross-fact alignment: one axis, two facts

Two facts can each compute the same per-entity aggregate — how many days a user had a session, how many days they ordered — and the business question puts them on one axis. A conformed pairing whose endpoints are the two fixed fields says so:

relationships:
  - name: sessions_orders_conformed
    from: sessions
    to: orders
    from_columns: [active_day, event_date]
    to_columns: [active_day, event_date]
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.8", "requires": ["cardinality"], "cardinality": "many_to_many"}'

{"metrics": ["user_num_total"], "group_by": ["sessions.active_day"]}

WITH m0 AS (SELECT f.active_day, COUNT(DISTINCT f.user_id) AS sessions
            FROM (SELECT user_id, COUNT(DISTINCT event_date) AS active_day
                  FROM main.lod_session GROUP BY user_id) AS f
            GROUP BY f.active_day),
     m1 AS (SELECT f.active_day, COUNT(DISTINCT f.user_id) AS buyers
            FROM (SELECT user_id, COUNT(DISTINCT event_date) AS active_day
                  FROM main.lod_order GROUP BY user_id) AS f
            GROUP BY f.active_day)
SELECT COALESCE(m0.active_day, m1.active_day) AS active_day,
       COALESCE(m0.sessions, 0) + COALESCE(m1.buyers, 0) AS user_num_total
FROM m0 FULL OUTER JOIN m1 ON m0.active_day = m1.active_day
                           OR m0.active_day IS NULL AND m1.active_day IS NULL

Each fact rolls up for itself; the branches merge on the shared output name. The two facts are never joined — a session row and an order row have nothing to match on, and joining them would multiply both sides' counts. That is the whole reason the edge is a pairing rather than a relationship.

Everything the two mechanisms already do carries over unchanged: either spelling of the paired axis may be grouped by, a row-level filter and time_range scope each fact's own rollup input through its own spelling of the paired time column, and a bare count fills a missing bucket with 0 while a SUM stays NULL (S-FAN-6). A single-fact metric may also be grouped by the other fact's spelling — the branch reads its own, and broadcasts if its measure aggregates a column the rollup does not export.

What D-LOD does not do yet

These are all structured not_implemented refusals naming what would lift them — or why they never lift — never silently-wrong SQL:

Query	Why
agg(fixed(…)) on a metric, or a fixed metric keyed by a dimension the view does not group by	re-aggregating fixed values across keys is the field carrier's shape — a grid over (view dimension × key); declare the fixed field and aggregate it with an ordinary metric
two metrics aggregating values pinned at different key families	one grid is deduplicated by one set of keys; each value would be repeated once per key of the other family. Query them separately
a metric homed on another dataset, with no conformed pairing between the two	there is nothing to align the two axes on; declare the pairing
a plain metric in the same query as an include metric, or a fixed field in the same query as either	the outer stage reads one grouping; two grains, one of them view-relative, are not nested. (A plain metric beside an exclude is fine — the attach leaves the view's own grouping alone.)
include and exclude in one query	they pull the branch's grouping in opposite directions
exclude with a window metric, under time_ranges, or beside a metric homed on another dataset	the attach reads one view-grain aggregation
grouping by or filtering on a view-relative metric	its grain is defined by the group-by, so it cannot be part of it; the error names the fixed field spelling that can
a D-LOD field with a window metric, or under time_ranges	a different stage stack
an include metric with a window metric, or under time_ranges	the outer stage reads only the finer rollup; there is no time column left for a window or a per-tag gate
a where condition on a D-LOD field beside an include, exclude or fixed() metric	the condition filters rollup values, and the view-relative or attached grain would have to be computed around them — the same nesting as grouping by the field
a group-by dimension that projects under the same name as a rollup key it is not (acct.user_id beside a rollup keyed by sessions.user_id)	duplicate_output_name: the grid would take one for the other and join on the wrong values. Rename the field, or group by the key itself
attribution of a metric that reads a level of detail, or across a D-LOD dimension	the analysis compares two windows in one statement, and a rollup is a function of each window's filter context — it would have to be computed once per window. The metric is refused up front as unsupported / level_of_detail (and is attributable: false in dosi list metrics); the dimension is skipped with a dimension_skipped warning before any statement runs
dosi select projecting or filtering on a D-LOD field, or on a field that reads one	the detail plane has no rollup stage; the error names the dosi query spelling

A keyless fixed field is not among these: lod with no dimensions is the table-scoped level of detail, broadcast onto every row so a row-level expression can read it.

Also fixed by construction: include / exclude are view-relative, so they are hosted on metrics rather than fields (above) — declaring one on a field names where it belongs. A D-LOD field must not be dimension.is_time: true, and neither may a field that reads one: a rollup value is not an axis a time range or grain can be pushed to. A temporal datatype alone does not make either one an axis — a first-order date typed Date is fine, it is just never a time dimension. Nested LOD — a dims entry or a base argument that is, or reads, a D-LOD field, or a base (of a field or of a derive) that is itself a level-of-detail metric, whatever order the document declares them in — is refused, as is a D-LOD read in a relationship column. A window.base naming a level-of-detail metric is refused too (not_implemented): a window over a level of detail is not supported.

4. Precedence & defaults summary

Point	Ossie core	Datus extension	Engine default when absent
relationship join type	undefined (LEFT assumed)	D-JOIN join_type	left
fact-to-fact conformed pairing	inexpressible (every relationship is a join)	D-CONFORM cardinality	many_to_one — the edge is a join
missing-group metric value	undefined	D-FILL fill_nulls_with	count→0 (S-FAN-6), else NULL
primary time dimension	undefined (is_time only)	D-TIME time_dimension (metric > dataset)	the dataset's single is_time field, else none
native time grain	no explicit grain; Date implies day	D-GRAIN time_granularity	inferred from type/encoding, otherwise unknown
time column storage format	logical datatype; no encoding layout	D-FORMAT time	assumed native DATE/TIMESTAMP
COUNT(*) home dataset	undefined in multi-dataset models	D-DATASET dataset	SQL-derived, else count_star_needs_dataset
window derivation	inexpressible (window SQL rejected)	D-WINDOW window	plain base aggregate
query-time parameters	inexpressible	D-PARAM params	none — every slot is the literal it declares
which fields are offered as dimensions	every non-time field is a dimension	D-DIM is_dimension	inferred from keys, relationships and aggregated columns
aggregate-as-dimension	inexpressible (a field is row-level)	D-LOD lod	none — a field is a stored column

5. Validation & engine modes

• Ossie gate (scripts/validate_osi.py): unaffected — custom_extensions is valid Ossie, so all fixtures still PASS ossie.
• Datus self-validation (engine in --osi-datus mode, the default, when it reads a DATUS entry): the payload must be a JSON object; known keys must match their declared type/enum; otherwise invalid_datus_extension. Unknown keys pass (forward-compat) unless named in requires. The envelope version gate (§2) runs here, once per payload — including on the SemanticModel carrier, which defines no keys of its own but is still held to the envelope.
• Basic mode (--osi-basic): the engine treats DATUS like any unknown vendor — the extension is inert (spec-compliant passthrough) and the compile collects one ignored_vendor_extension warning per carrying element — SemanticModel, Dataset, Field, Relationship, and Metric alike — naming the default that applies instead (D-JOIN → LEFT, D-FILL → no fill beyond S-FAN-6, D-TIME → single-is_time inference only, D-GRAIN → any grain, D-DATASET → SQL-derived attribution only, D-WINDOW → the plain base aggregate, D-FORMAT → the column is assumed to be a native date, which for a string or integer column means a warehouse type error or silently wrong rows, D-LOD → the field is dropped, D-CONFORM → the edge is an ordinary many→one join, so the two facts are joined row to row and duplicate-sensitive aggregates across it fan out). D-LOD is the one that cannot degrade to a value: without the declaration the field's expression is a bare aggregate, which is not a legal row-level attribute, so the field is dropped with an aggregate_field_dropped warning and any metric or query referencing it fails closed with unknown_field / unknown_column. The alternative would be AVG(COUNT(DISTINCT …)) reaching the warehouse. The payload is not parsed, so a malformed payload is also just ignored-with-warning — and no version diagnostic can occur in basic mode at all. The same model file is valid in both modes; only the semantics differ. (The single-is_time primary-time inference is not an extension — it applies in basic mode too, so metric_time still works wherever pure Ossie determines a unique time axis.)
• Implementation boundary: all extension reading — and the mode branch itself — lives in one module, crates/dosi-compiler/src/ext.rs. Future extensions land there with the same shape; the rest of the engine stays extension-agnostic.

6. Non-goals / boundaries

• Datus extensions never redefine an Ossie object's identity or a metric's SQL — only engine behavior at Ossie-undefined seams.
• They are not a place to smuggle vendor SQL. Metric expressions stay Ossie expression.dialects; the engine still infers metric semantics from that SQL.
• Semantics that belong in Ossie core (granularity, time spine, window/cumulative metrics) may eventually be standardized upstream; if/when adopted there, the corresponding Datus extension (D-GRAIN, D-WINDOW) is retired in favor of the core field.

7. Change log

Each released minor and the keys it added. The machine-readable form of this table is capabilities().history — dosi info --format json or GET /v1/capabilities.

Version	Date	Keys added	Impact if ignored
1.0	2026-07-11	join_type (D-JOIN), fill_nulls_with (D-FILL)	documented, documented
1.1	2026-08-02	time_dimension (D-TIME), time_granularity (D-GRAIN), dataset (D-DATASET)	degraded, documented, degraded
1.3	2026-08-09	(none — in-family growth of window)	(inherits window's documented)
1.4	2026-08-12	derive (D-DERIVE M1), measure (D-MEASURE)	degraded, silent
1.5	2026-08-26	params (D-PARAM M1)	degraded
1.6	2026-09-02	time (D-FORMAT)	silent
1.7	2026-09-11	is_dimension (D-DIM)	documented
1.8	2026-09-16	cardinality (D-CONFORM)	silent
1.9	2026-09-16	(none — in-family growth of window)	(inherits window's documented)
1.10	2026-09-23	lod (D-LOD; also the lod family of derive)	degraded

• 1.0 (2026-07-11): initial — D-JOIN (relationship join_type, implemented: dosi-compiler reads it into RelationshipIr.join_kind, planner emits INNER/LEFT accordingly), D-FILL (metric fill_nulls_with, implemented 2026-07-12: MetricIr.fill_nulls_with → planner COALESCEs the metric output; overrides the S-FAN-6 count default, any metric kind).
• 1.1 (2026-08-02): time & attribution — D-TIME (dataset/metric time_dimension, implemented: DatasetIr.primary_time_dimension + MetricIr.time_dimension, the reserved metric_time query name, per-branch time-range fallback), D-GRAIN (field time_granularity, implemented: FieldIr.time_granularity, finer-grain requests → grain_too_fine), D-DATASET (metric dataset, implemented: COUNT(*) home attribution). Basic-mode ignored_vendor_extension warnings now cover all five carrier objects (previously only Relationship and Metric).
• 1.1 (2026-08-04, no version bump): the envelope version gate itself — v is now read and enforced (§2), requires is introduced, each key carries a declared ignore-impact, and the whole registry is published through dosi info, GET /v1/capabilities, and dosi_engine.DATUS_EXT. No key was added or changed, so the version stays at 1.1. Payloads without a v — which is what the Datus agent emits today — are unaffected.
• 1.2 (2026-08-06): window metrics — D-WINDOW (metric window, implemented: MetricIr.window → the planner's WindowStage projection root; pop/rolling/cumulative sugars + general offset/frame form, cumulative reset with scan expansion + post-window trim; ignore-impact documented — the metric falls back to its plain base aggregate; spec in window-extension.md). Promotes the 13 baisheng derived-time cases Deferred→Supported (55/2). Initially DuckDB-executed; the all-dialect enablement (per-dialect offset-shift lowerings, all-dialect snapshots, live-corpus verification, datus_window corpus scenario) landed as a follow-up. In-family additions (same minor, fail-closed on engines predating them): mixed families / mixed shifts / mixed resets-with-start relaxed, and require_full_window on finite frames (2026-08-06) — see window-extension.md#restrictions.
• 1.3 (2026-08-09): window families & functions — grows the window key in-family, no new registry row (capabilities().history still ends at 1.2; the version signals the payload dialect). Additions: the rank family (row_number / rank / dense_rank / ntile / percent_rank / cume_dist over the metric value), the value family (first_value / last_value / nth_value), offset direction: "forward" (LEAD — next-period reference), frame order / partition / units / second modifiers (value-ordered and RANGE frames, partition modes, two-argument inputs), and registry levels W2 (stddev_pop/samp, var_pop/samp) + W3 (covar_pop/samp, corr). Every addition fails closed on a 1.2 engine: unknown top-level family keys fall into the exactly-one-family error, unknown in-family keys hit deny_unknown_fields, unknown function names fail enum parse — and a declared "v": "1.3" trips the version gate besides. Spec: window-extension.md (family sections + function registry).
• 1.4 (2026-08-12): derived metrics & measure naming — D-DERIVE M1 (derive key, implemented: filter + compose families, the resolve_metric_refs pass with cycle detection, the expression equivalence gate, compile-time conformed dimensions, and derive metadata on every host surface; reagg / cross-dataset filter predicates / window.base are reserved and reject not_implemented naming their milestone — note window.base previously fell into the silently-ignored unknown-key bucket and now fails closed) and D-MEASURE (measure key, implemented: explicit measure naming, signature-keyed dedup and collision checks unchanged — the fix for the non-ASCII stem collision, and the first silent-impact key: emit it under requires). M2 (2026-08-19, in-family — no new keys, no version bump): window.base flips from reserved to functional (windows over leaf / filter metrics), compose accepts window members (stage-tier lowering: arithmetic over window output) and filter members (flatten tier). A 1.4 payload using these shapes compiles on an M2 engine and fails closed (not_implemented / rejection) on the M1 engine — never a silent misread. Nested compose (in-family — no new keys, no version bump): a compose member may itself be a compose, expanded inline at compile time. An older engine rejects the shape with the same invalid_datus_extension it always did, so this too fails closed. The metric stays on the flatten tier, and derive_members becomes the transitive leaf list with folded coefficients; a window anywhere in the tree keeps the combination not_implemented.
• 1.6 (2026-09-02): time storage formats — D-FORMAT (field time block, implemented: FieldIr.time_format, SqlBackend::parse_time on the grouping side and a Rust-side literal fold on the filter side, with the grain inferred from the layout). time.granularity joins the flat time_granularity as an equal spelling — no deprecation, but the two may not disagree. The second silent-impact key: emit it under requires, so a datus-mode engine that lacks it fails at the version gate (basic mode never reads requires; see §2). A consumer that drops it does not fail loudly; it assumes a native date column and, for a month- or year-stored one, returns an empty result with no diagnostic. See D-FORMAT for configuration and limitations. Scope: string and integer layouts plus the native temporal types; epoch_seconds / epoch_millis are deferred because from_unixtime reads the session timezone and that needs its own round.

In-family addition (2026-09-08, still 1.6): time.format accepts continuous ordered fixed-width calendar prefixes with arbitrary fixed literal text. Filtering still folds date bounds to stored constants; grouping extracts components into a canonical date/time before dialect parsing. Existing layouts remain compatible. Older Datus engines reject new layouts rather than silently interpret them. - 1.7 (2026-09-11): field roles — D-DIM (field is_dimension, implemented: FieldIr.is_dimension, read by list_dimensions and by attribution's candidate list). The model can finally say that a column is a measurement, an identifier, free text or PII, so it stops being offered as a grouping dimension. It is a recommendation signal only: an explicitly named group_by is honored either way and no compiled SQL changes, hence documented impact and no requires entry. Where the key is absent the engine infers the role from model structure alone — keys, relationship columns and numerically aggregated columns — and never probes the warehouse. See D-DIM. - 1.8 (2026-09-16): conformed fact-to-fact relationships — D-CONFORM (cardinality key on a Relationship, implemented: RelationshipIr.kind, the join graph's conformed classes, per-branch resolution of group-by / filter / time-range columns and of metric_time to each fact's own spelling, merge on the output name). A many_to_many edge is never joined; only its paired columns are shared. The third silent-impact key: emit it under requires — without it the edge is a many→one join and duplicate-sensitive aggregates fan out. Design: ../design/d-conformed-edges.md; fixture: fixtures/datus_conform/. - 1.9 (2026-09-16): a ranked scorecard — grows the window key in-family the way 1.3 did, so there is no new registry row. Four additions driven by one shape.

frame.scope: "partition" states the frame as the entire partition instead of an offset from the current row, lowering to <func>(…) OVER (PARTITION BY …) with no ORDER BY and no frame clause. It is the second window function beside a rank — "how many rows am I ranked against" — which a ranked scorecard needs and which no combination of the existing keys could express: count with preceding: "unbounded" is a running count, and units: "range" still stops at the current row's peer group. scope is mutually exclusive with preceding, reset, require_full_window, units and order, each of which only describes an ordered frame; writing one is rejected rather than ignored.

window.base widens: a ratio or a compose can now be the series a window reads, so ranking a rate or taking its period-over-period is expressible. A cross-fact compose's series only becomes a value after the branch merge, so the window stage sits on the merge — the branches aggregate separately and their CTEs ride beside the stage's base. Such a base has no shared time axis (each fact carries its own), so the families that need one refuse on conformance.

Fails closed on a 1.8 engine twice over: FrameSpec and OrderSpec are deny_unknown_fields, and the payload's "v": "1.9" trips the version gate. See window-extension.md.

Nested windows and NULL placement complete the shape. order.nulls: "first" | "last" pins where NULLs sort in a window ordering — left unset, each warehouse applies its own default and they disagree (DuckDB and PostgreSQL last ascending, MySQL-family and StarRocks first), which a ratio base turns from theoretical into routine. The generator emits the clause only where it differs from the target's default, and where a dialect has no such clause at all (MySQL, TiDB) the placement lowers to a leading CASE WHEN … IS NULL sort key — so the rendering varies by dialect and the meaning does not. And window.base goes one step further than a ratio: it accepts a compose over another window's output, so a second window can read the first one's. Each level lowers to one more SQL stage; depth is capped at 3 and level 2+ must declare partition. The guards that matter run across levels, not within one: the output trim stays outside the outermost stage, and a partition-global metric at any level refuses to share a query with a scan-expanding one at any other. Also in-family: a stage-tier compose may have a compose member (it stays a member rather than being inlined, since on that tier each member is its own series). - 1.10 (2026-09-23): levels of detail — D-LOD, one grammar on two carriers: the lod key on a Field and the lod family of the existing derive key on a Metric (implemented).

On a field, fixed(base, dims…) makes a per-entity aggregate groupable, filterable and re-aggregatable through an ordinary query (FieldIr.lod → the planner's LodRollup stage; a D-FORMAT rollup key keeps its storage encoding and is parsed once, outside). When the view needs more than the rollup — a dimension outside its keys, a row-level field reading the value, several key families — the broadcast shape (LodBroadcast) joins the detail rows with one derived table per key family; an aggregate over a pinned value counts each of its keys once per group. No keys is the table-scoped FIXED, broadcast onto every row.

On a metric, include groups the inner stage by the query's own group-by plus the declared dimensions and collapses it with the declared outer aggregate; the grain and any D-FORMAT parse apply inside. exclude attaches a coarser aggregation of the same fact to the view's rows — LEFT JOIN on the dimensions they share, CROSS JOIN on none — recomputed rather than windowed so a COUNT(DISTINCT) base is right too. fixed is the field's declaration hosted on a metric so it can attach to the view: a table-scoped or key-scoped total computed before the dimension filter, which is what Tableau's {SUM([Sales])} is as a denominator and what exclude() cannot mean (above). A conformed pairing whose endpoints are two facts' own fixed fields aligns them on one axis: each fact rolls up for itself and the branches merge, the facts are never joined.

Filter placement follows Tableau's order of operations. The where clause is a dimension filter — applied to a fixed rollup's output when it names only the rollup's keys, to the detail rows otherwise, never inside the rollup — while time_range is the context filter that scopes every rollup. A row-level field that reads only D-LOD values aggregates once per key.

Basic mode drops a D-LOD field (aggregate_field_dropped) and fails closed on a D-LOD metric, whose expression is the nested aggregate its declaration describes (nested_aggregate), rather than computing one level of it. Design, cases and the Tableau comparison: ../design/d-lod.md; fixtures: fixtures/lod/, fixtures/tableau_official_*/.

Not yet implemented (roadmap)

The queued RFCs are all additive, so each lands as one minor bump with its own registry rows. Order may change; every entry occupies a minor of its own.

Minor	RFC	Keys	Carrier	Impact	requires?
≥1.11	rfc-semi-additive-metrics §4	semi_additive	Metric	silent — an end-of-period balance gets summed across snapshots	yes
≥1.11	rfc-minimal-declarations §3 D-DISTINCT-STATE	distinct_state	Field / Dataset	silent — a materialized bitmap column is aggregated as if it were an ordinary column	yes
≥1.7	rfc-derived-metrics M3-M4	derive reagg + cross-dataset filter (window.base and compose window/filter members shipped in-family at 1.4/M2)	Metric	degraded/error — reserved shapes reject not_implemented today, never silently compute	no
—	window-extension.md rank/share/streak families, W2+ functions	future window shapes (in-family additions, fail-closed on older engines)	Metric	degraded	no

Three silent keys have shipped: measure (D-MEASURE) at 1.4, time (D-FORMAT) at 1.6 and cardinality (D-CONFORM) at 1.8. If you author models against a Datus agent that emits silent-impact keys, keep an eye on the requires list it produces: that is what stops an older engine from quietly returning wrong numbers, and it only works because requires itself shipped at 1.1 — ahead of the first key that needs it.