Datus extensions to Apache Ossie (vendor spec)¶

Status: draft, in use by Datus (dosi-engine + the Datus generation agent).
Spec version: datus-ext/1 (the major series)
Current version: 1.11 — see §2.1 for what each minor added and how a version bump is decided. Any engine can report the version it implements: dosi info, GET /v1/capabilities, or dosi_engine.DATUS_EXT_VERSION.
Scope: semantics that Apache Ossie (formerly OSI) core spec 0.2.0.dev0 cannot yet express, carried inside the Ossie-sanctioned custom_extensions field so a document stays fully Ossie-valid and every non-Datus consumer ignores them.

For a task-oriented walkthrough with before/after examples, see the Datus extensions guide. This page is the normative contract.

Separately from model semantics, dosi-engine's connection configuration also speaks Datus: profiles use the agent.yml datasources: vocabulary and a full agent.yml can be passed as --connections directly — see connectors.md.

This is a private vendor extension: it does not need to be accepted by the Ossie working group to be usable. Where an extension generalizes a gap that belongs in the core spec (time granularity, cumulative windows), the long-term home is an Ossie RFC; this document is the interim vehicle Datus ships against today.

1. Why this is safe against Ossie¶

Ossie's schema is strict (additionalProperties: false everywhere), so we cannot add bare fields like relationship.join_type. But the spec provides a first-class escape hatch on every major object:

custom_extensions:
  - vendor_name: <string>
    data: <JSON string>

custom_extensions is present on SemanticModel, Dataset, Field, Relationship, and Metric. The upstream ossie validator accepts it, so a document carrying Datus extensions passes upstream validation unchanged. A consumer that doesn't understand vendor_name: DATUS simply skips it — the model still means exactly what pure Ossie says it means. The Datus extensions only refine engine behavior at points Ossie leaves undefined; they never change what a metric or dimension is.

2. The envelope¶

A Datus extension is one custom_extensions entry:

vendor_name: canonically the uppercase constant "DATUS" (matching the Ossie spec's uppercase enum style, e.g. ANSI_SQL). Matching is case-insensitive; the historical lowercase datus keeps working.
data: a JSON string (per the Ossie spec data is a string, not an object) decoding to a single JSON object — the payload. At most one DATUS entry per object. Two envelope keys are reserved: v and requires.

Consumption rules (dosi-engine):

Absent → the documented Ossie default (below). Extensions are purely additive; a model with none behaves exactly as today.
Unknown payload keys are ignored, not rejected — so a newer Datus agent can emit keys an older engine doesn't read without breaking it (forward-compat). The exception is requires, below.
Unknown vendor_name (anything but DATUS, compared case-insensitively) is ignored.
A malformed DATUS payload (not JSON, or a known key with the wrong type) is a structured error (invalid_datus_extension), not a silent default — a Datus-authored document is held to the Datus contract. Compilation still continues with the default, so one pass reports every problem in the model.

`v` — the declared version¶

v is MAJOR.MINOR. The string form is canonical ("v": "1.1"); a JSON number is accepted for convenience ("v": 1.1, and the bare integer "v": 1 meaning 1.0).

A JSON number can only carry minors 0–9. 1.10 and 1.1 are the same f64, so a number cannot express minor 10; a number with two or more fractional digits is rejected outright with a message telling you to use the string form. Once the current minor reaches 10, write "v" as a string always.

v is optional and omitting it is always safe — an unversioned payload behaves exactly as it did before the version gate existed, and produces no diagnostic. The version is declared per custom_extensions entry, not per document: a model-level v does not become a default for its datasets, fields, relationships, or metrics.

What stamping a version buys you, given the engine's own version:

Situation	Engine's response	Code
`v` absent or `null`	silent — behaves as before the gate	—
`v` malformed (bad string, wrong JSON type, number with ≥2 decimals)	error	`invalid_datus_extension`
`v` below `1.0`, or a major newer than the engine's	error	`unsupported_datus_ext_version`
minor newer than the engine's	warning naming the keys that were dropped; everything the engine knows still applies	`datus_ext_version_ahead`
a key newer than the `v` you declared	warning; the key is honored anyway	`datus_ext_key_newer_than_declared`
minor older than the engine's, keys all within it	silent	—

A future major is the one blanket rejection: a major bump means at least one key changed meaning, and an engine that predates the change has no way to know which — reading the payload anyway would produce quietly wrong numbers. Minors are additive by construction, so they only ever warn.

`requires` — keys that must not be dropped¶

data: '{"v": "1.4", "requires": ["semi_additive"], "semi_additive": {"reduce": "last"}}'

requires is an array of key names the producer declares must not be silently ignored. A consumer that does not implement a listed key raises datus_ext_key_required naming it, instead of dropping it.

This exists because a consumer cannot judge the danger of a key it has never heard of. "Unknown keys are ignored" is the right default for a key like join_type, where ignoring it falls back to a documented behavior — but the wrong default for a key like semi_additive, where ignoring it means summing an end-of-period balance and returning a number that is simply wrong. The producer knows which is which; requires is how it says so.

Emit requires for every key whose ignore-impact is silent (see §2.1) and for nothing else: letting an older engine degrade is better than refusing to serve it. dosi info / GET /v1/capabilities report each key's impact.

In Datus mode, a non-null silent key omitted from requires emits datus_ext_requires_missing, even when v is absent. The model still compiles; add the named key to requires to protect older consumers.

2.1 Versioning policy¶

Every key this engine reads is registered with the version that introduced it and what it costs a consumer to ignore it. The registry is built into the engine and is the source of truth: the version gate, the basic-mode warning text, dosi info and GET /v1/capabilities all read it at runtime, so none of them can drift from it. The change log in §7 is written by hand against that registry.

Ignore-impact levels, ordered by increasing danger — note that failing loudly ranks safer than quietly returning different numbers:

Impact	Meaning
`inert`	Presentation metadata the engine never consumed. Results are identical.
`degraded`	No wrong numbers — the request fails loudly instead (a structured error, or a query name that stops resolving).
`documented`	Numbers change, but predictably and in a documented direction (LEFT vs INNER, NULL vs 0).
`silent`	Numbers are wrong and the caller cannot tell. Must be listed under `requires`.

Two shipped keys are silent: measure (D-MEASURE, 1.4) and time (D-FORMAT, 1.6) — see the change log in §7. Every other key is degraded or documented, so basic mode remains coherent for models that use neither. A model that uses one must list it under requires, which is what stops a datus-mode engine that does not implement the key (an older Dosi, another vendor's reader of the envelope) from silently computing a different number: it fails at the version gate with datus_ext_key_required instead. Basic mode is a different case — it never reads the payload at all, requires included, and only warns ignored_vendor_extension; a model with a silent key should not be run in basic mode.

When to bump:

MINOR — purely additive: a new key, or a new accepted value for an existing key that no old payload could have used. Old payloads keep working unchanged; a new payload on an old engine gets datus_ext_version_ahead and loses only the new key. A new key whose impact is silent is still a minor bump, but the producer must list it under requires.
MAJOR — only when an existing key changes meaning, is renamed, or is withdrawn. Every payload declaring an older v is then rejected per changed key, with an instruction for updating the YAML.
Neither — engine-internal work that changes no key's contract. A version with no key behind it would make the registry's invariants vacuous.

3. Extension catalog¶

D-JOIN — relationship join type¶

Where: a Relationship's custom_extensions. Payload key: join_type, enum "left" | "inner". Default (absent): "left".

Controls how a fan-out branch follows this relationship (many→one) when it joins the "one" side in to reach a dimension or an off-base measure:

"left" (default): keep rows on the many side that have no match — orphan fact rows survive and land in the NULL group of any dimension drawn from the missing side. Reconciliation / audit semantics.
"inner": drop unmatched many-side rows. Attribution semantics — a measure is only counted where the entity it attributes to actually exists.

This makes the INNER-vs-LEFT choice an explicit property of the relationship instead of a hard-coded generator default (see docs/semantics.md S-JOIN).

relationships:
  - name: sales_to_regions
    from: sales
    to: regions
    from_columns: [region_id]
    to_columns: [region_id]
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.0", "join_type": "inner"}'

D-CONFORM — conformed relationships between facts¶

Where: a Relationship's custom_extensions. Payload key: cardinality, enum "many_to_one" | "many_to_many". Default (absent): "many_to_one" — the ordinary foreign-key edge the join walk follows. Impact if ignored: silent — list it in requires.

Two fact tables often share dimensions — a date, a platform — without either being the many side of the other: a login row does not belong to a payment row. Strict OSI can relate them only through a shared dimension table, so a cross-fact metric like payers per active player by date has no date to group by (unconformed_dimension). many_to_many declares the pairing directly: each from_columns / to_columns pair is the same dimension spelled on each side.

relationships:
  - name: login_payment_conformed
    from: netbar_login
    to: netbar_payment
    from_columns: [stat_date, platform_id]
    to_columns:   [pay_date,  platform_id]
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.8", "requires": ["cardinality"], "cardinality": "many_to_many"}'

A conformed edge is never joined row to row — that would fan every SUM / COUNT out. Each fact aggregates alone, projects its own spelling of the paired column under the query's output name, and the branches merge on that name exactly as S-FAN-3 merges any two branches:

{"metrics": ["pay_rate"], "group_by": ["netbar_login.stat_date"]}

WITH m0 AS (SELECT netbar_payment.pay_date AS stat_date, COUNT(DISTINCT netbar_payment.player_id) AS payers
            FROM main.netbar_payment GROUP BY netbar_payment.pay_date),
     m1 AS (SELECT netbar_login.stat_date AS stat_date, COUNT(DISTINCT netbar_login.player_id) AS users
            FROM main.netbar_login GROUP BY netbar_login.stat_date)
SELECT COALESCE(m0.stat_date, m1.stat_date) AS stat_date, CAST(m0.payers AS DOUBLE) / m1.users AS pay_rate
FROM m0 FULL OUTER JOIN m1 ON m0.stat_date = m1.stat_date OR m0.stat_date IS NULL AND m1.stat_date IS NULL

What the pairing makes possible:

Either spelling works in group_by, where_sql and time_range.dimension, for metrics on either fact — netbar_payment.pay_date groups a login-only metric too; the output column keeps the spelling the query used.
metric_time resolves per branch when the facts' primary time dimensions are paired: each branch filters and groups its own date.
A grain rolls up per branch (stat_date:month): each fact truncates its own column and the merge key is the truncated value. Facts stored at different grains may be paired; the coarser side's D-GRAIN floor applies.
Transitive: payment ↔ login and login ↔ register pair payment with register too — three facts, one date dimension.
Only the paired columns are shared. Grouping a cross-fact metric by an unpaired column is unconformed_dimension; filtering on one is no_join_path, with a hint that the filter cannot be applied to one side alone (make the filtered member a D-DERIVE filter metric instead). A conformed edge cannot be named in a relationship path (invalid_relationship_path) — there is no row on the other side to reach.
Listed once. list_dimensions for a metric reports a conformed dimension as one row, under the spelling on the metric's own dataset (any spelling is still a legal group_by). When the paired columns are the metric's time axis, that row is metric_time.
S-FAN-6 still holds: a count inside a ratio reads NULL for a date that has rows on one side only; only a bare count metric fills 0.
A pair may be stored differently on each side — a native DATE on one fact, a D-FORMAT %Y%m%d string on the other. A pairing says the columns are one dimension, not one encoding, so every bound the planner moves across is folded into the target column's encoding: the single time range, each tagged range's CASE gate, and the scan predicate those gates OR together. The one thing that cannot cross is a where_sql comparison — D-FORMAT refuses an encoded column there (the literal would not be in its encoding), and naming the pair's native side is refused the same way rather than silently rewritten (unsupported_filter, naming both spellings). Use time_range with that dimension instead.
A window metric may be grouped by either spelling as long as its axis is one column. A window over one fact asked for under the other fact's name is that fact's own series, ordered by its own column, under the name the query used. A window metric whose own members span two paired facts has no single column to order by and stays not_implemented.

Declared once, checked once: paired columns must agree on dimension.is_time and, when both declare a datatype, on the type; a dataset contributes at most one column to a conformed class; join_type is invalid on a many_to_many edge (invalid_datus_extension). Model validation still warns relationship_target_not_unique on the edge — correct under strict OSI, and the one signal a consumer that ignores the key gets.

Why silent: an engine that drops the key sees an ordinary many→one edge and joins the facts row to row. A COUNT DISTINCT compose then returns plausible numbers (every login without a same-day payment falls into the NULL bucket); a SUM across the join is refused as fan_out_risk only when the dimension sits on the other fact, and double-counts otherwise. Listing cardinality in requires makes a datus-mode engine without it fail at the version gate instead. Basic mode (§5) is that degradation by design.

D-FILL — metric null-fill for missing merge groups¶

Where: a Metric's custom_extensions. Payload key: fill_nulls_with, a JSON number. Default (absent): engine default — a bare COUNT metric fills 0 (docs/semantics.md S-FAN-6); every other metric keeps NULL.

An explicit fill_nulls_with takes precedence over the S-FAN-6 count default and applies to any metric kind (SUM, ratio, expression), COALESCE-ing the metric's output to the given number in both merge and single-branch plans.

After a multi-branch merge, a group present in only some branches projects NULL for the metrics evaluated in the missing branch. fill_nulls_with: <n> COALESCEs this metric's value to <n> (typically 0) so a group with no contributing rows reports a concrete number instead of a blank.

Scope guard (unchanged from S-FAN-6): fill is applied to the metric's output, never to a measure embedded in a ratio/expression — a count denominator must not become a literal 0 divisor. For a ratio, fill the ratio, and the engine still evaluates it null-safely.

Under a window stage the rule is the same: the merged measure column stays NULL, and the count default is applied where the bare count metric is projected as an output. A count that a compose or ratio reads as a member — or that sits inside a window.base series — keeps NULL, so a count denominator never becomes a literal 0 divisor. Adding a window metric to a query never moves a selected count from 0 to NULL.

metrics:
  - name: sales_total
    expression:
      dialects:
        - dialect: ANSI_SQL
          expression: SUM(sales.amount)
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.0", "fill_nulls_with": 0}'

D-TIME — primary (aggregation) time dimension¶

Where: a Dataset's or a Metric's custom_extensions. Payload key: time_dimension, a string name reference — on a dataset, the name of one of its own is_time fields; on a metric, "field" or "dataset.field". Default (absent): a dataset with exactly one is_time field gets it as implicit primary time (this inference reads no extension, so it applies in basic mode too); zero or several is_time fields → no primary time.

Declares which time column is a table's/metric's business time axis — the column the engine uses when a query asks for a time range or grain without naming a column. Resolution per metric (S-TIME-5):

the metric's own time_dimension (highest precedence);
else the unique primary time among the metric's datasets (explicit dataset time_dimension, else the single-is_time inference).

A metric-level bare "field" resolves against the metric's own datasets first (time-field names like etl_dt recur across datasets), widening to the whole model only if none of them has the field.

Queries consume this through the reserved name metric_time (--group-by metric_time:month, --time-dimension metric_time), and through the time-range fallback: a range with no named dimension and no time item in the group-by filters each metric's primary time — per aggregation branch, on that branch's own column. A metric with no resolvable primary time yields no_primary_time_dimension. Metrics on one base with different primary times are not an error: each time axis gets its own branch over that base, and the branches merge on the group keys, so each metric returns what it returns alone.

The reference must exist and have an effective is_time: true (explicit or from the core datatype default), else invalid_datus_extension.

datasets:
  - name: orders
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.1", "time_dimension": "order_date"}'
    fields:
      - name: order_date
        dimension: { is_time: true }
      - name: ship_date
        dimension: { is_time: true }
metrics:
  - name: shipped_revenue
    expression:
      dialects: [{ dialect: ANSI_SQL, expression: SUM(orders.amount) }]
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.1", "time_dimension": "ship_date"}'

D-GRAIN — native time granularity¶

Where: a Field's custom_extensions (the field must be dimension.is_time: true, explicitly or from the core datatype default). Payload key: time_granularity, enum "day" | "week" | "month" | "quarter" | "year". Default (absent): inferred from time.format or core datatype: Date (day); otherwise unknown — any query-time grain may be requested.

Declares the grain the column is stored at (a monthly snapshot's etl_dt, say). Requesting a strictly finer grain (:day on a month column, via an explicit group-by item or via metric_time) is a structured grain_too_fine error instead of silently wrong data. Truncation to the native grain or coarser is unchanged. Interim vehicle for the upstream RFC's dimension.time.granularity.

fields:
  - name: etl_dt
    dimension: { is_time: true }
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.1", "time_granularity": "month"}'

D-FORMAT — time storage format¶

When to configure it¶

Use D-FORMAT when a time field stores dates as strings or integers, such as '20260907' or 202609. Native DATE/TIMESTAMP fields normally need no configuration: that is the default when time.format is absent.

time.format describes the stored encoding, not the output display format. Declare it in a field's DATUS custom_extensions. Its effective is_time must be true, either explicitly or from the core datatype default. The declaration describes the result of the field's expression, which may be a calculation rather than a source column.

Quick start¶

Within a dataset's fields, declare a string date and an integer month as follows. storage is inferred from datatype: String gives "string", Integer gives "int". Without datatype, it still defaults to "string". Include "time" in requires so older Datus-mode engines that do not support it reject the model instead of ignoring it.

fields:
  - name: dt
    expression:
      dialects: [{dialect: ANSI_SQL, expression: dt}]
    datatype: String
    dimension: {is_time: true}
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.6", "requires": ["time"], "time": {"format": "%Y%m%d"}}'
  - name: etl_month
    expression:
      dialects: [{dialect: ANSI_SQL, expression: etl_month}]
    datatype: Integer
    dimension: {is_time: true}
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.6", "requires": ["time"], "time": {"format": "%Y%m", "storage": "int"}}'

For a dataset named events, a query's time-range fragment still uses YYYY-MM-DD, regardless of the stored encoding. The start is inclusive and the end exclusive:

time_range:
  dimension: events.dt
  start: "2026-01-01"
  end: "2026-04-01"

Writing filters¶

Use time_range to filter D-FORMAT encoded fields. Its bounds always use YYYY-MM-DD; the engine converts them to the declared storage encoding.

where_sql rejects any reference to a D-FORMAT encoded field with unsupported_filter, even when the literal uses the correct encoding. This also applies to computed dataset fields, references inside functions or CAST, IS NULL, IN, BETWEEN, and comparisons between fields. Remove the reference from where_sql and use time_range instead. Conditions that cannot be expressed by time_range, such as selecting isolated dates or NULL values, are not supported through where_sql for these fields.

Native temporal declarations (date, datetime, timestamp, timestamp_tz) remain allowed in where_sql, as do fields without a D-FORMAT encoding.

This restriction applies to query where_sql. D-DERIVE filter and filters inside metric expressions still require literals in the stored encoding; they do not convert ISO date literals automatically. For example, a %Y%m%d string field requires '20260401', not '2026-04-01'. Use dosi list dimensions to inspect time_format and time_storage.

Configuration reference¶

The time object accepts format, storage, and granularity. Besides the native type names below, format accepts a continuous, ordered, fixed-width prefix of %Y%m%d%H%M%S. %Y is a four-digit year; %m, %d, %H, %M, and %S are two-digit, zero-padded month, day, hour (00–23), minute, and second. Stop after any component, but do not skip, repeat, or reorder them.

Any fixed literal text may appear between components or as a prefix/suffix: %Y-%m, %Y年%m月%d日, %Y.%m.%dT%H:%M, and %Y%m%d%H%M%S are valid. Write %% for a literal percent sign. Unknown directives, variable-width modifiers, fractional seconds, and timezone directives are not supported. For example, %Y%d, %d/%m/%Y, %-m, and %Y%m%d%z are rejected.

These combinations extend the existing 1.6 contract without a version bump. Engines predating this enhancement reject layouts they do not recognize:

custom_extensions:
  - vendor_name: DATUS
    data: '{"v":"1.6","requires":["time"],"time":{"format":"%Y年%m月%d日"}}'

Every stored value must follow the declared layout, including literal text and padding. Bare-column range comparisons require a column collation that orders these fixed-width values chronologically; use an appropriate binary/code-point ordering when necessary. The declaration does not inspect or change collation.

The original six encodings remain supported:

`format`	Stored value	Inferred grain	Allowed `storage`
`%Y%m%d`	`20260907`	`day`	`string`, `int`
`%Y-%m-%d`	`2026-09-07`	`day`	`string`
`%Y/%m/%d`	`2026/09/07`	`day`	`string`
`%Y%m`	`202609`	`month`	`string`, `int`
`%Y`	`2026`	`year`	`string`, `int`
`%Y-%m-%d %H:%M:%S`	`2026-09-07 12:30:00`	Not inferred	`string`

Native temporal declarations do not parse or re-encode the field:

`format`	Meaning	Inferred grain
`date`	Native date	`day`
`datetime`, `timestamp`, `timestamp_tz`	Native date and time	Not inferred

Prefer datatype: Date, DateTime, or DateTimeTz for new native temporal fields; no time.format is needed. The legacy spellings remain supported: date must agree with Date, datetime and timestamp with DateTime, and timestamp_tz with DateTimeTz. Native temporal datatypes reject stored patterns such as %Y%m%d. Time has no supported D-FORMAT encoding.

Native declarations do not accept storage. For encoded fields, an explicit storage must agree with datatype (String → "string", Integer → "int"); other declared datatypes cannot use patterns. Omitted storage is inferred from the datatype, or defaults to "string" when datatype is absent. "int" accepts only uninterrupted component prefixes without literal text (including literal digits), such as %Y%m%d%H or %Y%m%d%H%M%S. The longest encoding has 14 digits: choose a sufficiently wide integer-valued column, such as BIGINT or DECIMAL(n,0), for a compact timestamp. A physical DECIMAL(n,0) column holding an integer date key may declare logical Integer; logical Decimal is not an integer encoding. storage cannot be declared without format. These checks use the field expression's result type, not the source column type. Basic mode ignores the Datus block with a warning, while retaining the core datatype time-role default.

granularity accepts day, week, month, quarter, or year. It may be omitted when the format implies it. time.granularity and time_granularity are equally supported spellings: neither is deprecated, but they must agree when both are present. A declared grain cannot be finer than the encoding. A time object may declare only granularity, but cannot be empty. Year/month/day prefixes infer year/month/day regardless of their literal text; prefixes including a clock component do not infer a grain. Hours, minutes, and seconds in storage do not enable sub-day grouping or date-time time_range bounds: those query features remain unsupported. Parsed output uses January/day one for omitted calendar components and zero for omitted clock components; for example, %Y-%m denotes the first day of that month, and %Y%m%d%H denotes the start of that hour.

Grouping and range boundaries¶

A %Y%m field can be grouped by month, quarter, or year, but not by day or week: its stored values contain no such detail. Likewise, %Y supports year grouping. Requesting a finer grain produces grain_too_fine.

Range boundaries must align with the stored periods. For %Y%m, end: "2026-03-15" produces grain_too_fine: use "2026-03-01" to exclude March or "2026-04-01" to include it. The error offers both aligned bounds; the engine does not choose whether to include a partial month. Year encodings require year boundaries. These checks apply to the query's bounds; scan expansion for window calculations is handled by the engine.

Selected time dimensions are parsed into date/time values, including when no grain is requested. Do not rely on results retaining the stored text layout. D-FORMAT does not provide an output display-format setting.

Validation and known limits¶

The declaration must match the data. Dosi checks supported formats and configuration combinations, and — where the field declares core Field.datatype — that the declared storage and native format agree with it. It does not inspect warehouse column types or values, so dosi validate cannot establish that %Y%m%d matches the actual field expression's output.

A wrong format or storage type can cause a warehouse error, NULL groups, or incorrect filtering without an error. For example, declaring %Y%m%d for values stored as '2026-09-07' also generates filter literals in the wrong layout. Check both the expression's type and representative stored values.

ClickHouse uses parseDateTime, whose DateTime result is limited to roughly 1970-01-01 through 2106-02-07. Earlier or later dates, including an SCD2 '99991231' end-date sentinel, are not supported reliably for grouping or parsed projections. Verified on ClickHouse 24.8.14: parseDateTime rejects 1960, 2107, and 9999 dates with CANNOT_PARSE_DATETIME. parseDateTime64 is unavailable in that version. The BestEffort alternatives are not a safe replacement: parseDateTimeBestEffort clamps out-of-range values, and parseDateTime64BestEffort changes 9999-12-31 to 2299-12-31. Bare encoded time_range comparisons do not use this parser, but that does not make an out-of-range grouping safe.

In basic mode, DATUS extensions are ignored with a warning, including requires. Use Datus mode for fields that depend on encoded-date handling. Epoch encodings such as epoch_seconds and epoch_millis are not supported.

How it affects SQL¶

Grouping parses an encoded field into a date/time value and truncates it when a coarser grain is requested. At the stored grain, or with no requested grain, the GROUP BY key can remain the stored column and the SELECT expression parses it.

For time_range, the engine instead converts the bounds into the stored encoding. The examples above generate predicates of these forms:

WHERE dt >= '20260101' AND dt < '20260401'
WHERE etl_month >= 202601 AND etl_month < 202604

Keeping parsing functions off the filter column helps warehouses use partition pruning. Actual pruning depends on the warehouse, field expression, and partition definition.

D-DATASET — metric home dataset¶

Where: a Metric's custom_extensions. Payload key: dataset, a string naming a dataset of the model. Default (absent): attribution is derived from the metric SQL alone.

A COUNT(*) names no column, so in a multi-dataset model with no other aggregate pinning a dataset, Ossie leaves its attribution undefined and the engine errors (count_star_needs_dataset). The hint resolves exactly that tie. It never overrides SQL-derived attribution — a metric whose aggregates name columns keeps their datasets regardless of the hint (extensions never redefine what the SQL says). An unknown dataset name is invalid_datus_extension.

metrics:
  - name: chat_message_count
    expression:
      dialects: [{ dialect: ANSI_SQL, expression: "COUNT(*)" }]
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.1", "dataset": "chat_record"}'

D-WINDOW — derived window metrics (period-over-period / rolling / cumulative)¶

Where: a Metric's custom_extensions. Payload key: window, a JSON object — full schema in window-extension.md. Default (absent): the metric is its plain base aggregate — no derivation.

Declares a window derivation over the metric's own expression (which must infer as a single plain aggregate): the three sugars pop / rolling / cumulative, or the general offset / frame form they desugar to. The time axis is not declared here — D-WINDOW composes with D-TIME (metric time_dimension > dataset primary; a missing axis fails at query time with no_primary_time_dimension). Offsets lower to a calendar-correct shifted-key self-join with automatic scan expansion + output trim; frames lower to agg(v) OVER (… ROWS …), with reset adding a DATE_TRUNC partition key and its own expansion + post-window trim. The remaining query restrictions (one shared time axis among the metrics that need one; a no-reset running metric or a partition-global one beside scan-expanding metrics under a time_range) are structured not_implemented errors; a reset finer than the queried grain is window_reset_too_fine. D-FILL never applies to derived outputs (NULL means "no comparable prior period"). Raw window SQL in metric expressions stays rejected (window_in_metric) — this extension is the only window path.

metrics:
  - name: revenue_mom_growth
    expression:
      dialects: [{ dialect: ANSI_SQL, expression: "SUM(orders.amount)" }]
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": 1, "window": {"type": "pop", "offset": "1 month"}}'

D-DERIVE — derived metrics (filter / compose / lod)¶

Where: a Metric's custom_extensions. Payload key: derive, a JSON object with a type discriminant — plus the window key's base field (M2, below). Default (absent): the metric is self-contained, no structure metadata. Supported forms: M1 ships filter + compose; M2 adds window.base and compose-with-window-members; 1.10 adds lod (D-LOD, documented with the rest of D-LOD). reagg and cross-dataset filter predicates are reserved and reject not_implemented with a hint naming their replacement or milestone.

Declares how a metric derives from other metrics of the same model — structure Ossie expressions cannot carry. Two degradation tiers: on the flatten tier (filter, compose over leaves/filter members) the metric's expression is a legal, semantically equivalent Ossie aggregate — basic mode computes it and gets the same numbers, and the engine enforces the equivalence at compile time (derive_expression_mismatch); on the stage tier (window.base, compose with window members) no flat expression can express the semantics, so the expression is only the nearest approximation and basic mode legitimately diverges (with the ignored-extension warning). Either way, treat the expression as tool-maintained output, not something to edit by hand.

filter — {"type":"filter","base":<metric>,"where":<predicate>}: a dimensional-filter variant of base. The predicate folds into the aggregate argument (CASE WHEN pred THEN arg END; COUNT bases count THEN 1), never a WHERE — non-matching time buckets keep existing (value NULL), which is what makes the variant safe under ROWS-framed windows. M1 limits the predicate to the base's own dataset (a cross-dataset predicate is not_implemented until M4) and base to a plain non-derived single aggregate. The engine records the subset relation (subset_of) in metric metadata.
compose — {"type":"compose","expr":<arithmetic>,"fill":<n>}: scalar arithmetic over metric names ("revenue - total_refunds"). Identifiers resolve strictly to metric names — a qualified or unknown name is an error with candidates, never a silent column read. Aggregate calls are rejected (composition never re-aggregates). fill replaces a member's NULL after branch alignment, before the arithmetic (the cross-fact "missing bucket poisons the sum" fix). The engine detects linear combinations (Σ cᵢ·metricᵢ), records members + coefficients + the leaf-measure decomposition, and fixes the conformed dimension set (dimensions every member branch reaches, under the unique-join-path rule) at compile time — dosi list exposes it, and a query grouping outside it gets unconformed_dimension naming the member that misses the dimension. Members sharing no dimension warn compose_no_conformed_dimensions at compile time. Attribution metadata is labeled exact only for linear composes over SUM/COUNT-tier measures (a filtered COUNT(DISTINCT) overlaps its complement — never exact).
lod — {"type":"lod","expr":<level of detail>} declares an include, exclude or fixed metric. Arithmetic over these metrics uses compose. The core expression carries the required nested aggregate representation; basic mode fails closed. See the unified LOD declarations and expression rules.

M2 additions:

window.base — the window key (D-WINDOW) accepts "base": <metric>: the window computes over the named metric's series instead of this metric's own expression. The base may be a leaf, a filter metric, a ratio (1.9), a compose — whether its measures sit in one dataset or several — or a compose over another window's output (1.9). A cross-fact base's series only becomes a value after the branch merge, so the stage sits on the merge instead of on one aggregate. Rejected: naming a window metric directly (the inner window would have no stage of its own to sit on — wrap it in a compose, which gives it one). Metadata records derive_family: "window", the derive_base chain and the inherited subset_of. Windows over a filter base inherit the CASE-WHEN bucket preservation: buckets with no matching rows still exist, so ROWS frames stay calendar-correct and cumulative sums skip the NULLs.

A ratio or compose base is what makes "rank the markets by close rate", "this rate versus last month's" and "am I above the average rate" expressible — none of which a single aggregate can state.

Nested windows (1.9) go one step further: a window reading a compose over another window's output is a second level, lowered as one more SQL stage. The scorecard shape needs it — rank the markets within a brand, then weight the scores within a market — because the two comparisons partition in different directions and one OVER has one PARTITION BY. This is not the re-aggregation I-1 bans: the row count is unchanged, each level recomputes with the query, and the weighted value lands on every row rather than folding them away. Levels are capped at 3, and past the first one partition must be declared explicitly — the query_dimensions default reads clearly on a single stage, but stacked, "which dims, minus whose exclusions" is no longer answerable from one declaration. Note what the window does and does not average: a whole-partition avg over a ratio base is the unweighted mean of the per-row rates, which is not the pooled ratio of sums. Both are legitimate and they differ; the window is the one that says "mean of the rates". - compose with window members — a compose whose expr references window metrics runs the arithmetic over the window output ("cumulative AOV" = revenue_cum / order_count_cum: numerator and denominator each accumulate, then divide). Members may mix window metrics with leaves; all members share one window stage, so they must share a time axis (cross-fact window members are rejected). Attribution is always approximate and there is no leaf-measure decomposition — window output cannot be served from materialized aggregates. - filter members in compose — a compose member may be a filter metric ("new_order_count / order_count"); it composes on the flatten tier exactly like a leaf.

Compose members that are themselves composes are expanded inline at compile time: the compiled artifact is flat, exactly as if the inner arithmetic had been written out. That is what lets a formula be stated once and reused — and, because the outer's expression is checked against the expansion of the inner's declaration, editing the inner and leaving a stale copy in a consumer is a derive_expression_mismatch rather than a number nobody questions.

Four consequences worth knowing before you nest:

The expression is the full expansion. Write out the whole flattened arithmetic, including the parentheses the nesting implies and the CAST(… AS DOUBLE) an inner bare-ratio member compiles to. It is tool-maintained output — regenerate it from the declaration rather than editing it by hand.
fill pushes down to the leaves. Each leaf reference takes the innermost fill declared on its path, so an outer fill: 0 still repairs a missing branch inside a member that declared none. The decision is per reference: one leaf reached through two members that fill it differently keeps both.
derive_members is the flattened leaf list on the flatten tier, with coefficients folded across the levels (quadrupled = doubled × 2 over doubled = revenue × 2 reports revenue at 4). Conformed dimensions intersect along the DAG, and unconformed_dimension therefore names the leaf that misses the dimension, not the intermediate compose. dosi lineage still draws the edge to the member you wrote.
On the stage tier a compose member stays a member. A window anywhere in the tree puts the metric on the stage tier, and there inlining would be wrong: each member is its own series out of the window stage, so dissolving an intermediate compose into leaves would lose arithmetic the stage never projects (cum_aov / revenue would lose the division and become two window members over a leaf). derive_members is therefore the DIRECT members there, the planner computes each nested compose as a hidden stage output, and substitution runs innermost-first at lowering. The tier decision reads the whole tree, so a window on an intermediate compose stages everything above it too.
Nesting is capped at 256 member references in one expansion — an inlined member is substituted at every occurrence, so referencing one twice doubles the expansion per level.

Cycles among metric references are metric_reference_cycle with the full path. Metadata surfaces on every host (dosi list, REST, MCP, Python): derive_family, derive_base, derive_expr, subset_of, derive_members, leaf_measures, attribution, conformed_dimensions, and required_dimensions — the dims a query has to group by, gathered over the whole chain (window-extension.md#frozen-degrees).

metrics:
  - name: net_revenue
    expression:
      dialects: [{ dialect: ANSI_SQL, expression: "COALESCE(SUM(orders.amount), 0) - COALESCE(SUM(refunds.ref_amount), 0)" }]
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.4", "derive": {"type": "compose", "expr": "revenue - total_refunds", "fill": 0}}'
  # Reuses net_revenue; `expression` is the expansion, so a change to
  # net_revenue's declaration breaks this metric loudly.
  - name: net_revenue_per_order
    expression:
      dialects: [{ dialect: ANSI_SQL, expression: "(COALESCE(SUM(orders.amount), 0) - COALESCE(SUM(refunds.ref_amount), 0)) / COUNT(orders.order_id)" }]
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.4", "derive": {"type": "compose", "expr": "net_revenue / order_count"}}'

D-MEASURE — explicit measure name¶

Where: a Metric's custom_extensions. Payload key: measure, {"name": <identifier>}. Default (absent): the sanitizer-generated stem (S-MEASURE-1).

Names the metric's single synthesized measure explicitly. Measure names are otherwise derived by sanitizing the aggregate argument ASCII-only, so two aggregates differing only in non-ASCII content (CASE literals such as the Greek 'νέο' / 'παλιό' — "new" / "old") collapse to one stem and fail measure_name_collision — the model cannot compile at all. The explicit name is a label: dedup still keys on the aggregate signature (an undeclared textual twin shares the declared name), the name enters the same collision checks (two explicit names on one signature, or one name on two signatures, still collide loudly), and it applies only to single-aggregate metrics. Because measure names surface in result columns and metadata, ignoring this key silently renames them — emit it under requires so older engines refuse instead (the first silent-impact key since requires shipped).

metrics:
  - name: new_product_revenue
    expression:
      dialects: [{ dialect: ANSI_SQL, expression: "SUM(CASE WHEN orders.category = 'νέο' THEN orders.amount END)" }]
    custom_extensions:
      - vendor_name: DATUS
        data: '{"v": "1.4", "requires": ["measure"], "measure": {"name": "orders_new_product_amount_sum"}}'

D-PARAM — query-time parameters¶

Where: a Metric's custom_extensions. Payload key: params, a list of {name, type, default, allowed | min/max, description}. Default (absent): no parameters — the metric is exactly what it declares.

A metric's window width, offset count, rank bucket count, navigation position, or a literal in its filter predicate becomes a typed query-time parameter: one moving_avg(n) instead of r7_revenue / r30_revenue / r90_revenue, one ltv(n) instead of eight ltvN columns. Parameters fill typed holes only — never SQL text — so a model validates once for every legal binding, and a binding can never inject anything.

metrics:
  - name: moving_avg
    expression:
      dialects: [{ dialect: ANSI_SQL, expression: "SUM(sales.amount)" }]
    custom_extensions:
      - vendor_name: DATUS
        data: |
          {"v": "1.5",
           "params": [{"name": "n", "type": "int", "default": 3, "min": 1, "max": 12,
                       "description": "trailing window width in buckets of the queried grain"}],
           "window": {"type": "rolling", "function": "avg", "periods": {"param": "n"}}}
  - name: paid_within_n
    expression:
      dialects: [{ dialect: ANSI_SQL, expression: "SUM(CASE WHEN base.datediff_pay < 7 THEN base.sum_pay_amount END)" }]
    custom_extensions:
      - vendor_name: DATUS
        data: |
          {"v": "1.5",
           "params": [{"name": "n", "type": "int", "default": 7, "allowed": [1, 3, 7, 14, 30, 60, 90, 180]}],
           "derive": {"type": "filter", "base": "pay_total", "where": "datediff_pay < :n"}}

Declaration. Every parameter has a name ([a-z][a-z0-9_]*, unique within the metric), a type (int | float | string), a mandatory default, and a domain — allowed (an explicit value list) or min/max (a closed range, either side optional); the two are mutually exclusive. The domain is the governance boundary: it says which definitions this metric is allowed to take. A string parameter takes no min/max. A parameter of any type may also declare no domain at all — a date spelled the way its column stores it, a code the model cannot enumerate — and compiles with the datus_param_unbounded warning. An open string still binds as a quoted literal, never as SQL text, so it widens what a query may ask, not what it may inject. default must lie in the domain.

Slots. A parameter is referenced from exactly these places:

Slot	Where	Syntax	Parameter type
`offset.count`	`window.offset.count` (object form; the `"1 month"` sugar takes no parameter)	`{"param": "n"}`	`int`, domain ≥ 1
`frame.preceding`	`window.frame.preceding`	`{"param": "n"}`	`int`, domain ≥ 0
`rolling.periods`	`window.periods` under `type: rolling`	`{"param": "n"}`	`int`, domain ≥ 1
`rank.buckets`	`window.rank.buckets` (`ntile`)	`{"param": "k"}`	`int`, domain ≥ 1
`value.n`	`window.value.n` (`nth_value`)	`{"param": "k"}`	`int`, domain ≥ 1
`filter.literal`	a literal inside a D-DERIVE `filter` metric's `where`	`:n` (bind syntax)	`int` / `float` / `string`

The window slots are integer counts, so they take int parameters whose domain has a declared lower bound at or above the slot's own minimum; a filter.literal hole takes any of the three types (region = :r, amount > :t). One parameter may fill several slots, and one predicate may hold several parameters (datediff_pay < :n AND platid = :plat). A parameter-only predicate (:n < 7) is rejected — a filter must still test a column. :name anywhere else — a metric expression, a query-time where_sql — is an error, never a silent literal. A parameter no slot references compiles with datus_param_unused.

The compiled model is instantiated at every parameter's default, so a model that declares parameters plans exactly like one that does not until a query binds something. For a filter slot the authored expression must be that default instantiation (< 7 above): the D-DERIVE equivalence gate holds the two together, and a bound value re-materializes the same measure a hand-written literal would produce — paid_within_n at n = 30 and a hand-written SUM(CASE WHEN datediff_pay < 30 …) share one column of the base CTE.

Binding. A query binds by name, for every queried metric that declares the parameter — directly or through its derive members (a compose inherits its members' parameters, so ltv = paid_within_n / new_user_cnt takes n):

{"metrics": ["new_user_cnt", "ltv"], "group_by": [{"field": "base.register_date"}],
 "params": {"n": 30}}                          // one binding
{"metrics": ["new_user_cnt", "ltv"], "group_by": [{"field": "base.register_date"}],
 "params": {"n": [1, 3, 7, 14, 30, 60, 90, 180]}}   // a list: one column per value

Omitted parameters take their defaults; a binding that no queried metric declares is unknown_metric_param (with the declared names as candidates); a value outside the type or domain — including 7.5 or "7" for an int, or a list where the parameter is used by attribution — is param_out_of_domain with a ready-to-send suggested_retry. The one implicit conversion: a float parameter accepts a JSON integer. Lists expand the metric into one column per value (several lists take the cartesian product, first declared parameter outermost), capped at 64 columns per query (param_expansion_too_large).

Output. A default binding leaves the column named after the metric; a non-default binding names it {metric}__{param}_{value} (moving_avg__n_6, ltv__n_30; float 1.5 → _1_5, strings sanitized: o'neil → o_neil), several parameters in declaration order, and under a list binding every column is suffixed. The rule exists so two dashboards can never show the same metric name over different windows — and every response carries the machine-readable form regardless: outputs, one entry per metric column with its metric and resolved params (defaults included), in the CLI JSON, the REST and MCP responses, the Python dicts, explain, and as Arrow field metadata (dosi.metric, dosi.params).

The sanitizer is ASCII-only, so it cannot always keep a string domain's values apart: an all-CJK allowed list collapses onto one suffix, and so does us-east next to us_east. Such a value takes an opaque token instead — h followed by 8 hex digits of a frozen FNV-1a hash of the value (by_channel__c_h0898d4b7), widened to 16 on the astronomically unlikely collision. The token is a function of the value and its declared domain and nothing else, so it does not move when the query binds a different set, when allowed is reordered, or when a value is added to it. A value keeps its stem whenever that stem is unique in the domain and is not itself of the reserved h<hex> shape, so a readable ASCII domain is named exactly as before. The model is told at compile time which values went opaque, and with what token, by the datus_param_opaque_token warning; outputs[].params carries the value, and dosi query --format text prints a legend under the table.

A string parameter with no allowed has no other values to compare against, so its token has to be injective on the value alone: a value that already is its own stem (v1, us_east) names the column, and any other (us-east, 自营, o'neil) takes the 16-digit opaque token. There is no compile-time warning for these — the domain is not known until the query binds it.

An order key names a metric (by its model name) or a group-by item (by its output name, or by the spelling the query used — sales.sale_date:month as well as sale_date__month), never a column the engine assigned. A metric key resolves while the metric is one column, suffix and all; a metric expanded by a list binding is unknown_order_key, because there is no spelling that picks one of its columns. To rank by one binding, bind that one value — the retry on the error is the params fragment that does it.

Attribution binds one value per parameter (a list is refused with a single-value retry) and echoes the resolved bindings in comparison_metadata.params.

Catalog. dosi list metrics, GET /v1/metrics and MCP list_metrics carry each metric's params (declaration plus the slots it fills); describe_metric adds param_schema, a JSON Schema for the params map (allowed → enum, min/max → minimum/maximum) — an agent reads it, then binds.

Compatibility. Under --osi-basic the D-WINDOW / D-DERIVE hosts are ignored, so params is inert with them (one ignored_vendor_extension warning) and any query-time params is unknown_metric_param. An engine older than 1.5 rejects {"param": …} in a window slot structurally (the field is an integer there) and the "v": "1.5" envelope at the version gate — which is the guard that matters for :name in a filter, since a 1.4 engine would otherwise render the bind marker into SQL. Note that older servers tolerate unknown query fields: a 1.4 server silently drops a client's params and answers with the defaults. A current one refuses any key a query does not define with unknown_query_key. Check GET /v1/capabilities (or dosi info) for params before relying on it.

A level-of-detail field's base¶

A field's parameters belong to its filter base. Queries reading the field bind them; field bases take single values, and independent windows use separate names. The core expression must agree with defaults. See Independent windows and parameters for the complete rules.

D-DIM — which fields are offered as dimensions¶

Where: a Field's custom_extensions. Payload key: is_dimension, a boolean. Default (absent): the engine infers the field's role (see below).

Ossie types every non-time field as a regular dimension, so a model cannot say that a column is a measurement (unit_cost), an identifier (c_customer_id), free text, or PII. Grouping by such a column is not a structural error — the SQL runs, the numbers are right, and the analysis is meaningless. is_dimension is how the model says so once, at modelling time, instead of every caller re-deciding it at every call.

It is a listing signal, not a permission:

Written	Effect
absent	the engine infers (below)
`false`	the field is not recommended: `list_dimensions` marks it `is_dimension: false`, and attribution does not offer it as a candidate. Naming it in `group_by` still works.
`true`	the field is recommended even where inference would exclude it — the way to group a numeric column into buckets

Because it never reaches the planner, ignoring the key changes no number; it only costs a caller a worse shortlist (documented impact).

datasets:
  - name: store_sales
    fields:
      - name: ss_quantity            # a measurement no metric aggregates
        expression: { dialects: [{ dialect: ANSI_SQL, expression: ss_quantity }] }
        custom_extensions:
          - vendor_name: DATUS
            data: '{"v": "1.7", "is_dimension": false}'
      - name: i_current_price        # numeric, but analyzed in price bands
        expression: { dialects: [{ dialect: ANSI_SQL, expression: i_current_price }] }
        custom_extensions:
          - vendor_name: DATUS
            data: '{"v": "1.7", "is_dimension": true}'

Inference when the key is absent. The engine uses only model structure — it never probes the warehouse (a COUNT(DISTINCT …) on a high-cardinality column is not a cost the engine may impose). A field is not offered when it is a primary-key member, a single-column unique key of the metric's own home dataset, a column named in a many-to-one relationship's join pair (a D-CONFORM pairing is not a join; its columns are the dimension itself), or a column some metric measures — the value read by a SUM/AVG/STDDEV*/VAR*. Everything else is offered, and list_dimensions reports which rule applied, as source.

Three aggregates that mention a column without measuring it, and therefore leave it a dimension:

COUNT — COUNT(DISTINCT region) ("how many active regions") is an ordinary metric over an ordinary dimension. This holds for the metric being listed too: the answer for a field does not depend on which metric you ask about.
MIN / MAX — they order strings and dates perfectly well; MIN(activity_code) picks one identifier, it does not measure it.
A condition inside an aggregate — SUM(CASE WHEN is_instant THEN 1 ELSE 0 END) counts rows where a flag holds; the flag stays a dimension. Only the THEN/ELSE values are measured. A D-DERIVE filter folds into exactly this shape, so it behaves the same way.

Inference cannot see a business key that is neither a declared key nor a relationship column, a numeric column no metric reads, free text, or PII — those are exactly the fields worth declaring.

D-LOD — fixed level-of-detail fields¶

LOD concepts are explained in Understand LOD, and the complete rules are in the LOD authoring reference. A field carries lod: {expr: ...}; a metric carries derive: {type: "lod", expr: ...} in the DATUS JSON string. The extension envelope and version rules on this page still apply.

Full rules: LOD modeling.

Where a filter applies: Tableau's order of operations¶

Context and time ranges scope fixed calculations; ordinary display filters do not. Mixed raw-field/LOD OR, NOT and comparisons are supported. Selected metric conditions filter completed results.

Full rules: LOD modeling.

Broadcast: reading the value next to the row¶

Fixed fields can be read beside detail rows. Reaggregating a fixed value counts each key once per display group; raw-row metrics keep row weight. Incompatible weights require separate queries.

Full rules: LOD modeling.

The rollup keeps the storage encoding¶

A fixed time key retains its original storage grain inside the calculation; display time grains apply outside it. A LOD value cannot be a routable time axis.

Full rules: LOD modeling.

The expression is not a sentinel¶

The core expression must match the declaration. A fixed field carries the base aggregate; an include metric carries its outer aggregate; exclude and fixed metrics require a SUM wrapper around the base aggregate.

Full rules: LOD modeling.

View-relative: `include` and `exclude` on a metric¶

Include adds dimensions and requires an outer aggregate. Exclude removes dimensions and recomputes the base at that grain. The restriction against ordinary metrics alongside applies to include; ordinary metrics may coexist with exclude and fixed metrics.

Full rules: LOD modeling.

`fixed` on a metric: a value the view does not group by¶

A fixed metric attaches a context-scoped baseline. Its declared keys must be ungrained group-by fields; no keys means an overall baseline. Ordinary display filters do not narrow it.

Full rules: LOD modeling.

Cross-fact alignment: one axis, two facts¶

Each fact computes its own fixed field and metrics before D-CONFORM aligns the grouped results. This is not a join between fact rows, nor a distinct union of the classified entities.

Full rules: LOD modeling.

Across datasets: a value from another dataset's rows¶

Since 1.11, a host field can aggregate another dataset through explicit key correspondence. The host defines the population. Missing COUNT is 0, other aggregates are NULL, and cross-dataset NULL keys do not match.

Full rules: LOD modeling.

What D-LOD does not do yet¶

See the supported-combination matrix and validation rules in the unified guide. Selected LOD metrics support result filtering; they cannot be grouping dimensions. Rejections have specific authoring or query error codes, not one universal not_implemented code.

Full rules: LOD modeling.

4. Precedence & defaults summary¶

Point	Ossie core	Datus extension	Engine default when absent
relationship join type	undefined (LEFT assumed)	`D-JOIN join_type`	`left`
fact-to-fact conformed pairing	inexpressible (every relationship is a join)	`D-CONFORM cardinality`	`many_to_one` — the edge is a join
missing-group metric value	undefined	`D-FILL fill_nulls_with`	count→`0` (S-FAN-6), else `NULL`
primary time dimension	undefined (`is_time` only)	`D-TIME time_dimension` (metric > dataset)	the dataset's single `is_time` field, else none
native time grain	no explicit grain; `Date` implies day	`D-GRAIN time_granularity`	inferred from type/encoding, otherwise unknown
time column storage format	logical `datatype`; no encoding layout	`D-FORMAT time`	assumed native `DATE`/`TIMESTAMP`
`COUNT(*)` home dataset	undefined in multi-dataset models	`D-DATASET dataset`	SQL-derived, else `count_star_needs_dataset`
window derivation	inexpressible (window SQL rejected)	`D-WINDOW window`	plain base aggregate
query-time parameters	inexpressible	`D-PARAM params`	none — every slot is the literal it declares
which fields are offered as dimensions	every non-time field is a dimension	`D-DIM is_dimension`	inferred from keys, relationships and aggregated columns
aggregate-as-dimension	inexpressible (a field is row-level)	`D-LOD lod`	none — a field is a stored column

5. Validation & engine modes¶

Upstream Ossie validation (dosi validate --osi-basic, see the CLI reference): unaffected — custom_extensions is valid Ossie, so a model carrying Datus extensions still passes it.
Datus self-validation (engine in --osi-datus mode, the default, when it reads a DATUS entry): the payload must be a JSON object; known keys must match their declared type/enum; otherwise invalid_datus_extension. Unknown keys pass (forward-compat) unless named in requires. The envelope version gate (§2) runs here, once per payload — including on the SemanticModel carrier, which defines no keys of its own but is still held to the envelope.
Basic mode (--osi-basic): the engine treats DATUS like any unknown vendor — the extension is inert (spec-compliant passthrough) and the compile collects one ignored_vendor_extension warning per carrying element — SemanticModel, Dataset, Field, Relationship, and Metric alike — naming the default that applies instead (D-JOIN → LEFT, D-FILL → no fill beyond S-FAN-6, D-TIME → single-is_time inference only, D-GRAIN → any grain, D-DATASET → SQL-derived attribution only, D-WINDOW → the plain base aggregate, D-FORMAT → the column is assumed to be a native date, which for a string or integer column means a warehouse type error or silently wrong rows, D-LOD → the field is dropped, D-CONFORM → the edge is an ordinary many→one join, so the two facts are joined row to row and duplicate-sensitive aggregates across it fan out). D-LOD is the one that cannot degrade to a value: without the declaration the field's expression is a bare aggregate, which is not a legal row-level attribute, so the field is dropped with an aggregate_field_dropped warning and any metric or query referencing it fails closed with unknown_field / unknown_column. The alternative would be AVG(COUNT(DISTINCT …)) reaching the warehouse. The payload is not parsed, so a malformed payload is also just ignored-with-warning — and no version diagnostic can occur in basic mode at all. The same model file is valid in both modes; only the semantics differ. (The single-is_time primary-time inference is not an extension — it applies in basic mode too, so metric_time still works wherever pure Ossie determines a unique time axis.)

6. Non-goals / boundaries¶

Datus extensions never redefine an Ossie object's identity or a metric's SQL — only engine behavior at Ossie-undefined seams.
They are not a place to smuggle vendor SQL. Metric expressions stay Ossie expression.dialects; the engine still infers metric semantics from that SQL.
Semantics that belong in Ossie core (granularity, time spine, window/cumulative metrics) may eventually be standardized upstream; if/when adopted there, the corresponding Datus extension (D-GRAIN, D-WINDOW) is retired in favor of the core field.

7. Change log¶

Each released minor and the keys it added. The machine-readable form of this table is capabilities().history — dosi info --format json or GET /v1/capabilities.

Version	Date	Keys added	Impact if ignored
1.0	2026-07-11	`join_type` (D-JOIN), `fill_nulls_with` (D-FILL)	`documented`, `documented`
1.1	2026-08-02	`time_dimension` (D-TIME), `time_granularity` (D-GRAIN), `dataset` (D-DATASET)	`degraded`, `documented`, `degraded`
1.3	2026-08-09	(none — in-family growth of `window`)	(inherits `window`'s `documented`)
1.4	2026-08-12	`derive` (D-DERIVE M1), `measure` (D-MEASURE)	`degraded`, `silent`
1.5	2026-08-26	`params` (D-PARAM M1)	`degraded`
1.6	2026-09-02	`time` (D-FORMAT)	`silent`
1.7	2026-09-11	`is_dimension` (D-DIM)	`documented`
1.8	2026-09-16	`cardinality` (D-CONFORM)	`silent`
1.9	2026-09-16	(none — in-family growth of `window`)	(inherits `window`'s `documented`)
1.10	2026-09-23	`lod` (D-LOD; also the `lod` family of `derive`)	`degraded`
1.11	2026-10-07	(none — in-family growth of `lod`)	(inherits `lod`'s `degraded`)

1.0 (2026-07-11): initial — D-JOIN (relationship join_type, implemented: the join is emitted as INNER or LEFT accordingly), D-FILL (metric fill_nulls_with, implemented 2026-07-12: the metric output is COALESCEd; overrides the S-FAN-6 count default, any metric kind).
1.1 (2026-08-02): time & attribution — D-TIME (dataset/metric time_dimension, implemented: a dataset's primary time dimension and a metric's own, the reserved metric_time query name, per-branch time-range fallback), D-GRAIN (field time_granularity, implemented: finer-grain requests → grain_too_fine), D-DATASET (metric dataset, implemented: COUNT(*) home attribution). Basic-mode ignored_vendor_extension warnings now cover all five carrier objects (previously only Relationship and Metric).
1.1 (2026-08-04, no version bump): the envelope version gate itself — v is now read and enforced (§2), requires is introduced, each key carries a declared ignore-impact, and the whole registry is published through dosi info, GET /v1/capabilities, and dosi_engine.DATUS_EXT. No key was added or changed, so the version stays at 1.1. Payloads without a v — which is what the Datus agent emits today — are unaffected.
1.2 (2026-08-06): window metrics — D-WINDOW (metric window, implemented: the WindowStage step in --explain; pop/rolling/cumulative sugars + general offset/frame form, cumulative reset with scan expansion + post-window trim; ignore-impact documented — the metric falls back to its plain base aggregate; spec in window-extension.md). Initially DuckDB-executed; the all-dialect enablement (per-dialect offset-shift lowerings, verified on live warehouses) landed as a follow-up. In-family additions (same minor, fail-closed on engines predating them): mixed families / mixed shifts / mixed resets-with-start relaxed, and require_full_window on finite frames (2026-08-06) — see window-extension.md#restrictions.
1.3 (2026-08-09): window families & functions — grows the window key in-family, no new registry row (capabilities().history still ends at 1.2; the version signals the payload dialect). Additions: the rank family (row_number / rank / dense_rank / ntile / percent_rank / cume_dist over the metric value), the value family (first_value / last_value / nth_value), offset direction: "forward" (LEAD — next-period reference), frame order / partition / units / second modifiers (value-ordered and RANGE frames, partition modes, two-argument inputs), and registry levels W2 (stddev_pop/samp, var_pop/samp) + W3 (covar_pop/samp, corr). Every addition fails closed on a 1.2 engine: unknown top-level family keys fall into the exactly-one-family error, unknown in-family keys are rejected, unknown function names fail enum parse — and a declared "v": "1.3" trips the version gate besides. Spec: window-extension.md (family sections + function registry).
1.4 (2026-08-12): derived metrics & measure naming — D-DERIVE M1 (derive key, implemented: filter + compose families, metric references resolved with cycle detection, the expression equivalence gate, compile-time conformed dimensions, and derive metadata on every host surface; reagg / cross-dataset filter predicates / window.base are reserved and reject not_implemented naming their milestone — note window.base previously fell into the silently-ignored unknown-key bucket and now fails closed) and D-MEASURE (measure key, implemented: explicit measure naming, signature-keyed dedup and collision checks unchanged — the fix for the non-ASCII stem collision, and the first silent-impact key: emit it under requires). M2 (2026-08-19, in-family — no new keys, no version bump): window.base flips from reserved to functional (windows over leaf / filter metrics), compose accepts window members (stage-tier lowering: arithmetic over window output) and filter members (flatten tier). A 1.4 payload using these shapes compiles on an M2 engine and fails closed (not_implemented / rejection) on the M1 engine — never a silent misread. Nested compose (in-family — no new keys, no version bump): a compose member may itself be a compose, expanded inline at compile time. An older engine rejects the shape with the same invalid_datus_extension it always did, so this too fails closed. The metric stays on the flatten tier, and derive_members becomes the transitive leaf list with folded coefficients; a window anywhere in the tree keeps the combination not_implemented.
1.6 (2026-09-02): time storage formats — D-FORMAT (field time block, implemented: the column is parsed per dialect on the grouping side, and filter bounds are folded into the stored format before the SQL is built, with the grain inferred from the layout). time.granularity joins the flat time_granularity as an equal spelling — no deprecation, but the two may not disagree. The second silent-impact key: emit it under requires, so a datus-mode engine that lacks it fails at the version gate (basic mode never reads requires; see §2). A consumer that drops it does not fail loudly; it assumes a native date column and, for a month- or year-stored one, returns an empty result with no diagnostic. See D-FORMAT for configuration and limitations. Scope: string and integer layouts plus the native temporal types; epoch_seconds / epoch_millis are deferred because from_unixtime reads the session timezone and that needs its own round.

In-family addition (2026-09-08, still 1.6): time.format accepts continuous ordered fixed-width calendar prefixes with arbitrary fixed literal text. Filtering still folds date bounds to stored constants; grouping extracts components into a canonical date/time before dialect parsing. Existing layouts remain compatible. Older Datus engines reject new layouts rather than silently interpret them. - 1.7 (2026-09-11): field roles — D-DIM (field is_dimension, implemented: read by list_dimensions and by attribution's candidate list). The model can finally say that a column is a measurement, an identifier, free text or PII, so it stops being offered as a grouping dimension. It is a recommendation signal only: an explicitly named group_by is honored either way and no compiled SQL changes, hence documented impact and no requires entry. Where the key is absent the engine infers the role from model structure alone — keys, relationship columns and numerically aggregated columns — and never probes the warehouse. See D-DIM. - 1.8 (2026-09-16): conformed fact-to-fact relationships — D-CONFORM (cardinality key on a Relationship, implemented: conformed classes in the join graph, per-branch resolution of group-by / filter / time-range columns and of metric_time to each fact's own spelling, merge on the output name). A many_to_many edge is never joined; only its paired columns are shared. The third silent-impact key: emit it under requires — without it the edge is a many→one join and duplicate-sensitive aggregates fan out. See D-CONFORM. - 1.9 (2026-09-16): a ranked scorecard — grows the window key in-family the way 1.3 did, so there is no new registry row. Four additions driven by one shape.

frame.scope: "partition" states the frame as the entire partition instead of an offset from the current row, lowering to <func>(…) OVER (PARTITION BY …) with no ORDER BY and no frame clause. It is the second window function beside a rank — "how many rows am I ranked against" — which a ranked scorecard needs and which no combination of the existing keys could express: count with preceding: "unbounded" is a running count, and units: "range" still stops at the current row's peer group. scope is mutually exclusive with preceding, reset, require_full_window, units and order, each of which only describes an ordered frame; writing one is rejected rather than ignored.

window.base widens: a ratio or a compose can now be the series a window reads, so ranking a rate or taking its period-over-period is expressible. A cross-fact compose's series only becomes a value after the branch merge, so the window stage sits on the merge — the branches aggregate separately and their CTEs ride beside the stage's base. Such a base has no shared time axis (each fact carries its own), so the families that need one refuse on conformance.

Fails closed on a 1.8 engine twice over: it rejects the new keys inside frame and order as unknown, and the payload's "v": "1.9" trips the version gate. See window-extension.md.

Nested windows and NULL placement complete the shape. order.nulls: "first" | "last" pins where NULLs sort in a window ordering — left unset, each warehouse applies its own default and they disagree (DuckDB and PostgreSQL last ascending, MySQL-family and StarRocks first), which a ratio base turns from theoretical into routine. The generator emits the clause only where it differs from the target's default, and where a dialect has no such clause at all (MySQL, TiDB) the placement lowers to a leading CASE WHEN … IS NULL sort key — so the rendering varies by dialect and the meaning does not. And window.base goes one step further than a ratio: it accepts a compose over another window's output, so a second window can read the first one's. Each level lowers to one more SQL stage; depth is capped at 3 and level 2+ must declare partition. The guards that matter run across levels, not within one: the output trim stays outside the outermost stage, and a partition-global metric at any level refuses to share a query with a scan-expanding one at any other. Also in-family: a stage-tier compose may have a compose member (it stays a member rather than being inlined, since on that tier each member is its own series). - 1.10 (2026-09-23): levels of detail — D-LOD, one grammar on two carriers: the lod key on a Field and the lod family of the existing derive key on a Metric (implemented).

On a field, fixed(base, dims…) makes a per-entity aggregate groupable, filterable and re-aggregatable through an ordinary query (the LodRollup step in --explain; a D-FORMAT rollup key keeps its storage encoding and is parsed once, outside). When the view needs more than the rollup — a dimension outside its keys, a row-level field reading the value, several key families — the broadcast shape (LodBroadcast) joins the detail rows with one derived table per key family; an aggregate over a pinned value counts each of its keys once per group. No keys is the table-scoped FIXED, broadcast onto every row.

On a metric, include groups the inner stage by the query's own group-by plus the declared dimensions and collapses it with the declared outer aggregate; the grain and any D-FORMAT parse apply inside. exclude attaches a coarser aggregation of the same fact to the view's rows — LEFT JOIN on the dimensions they share, CROSS JOIN on none — recomputed rather than windowed so a COUNT(DISTINCT) base is right too. fixed is the field's declaration hosted on a metric so it can attach to the view: a table-scoped or key-scoped total computed before the dimension filter, which is what Tableau's {SUM([Sales])} is as a denominator and what exclude() cannot mean (above). A conformed pairing whose endpoints are two facts' own fixed fields aligns them on one axis: each fact rolls up for itself and the branches merge, the facts are never joined.

Filter placement follows Tableau's order of operations. The where clause is a dimension filter — applied to a fixed rollup's output when it names only the rollup's keys, to the detail rows otherwise, never inside the rollup — while time_range is the context filter that scopes every rollup. A row-level field that reads only D-LOD values aggregates once per key.

Basic mode drops a D-LOD field (aggregate_field_dropped) and fails closed on a D-LOD metric, whose expression is the nested aggregate its declaration describes (nested_aggregate), rather than computing one level of it. See D-LOD. - 1.11 (2026-10-07): levels of detail across datasets — grows the lod key in-family, no new registry row. A field's fixed base may aggregate another dataset than the field's own when every key corresponds to a column of it through a declared relationship: a many→one edge between the two in either direction, a many→one path from the base to the host, or a D-CONFORM pairing (a rollup over a scan of its own). The rollup is LEFT JOINed onto the host's rows on plain equality and a count reads 0 where nothing matched, so > 0, = 0 and >= n are the semi-join, the anti-join and a threshold; the context layer reaches the rollup through the base's own edges or the pairing, and a conjunct it cannot read there is left out (D-LOD across). A 1.10 engine fails closed: it refuses the base as aggregating another dataset.

Not yet implemented (roadmap)¶

The queued RFCs are all additive, so each lands as one minor bump with its own registry rows. Order may change; every entry occupies a minor of its own.

Minor	RFC	Keys	Carrier	Impact	`requires`?
≥1.12	`rfc-semi-additive-metrics` §4	`semi_additive`	Metric	`silent` — an end-of-period balance gets summed across snapshots	yes
≥1.12	`rfc-minimal-declarations` §3 D-DISTINCT-STATE	`distinct_state`	Field / Dataset	`silent` — a materialized bitmap column is aggregated as if it were an ordinary column	yes
≥1.7	`rfc-derived-metrics` M3-M4	`derive` reagg + cross-dataset filter (`window.base` and compose window/filter members shipped in-family at 1.4/M2)	Metric	`degraded`/error — reserved shapes reject `not_implemented` today, never silently compute	no
—	`window-extension.md` rank/share/streak families, W2+ functions	future `window` shapes (in-family additions, fail-closed on older engines)	Metric	`degraded`	no

Three silent keys have shipped: measure (D-MEASURE) at 1.4, time (D-FORMAT) at 1.6 and cardinality (D-CONFORM) at 1.8. If you author models against a Datus agent that emits silent-impact keys, keep an eye on the requires list it produces: that is what stops an older engine from quietly returning wrong numbers, and it only works because requires itself shipped at 1.1 — ahead of the first key that needs it.

Datus extensions to Apache Ossie (vendor spec)¶

1. Why this is safe against Ossie¶

2. The envelope¶

v — the declared version¶

requires — keys that must not be dropped¶

2.1 Versioning policy¶

3. Extension catalog¶

D-JOIN — relationship join type¶

D-CONFORM — conformed relationships between facts¶

D-FILL — metric null-fill for missing merge groups¶

D-TIME — primary (aggregation) time dimension¶

D-GRAIN — native time granularity¶

D-FORMAT — time storage format¶

When to configure it¶

Quick start¶

Writing filters¶

Configuration reference¶

Grouping and range boundaries¶

Validation and known limits¶

How it affects SQL¶

D-DATASET — metric home dataset¶

D-WINDOW — derived window metrics (period-over-period / rolling / cumulative)¶

D-DERIVE — derived metrics (filter / compose / lod)¶

D-MEASURE — explicit measure name¶

D-PARAM — query-time parameters¶

A level-of-detail field's base¶

D-DIM — which fields are offered as dimensions¶

D-LOD — fixed level-of-detail fields¶

Where a filter applies: Tableau's order of operations¶

Broadcast: reading the value next to the row¶

The rollup keeps the storage encoding¶

The expression is not a sentinel¶

View-relative: include and exclude on a metric¶

fixed on a metric: a value the view does not group by¶

Cross-fact alignment: one axis, two facts¶

Across datasets: a value from another dataset's rows¶

What D-LOD does not do yet¶

4. Precedence & defaults summary¶

5. Validation & engine modes¶

6. Non-goals / boundaries¶

7. Change log¶

Not yet implemented (roadmap)¶

`v` — the declared version¶

`requires` — keys that must not be dropped¶

View-relative: `include` and `exclude` on a metric¶

`fixed` on a metric: a value the view does not group by¶