Skip to content

Release notes

0.2.0

New features

  • Native execution on Athena, Redshift and SAP HANA. Metric queries, detail queries and attribution run on Amazon Athena (engine v3), on Amazon Redshift through the Redshift Data API (provisioned clusters and Serverless workgroups), and on SAP HANA through SAP's HDBODBC client, which you install separately. See Athena, Redshift and SAP HANA.
  • Derived subtypes in an ontology (experimental). An entity type that extends another with a derived_by rule, such as a Premier Customer defined by a customer's margin and order count, can be queried (select --from PremierCustomer), grouped by (--group-by PremierCustomer) and filtered on (Customer is PremierCustomer). Its thresholds can be parameters, declared under concept_params and bound with --param. See Derived subtypes.
  • The MCP server follows the model while it changes. Under --mcp-stdio the server reloads the model, the ontology and the connections file when they change, so an agent that writes or edits the model needs no restart. A missing or invalid model is a tool error (no_model, model_invalid) instead of a server that does not start, and --env-file supplies passwords set after the session started. See The model can change during the session.

Enhancements

  • Every MCP tool is marked read-only (readOnlyHint), so hosts that run only read-only tools unattended, such as Codex's exec mode, can call them. Filters and parameter values are still evaluated by the warehouse, so give the connection a read-only database user. See stdio.
  • With --model <dir>, --ontology <file> loads only the mappings onto that ontology, so one directory can hold mappings onto several. See Three ways to write it.
  • A metric's CASE WHEN condition may read a column of a dataset its rows reach many-to-one, such as stores.is_kpi in a count of repairs. See Column → dataset attribution.
  • derive_expression_mismatch carries expected_expression, the expansion a derived metric's expression must hold, ready to be written in place. See Derived metrics.
  • Generated measure names keep letters of any script, so two aggregates whose filters differ only in a non-Latin literal no longer fail with measure_name_collision. Names that still coincide get a short tag. See Measures.
  • The docs and bundled examples write a model with its fields at the top level, the shape upstream Apache Ossie validates. The wrapped semantic_model: shape still loads without a warning, but fails upstream validation. See One model per document.

Bug fixes

  • Two explicit measure names (D-MEASURE) that swapped the generated names of two aggregates made a metric read one of them twice, silently: SUM(price) + SUM(qty) computed SUM(qty) + SUM(qty).
  • A filter metric (D-DERIVE) over a base whose CASE WHEN condition reads another dataset got an expression that validation then refused with cross_dataset_aggregate.
  • A field that named another calculated field reading a fixed value, such as premier written as customer_margin >= 3000 AND customer_orders >= 6, failed at the warehouse with a missing column. The named field's expression is now inlined. See Fixed fields.
  • On Hive, nulls: first and nulls: last in order_by were ignored, and date expressions in null-safe joins were rendered wrongly. NTH_VALUE windows, which Hive does not support, are refused with a structured error.

Behavior changes

  • dosi no longer looks for a Datus agent.yml. Without a connections file of its own, it used to read ./conf/agent.yml or ~/.datus/conf/agent.yml, and failed on a real Datus file. It now looks only at --connections, DOSI_CONNECTIONS, ./dosi-connections.yaml and ~/.config/dosi/connections.yaml; write the datasources you need into dosi-connections.yaml. Datus itself hands its datasources to dosi and is not affected. See Where the file is discovered.
  • DuckDB files are opened read-only. Other processes, such as the duckdb CLI or a notebook, can read a file while dosi has it open. dosi never writes to the file, and a path that does not exist is an error instead of a new empty database. See DuckDB.
  • Without --execute, dosi query and dosi select compile for the connection marked default: true instead of DuckDB, so the SQL shown is the SQL that runs; stderr names the dialect. MCP requests without a connection use that connection too. See dosi query.
  • The bundled examples name their tables without a schema (FROM orders, not FROM main.orders), so the session's default schema decides where the tables are.

Known issues

  • A division with arithmetic on either side, such as (a + b) / c, uses the warehouse's own /: PostgreSQL, Trino and Redshift truncate integer division, and MySQL and TiDB keep four decimal places. AVG also follows the warehouse: Trino keeps a DECIMAL column's scale, and MySQL, TiDB and Doris return four decimal places. Write CAST(... AS DOUBLE) where you need the full value.
  • On Oracle, a column whose name is a reserved word, such as LEVEL, is not quoted, so its dataset cannot be queried.
  • On Doris 4.1.3, an Arrow Flight query whose result is already in the SQL cache fails with IllegalStateException. Run SET GLOBAL enable_sql_cache = false. See Arrow-native results.
  • On Redshift, a window frame by value (RANGE) is refused by the warehouse. See Execution and diagnostics.
  • Upstream Apache Ossie now accepts metrics scoped to a dataset; Dosi does not read them yet, so a model that uses them passes upstream validation and is refused by Dosi.

0.1.14

New features

  • Connections from dbt or Cube. dosi migrate conf writes dosi-connections.yaml from the connections your dbt (or MetricFlow) profiles.yml or Cube .env already holds, and dosi connection test <name> checks one. See From dbt or Cube.
  • One metric or dataset in full. dosi show metric|dataset <name> shows how a metric is computed, what it can be grouped by, the parameters it takes and what a query must group by. Python Engine.metric() / Engine.dataset(), MCP describe_metric / describe_dataset and REST GET /v1/metrics/{name} / /v1/datasets/{name} return the same. See dosi show.
  • Levels of detail across fact tables and over composed metrics. An include, exclude or fixed metric can be queried together with metrics of other fact tables, and can be built on a compose, ratio or expression metric. See LOD reference — base metrics.
  • Warnings when a filter does not apply as written. Compile and execute results carry warnings, each with a hint on how to change the query, for example a context_filter condition that one part of the query cannot read. Text output prints them on stderr. See Errors.
  • Ontologies from mapping documents, and in Python (experimental). --mapping loads an ontology from a mapping document, and the Python Engine browses an ontology (describe(), paths()) and queries by concept paths. See Ontology.

Enhancements

  • A model document can be written in the flat shape upstream Apache Ossie has moved to, with the model's fields at the top level. The wrapped semantic_model: shape still loads and is still the one to prefer. See One model per document.
  • A window over two fact tables with paired dates, such as the period-over-period of revenue - refunds, no longer needs a metric-level time_dimension. See Restrictions.

Bug fixes

  • A condition on a fixed field of a dataset the metric only joins, such as sessions where users.lifetime_revenue >= 150, was silently left out of the SQL. It is now refused, naming the two ways that work. See LOD reference — limits.
  • A count filtered by a level-of-detail value, such as logins of players who paid, counted each player once instead of each login.
  • A filtered count of a nullable column also counted the rows where the column is NULL. A model that writes THEN 1 for a column other than the primary key now fails validation, and the hint gives the expression to use.
  • A composed metric with no time_dimension of its own grouped every member by one dataset's date instead of each member's own.
  • A composed division with fill: 0 truncated as integer division on PostgreSQL, Hologres, GaussDB, Oracle and Trino, and a 0 denominator read infinity or failed.
  • On ClickHouse a division over DECIMAL columns kept the dividend's scale, so 410 / 600 read 0.68.
  • Window metrics:
    • A time range starting or ending inside a bucket of the query's grain dropped that bucket. It is now refused with grain_too_fine, offering both boundaries.
    • Grouping by the time axis at two grains, such as quarter and month, gave wrong values. The finer grain now orders the window.
    • An offset that does not land on a bucket, such as month-over-month grouped by quarter, returned NULL on every row. It is now refused with window_offset_misaligned.
    • A nested window, such as the change of a month-over-month change, under a start date returned wrong values on its first rows.
    • A quarter offset failed on PostgreSQL, Hologres, GaussDB and DWS.
  • A fixed field keyed by a date stored as text (D-FORMAT, '%Y%m%d') failed when grouped by another dimension.
  • A field whose expression is a comparison, such as is_first_order (order_date = first_order_date), produced invalid SQL when a filter compared it to TRUE. Fields inside INSTR, LOCATE and STR_TO_DATE were passed to the warehouse unresolved.
  • A calculated field that reads only fixed fields, such as an ontology classification like Premier Customer, could not be used as a filter.
  • A PostgreSQL connection whose sslrootcert holds several certificates, such as AWS RDS's global-bundle.pem, failed to connect. See TLS.
  • Arrow Flight reads from StarRocks and Doris failed when the warehouse sent more than 4 MiB in one record batch.

Behavior changes

  • A level-of-detail metric is refused when the query does not group by the dimensions it names. A keyed exclude used to return its base unchanged; it is now refused with lod_dimension_not_grouped, as is a fixed metric whose key is missing. A window metric without its time axis is refused with time_axis_not_grouped instead of unknown_dimension. The catalog lists what to group by in required_dimensions, and the error in missing_dimensions. See LOD reference.
  • A fixed field over another dataset follows the query's period. time_range and context_filter now apply to it on that dataset's own date; before, it read all history. For a value that must not follow the period, such as a first order date, put the dates in where_sql instead. A dataset with several date fields and none declared as its time is refused (time_axis_ambiguous). See Filters.
  • A where_sql condition that only some metrics in the query can read is refused with where_needs_context, which says to move it to context_filter.
  • NULLs sort last in order_by on every warehouse, so a limit keeps the same rows everywhere. Add nulls: first to an order key to put them first. See Ordering.
  • An output column name longer than 63 bytes, which a metric with two date parameters can reach, is shortened to a hash, such as revenue_cur__h5e1698108f4d9c01. PostgreSQL, GaussDB and DWS used to truncate it silently. outputs[].params still lists each value. See D-PARAM.
  • --mapping takes the path of a mapping document instead of the name of an embedded mapping. In Python, mapping= becomes mapping_path= / mapping_text=, and ontology_info() reports a mappings list.

Known issues

  • A division with arithmetic on either side, such as (a + b) / c, uses the warehouse's own /: PostgreSQL, Trino and Redshift truncate integer division, and MySQL and TiDB keep four decimal places. AVG also follows the warehouse: Trino keeps a DECIMAL column's scale, and MySQL, TiDB and Doris return four decimal places. Write CAST(... AS DOUBLE) where you need the full value.
  • On Oracle, a column whose name is a reserved word, such as LEVEL, is not quoted, so its dataset cannot be queried.
  • On Doris 4.1.3, an Arrow Flight query whose result is already in the SQL cache fails with IllegalStateException. Run SET GLOBAL enable_sql_cache = false. See Arrow-native results.

0.1.13

New features

  • Levels of detail. An aggregate can now be computed at a grain other than the query's. A field pins a fixed grain, such as COUNT(DISTINCT event_date) per user, and becomes an axis, so "users active exactly N days" is one query. A metric pins include, exclude or fixed relative to the query's grouping: an average per user reported by channel, or a share against a fixed total. Ordinary display filters do not narrow that total; context and time ranges still do. See LOD modeling.
  • Two facts aligned on a level of detail. When each fact has its own fixed field — how many days a user had a session, how many days they ordered — a conformed pairing (D-CONFORM) between the two makes them one axis, so a metric from either fact can be grouped by it.
  • A level of detail over another dataset. A fixed field's base can now aggregate a related dataset, such as each flight's open alerts, when the keys meet through a declared many-to-one relationship or a conformed pairing. A row with nothing on the other side counts 0, so > 0 and = 0 in a where answer "has one" and "has none". See D-LOD across datasets.
  • A level of detail with a window of its own. When a fixed field's base is a parameterized filter metric, the query's params bind it, so "January actives with no February login" is one query: the model declares the window as parameters and the query supplies the days. See A level-of-detail field's base.
  • Detail queries read levels of detail. dosi select can project, filter and order by a fixed field of its --from dataset, and --param binds that field's window. It also gains --distinct and --context-filter, which scopes the rollup the way a time range does. See Level-of-detail fields.

Enhancements

  • The DATUS extension version is now 1.11. Models that declare an older version load as before.
  • A string parameter may leave its domain open instead of listing allowed, for a date spelled the way its column stores it or a code the model cannot enumerate. It compiles with a datus_param_unbounded warning and still binds as a quoted literal. See Parameterized metrics.
  • When every value a fixed rollup computes comes from a filter metric, such as a field with a window of its own, the filter is now also the rollup's WHERE, so a warehouse can prune partitions instead of scanning the whole table. The values are unchanged.
  • list_dimensions for a metric now reports the metric's time column once, as metric_time, instead of listing the column beside it. Other time columns are still listed, and the column remains a legal group_by.

Bug fixes

  • dosi validate --osi-basic reported a model as invalid when the upstream Ossie validator could not run at all, for instance because its Python libraries were missing. Anything that stops the validator from running is a notice, not a verdict, and the validator now provisions its own dependencies through uv when it is installed.
  • ROUND in a field or metric expression rounded halves to even on PostgreSQL, MySQL, TiDB and ClickHouse, so ROUND(0.5) was 0 there and 1 everywhere else, and a binned dimension put its .5 rows in a different bucket. It now rounds halves away from zero on every warehouse.
  • A DATE_TRUNC('MONTH', ...) written into a model expression failed on MySQL and TiDB, which have no such function. It is now rewritten for them the way a query's own time grain is.
  • On ClickHouse, a time dimension stored as text failed the whole query when any of its values was NULL. NULL values now land in the NULL group, as on other warehouses.
  • list_dimensions for a metric hid the columns of a D-CONFORM pairing as foreign keys and listed a conformed dimension once per spelling. A paired column is now judged like any other field and listed once, under the metric's spelling.
  • A count metric used as a member of a compose with a window member read 0, not NULL, for a group its fact had no rows for, so a weighted score such as (… ) / store_count became 0 / 0 — NaN on DuckDB, an error on PostgreSQL. Only a count that is itself selected fills 0 (S-FAN-6); a count inside an expression now keeps NULL under a window stage too.

Behavior changes

  • A metric or detail query that carries a key its type does not define is now refused with unknown_query_key, naming the keys it does define. Before, the key was silently ignored, so a misspelled context_filter or a where meant as where_sql returned unfiltered numbers.

Known issues

  • A division with arithmetic on either side, such as (a + b) / c or a / NULLIF(b, 0), still runs with the warehouse's own /: PostgreSQL, Trino and Redshift truncate integer division, and MySQL and TiDB keep four decimal places. Only a plain SUM(a) / SUM(b) is converted to floating point; elsewhere write CAST(... AS DOUBLE) on the numerator.
  • 0.1.13 still reads only the older document format, the semantic_model: list. See the 0.1.11 note below.

0.1.12

New features

  • Ranked scorecards. A window metric can now rank a ratio or a composed metric, and it can count its whole partition with frame.scope: "partition" (the N beside a rank). Windows can stack up to three levels, such as ranking markets within a brand and then weighting scores within a market. order.nulls pins where NULLs sort, which otherwise differs between warehouses. See D-WINDOW.
  • Filtering on metrics. where_sql can now name a selected metric (revenue > 1000). It filters the result rows, like SQL HAVING. The new context_filter (CLI --context-filter, REST, MCP and Python) always filters the input rows. Use it to rank within a subset. See Filters.
  • How a metric is built, and what a query must group by. The metric catalog in the CLI, REST, MCP describe_metric and Python now has two new fields. derive_expr gives a composed metric's formula. required_dimensions lists the fields a query has to group by for the metric's windows to mean what they define.

Enhancements

  • A composed metric whose members use different time columns on the same table used to be refused with MetricTimeConflict. Now each time column gets its own branch, so every metric returns what it returns when queried alone. See D-TIME.
  • Oracle: a missing or incompatible Instant Client now fails at once with DPI-1047 and a link to the setup guide, instead of a 30-second pool timeout. The Oracle connector guide has been rewritten around installation and troubleshooting.
  • The DATUS extension version is now 1.9. Models that declare an older version load as before.

Bug fixes

  • A window's partition could change depending on the other metrics in the query. Adding a cumulative metric, for example, made another metric's daily rank span every day.
  • A dimension-table measure placed on two fact tables in one window query read the wrong fact's numbers on DuckDB and was rejected by StarRocks.
  • The dosi lineage view legend now draws each edge kind the same way the graph does.

Behavior changes

  • In a window query, a where_sql condition on group-by fields now filters the result instead of the rows being ranked. The rows you keep also keep the ranks and partition counts of the whole population. To get the 0.1.11 numbers, move the condition to context_filter.
  • A window whose partition.exclude names a field the query does not group by is now refused with window_exclude_not_grouped. Before, every row ranked first in a partition of one. Add the field to group_by; required_dimensions lists the fields a query needs.
  • A dimension count beside a fact measure, such as COUNT(DISTINCT stores.store_id) in repairs per store, now counts every store, not only the stores that have a repair. The ratio's value changes. To count only the entities seen in the fact, count the fact's key: COUNT(DISTINCT repairs.store_id). See Fan-out protection.

Known issues

  • A division with arithmetic on either side, such as (a + b) / c or a / NULLIF(b, 0), runs with the warehouse's own /. PostgreSQL, Trino and Redshift truncate integer division, and MySQL and TiDB keep four decimal places. Only a plain SUM(a) / SUM(b) is converted to floating point for you. Until that changes, write CAST(... AS DOUBLE) on the numerator.
  • 0.1.12 still reads only the older document format, the semantic_model: list. See the 0.1.11 note below.

0.1.11

New features

  • Recommended dimensions for each metric. dosi list dimensions --metric lists only the fields worth grouping that metric by, recommended ones first. The REST API, MCP tool, Python Engine.dimensions(metric=) and the lineage graph return the same list. Dosi leaves out keys, join columns and the columns a metric aggregates. A field can override that with is_dimension (D-DIM): false to hide an id or free-text column, true to include a numeric column. The list is only a recommendation: --group-by still accepts any reachable field, and the generated SQL is unchanged. dosi validate suggests where a declaration is missing or contradicts the model; --strict turns those suggestions into errors.
  • Ratios across two fact tables. Two fact tables that share a date or other column without one belonging to the other, such as logins and payments, can now be paired with cardinality: many_to_many (D-CONFORM). A metric like payers per active player then groups and filters by the shared column, and Dosi never joins the two tables directly.
  • dosi lineage. dosi lineage view opens the metric lineage graph in your browser, with derived metrics laid out left to right in the order they are built. dosi lineage dump prints the graph as JSON. See Lineage.
  • Ontologies (experimental). --ontology loads an Apache Ossie ontology, with or without a core model. You can query by its concepts, browse them with dosi describe and dosi paths, and see them on the lineage graph. Flags and output may still change.

Enhancements

  • Attribution now requires dimensions. Without them it stops before querying the warehouse and suggests the metric's recommended dimensions, ready to paste. See Which dimensions?.
  • The DATUS extension version is now 1.8. Models that declare an older version load as before.

Bug fixes

  • Arrow Flight reads from StarRocks failed every time. On StarRocks 4.0.14, also set arrow_flight_proxy_enabled = false. See Arrow-native results.

Known issue

  • Upstream Ossie changed its document format after this release: a document now holds one model, with name and datasets at the top level instead of a semantic_model: list. 0.1.11 cannot load the new format. A later release reads both. With --osi-basic, use the upstream validator at the commit named in dosi validate; upstream main rejects every model 0.1.11 accepts.

0.1.10

New features

  • Detail queries. dosi select returns rows instead of aggregated metrics — "the 100 largest orders from Enterprise customers". Name the fields you want; Dosi works out the joins, and refuses rather than hand you duplicated rows. See Detail queries.
  • Time columns stored as text or numbers. A time column no longer has to be a real DATE. Declare what it holds — 20240131, 2024-01, 2024年01月31日 — and Dosi groups and filters it correctly, without losing partition pruning. Queries are unchanged: you always write 2024-01-31. See D-FORMAT.
  • A composed metric can build on another composed metric. A formula is written once and reused, instead of restated in every metric that uses it — and a restatement that had drifted used to compile silently.

Enhancements

  • Models using the standard OSI datatype property now load, Apache Ossie's TPC-DS reference model included.
  • An --order key names a metric or a field you grouped by, never a column name the engine generated.

Bug fixes

  • A parameter whose values were Chinese, or any non-ASCII text, produced the same column name for every value; binding several at once failed the query.
  • Time columns stored as integers failed on MySQL, TiDB and ClickHouse when rolled up to month or year.

0.1.9

Dosi is now under the Elastic License 2.0: free to use and redistribute, with no hosted-service resale. It replaces the proprietary licence 0.1.5 introduced, which granted no rights at all without a separate agreement.

New features

  • BigQuery runs queries. It was compile-only — Dosi generated the SQL and refused to execute it. It now passes the full test corpus against a live project. See BigQuery.
  • Metric lineage (Python only) — Engine.lineage() returns the graph behind your metrics: table → dataset → metric → derived metric. Pass redact_sql=True if the graph leaves your trust boundary.
  • Model authoring contract (Python only) — render_datus_authoring_spec() returns the rules this engine enforces, with a digest, so a tool that generates models can check it is authoring against the same version.
  • Installing from a mirror. install.sh takes DOSI_SOURCE=github|oss|cloudflare. Use DOSI_SOURCE=oss from mainland China, where the GitHub download runs at tens of KB/s and often stalls. Archives are checksum-verified either way. The Cloudflare mirror is not populated in this release.

Enhancements

  • install.sh works on macOS, where it used to refuse any non-Linux host.
  • Validation returns what each metric actually compiled to, and Python connection settings can be passed in memory without the plaintext-secret warning — which only ever made sense for a file on disk.
  • A new BigQuery docs page, plus a 200k-row example database at https://dosi.datus.ai/orders-200k.duckdb.

Bug fixes

  • A rolling window of 1 period, or a preceding bound of 0, generated SQL that Doris and StarRocks reject outright, so an ordinary --param n=1,3,6 failed on those two engines.

0.1.8

No wheel was published for this version, so dosi-engine users go from 0.1.7 straight to 0.1.9, which includes everything here. The binaries shipped for Linux x86_64, Linux aarch64 and Apple Silicon only — on an Intel Mac, install 0.1.9.

New features

  • Attribution: why did this metric change? Give Dosi a metric, the dimensions to look at and two time windows; it runs the queries itself and returns a ranked breakdown of where the change came from, each segment with a ready-made filter for the next drill-down. The method comes from the metric's own algebra, and a metric it cannot decompose returns a structured "unsupported" with the reason rather than a wrong number. See Explain metric changes.
  • Parameterized metrics. A metric's window width, offset, rank bucket or filter value can be a query-time parameter: one moving_avg(n) replaces r7 / r30 / r90, and one query can ask for several values at once, a column each. See Parameterized metrics.

Enhancements

  • dosi list metrics says whether a metric can be attributed, and how, so you can check before running an analysis.
  • A query can carry several named time windows and get them all back from a single statement.

Behaviour change

  • If you call Engine.attribute() from the 0.1.7 wheel: the share fields can now be absent for a segment whose denominator is not positive, so code that reads them unconditionally needs to handle that.

0.1.7

This release is split: the binaries and the wheel were built three days apart, from different commits, so the wheel contains more.

New features

  • A second, smaller download — dosi-<version>-<target>-lean.tar.gz is the executables alone. It compiles queries and reaches every warehouse; local DuckDB execution uses a duckdb CLI on your PATH.
  • Derived metrics (D-DERIVE) M2 — windows over derived metrics, and composing over window members.
  • SQLite files are readable through DuckDB.
  • An error-code reference, with a catalog of what each rejection means.
  • Attribution, first version (wheel only) — Engine.attribute(). 0.1.8 extends it; read its behaviour change above before upgrading.

Enhancements

  • DuckDB starts about 200 ms faster per process — a one-shot dosi query --execute went from 249 ms to 41 ms of CPU. Query speed itself is unchanged; the cost was start-up.

Bug fixes

  • Redshift now renders as Redshift. It had been falling through to the generic generator and emitting Postgres SQL.
  • The wheel builds under cargo 1.93 and later.

0.1.6

Wheel only — dosi and dosi-server users go from 0.1.5 to 0.1.7.

New features

  • A Linux aarch64 wheel, so Graviton and arm64 containers no longer need a source build.
  • SQLite files are readable through DuckDB, which makes a local .sqlite file a usable connection for tests and small models.
  • CONNECTION_TYPES is exported, so a caller can list which warehouses a given wheel was built with.

0.1.5

The first public release of Dosi. From here the project ships as binaries and wheels only; no source archive is published. Install with curl -fsSL https://dosi.datus.ai/install.sh | sh for the dosi and dosi-server executables, or pip install dosi-engine for the Python bindings.

It loads an OSI semantic model, validates it, compiles a metric query to SQL for your warehouse, and runs it. Fifteen warehouses — DuckDB · StarRocks · ClickHouse · Doris · TiDB · Trino · Postgres · MySQL · Snowflake · BigQuery · Databricks · Redshift · Hologres · GaussDB · Oracle. Four ways in: the dosi CLI, REST, MCP, and the Python wheel. Two modes: --osi-datus honours the Datus extensions, --osi-basic is strict standard OSI.

Nothing before 0.1.5 was published.