Skip to content

Dosi vs MetricFlow vs Cube

A three-way comparison from the 2026-09-03 run: dosi / dosi-server 0.1.8, MetricFlow 0.212.0 (dbt-metricflow 0.14.0) and Cube Core 1.7.33 with the Tesseract planner. It covers 24 queries in six tiers (baseline, filters and ordering, joins, time windows, derived metrics, parameters) at three data sizes (1,820, 1M and 10M bookings).

Methodology

All three engines get the same semantic model, written natively in each (five datasets from MetricFlow's simple_model: bookings, views, revenue, listings, users), the same seeded DuckDB data and the same queries. Each engine runs through the binary it ships: dosi and dosi-server, mf, and cubejs-server. Before anything is timed, each query's result is checked to be equal across all three engines, and every one of the 24 × 3 results matched at every data size.

shape dosi MetricFlow Cube
cold CLI dosi query, one process per query mf query --explain / mf query, one process per query (MetricFlow ships no server) —
cold server dosi-server spawn → first execute answered — cubejs-server spawn → first /v1/load answered
warm compile POST /v1/query/compile — POST /cubejs-api/v1/sql
SQL floor the SQL each engine generated, run as-is in DuckDB same same
concurrency k closed-loop keep-alive clients, distinct queries k parallel mf query processes k closed-loop clients
model size dosi validate, dosi query, server boot → first compile dbt parse, mf query --explain boot → first /v1/sql

Environment

  • AWS EC2, 4 vCPU Intel Xeon Platinum 8259CL @ 2.50GHz, 15 GiB RAM, Linux 6.1 (al2023), shared with other resident services
  • dosi 0.1.8 release binaries with in-process DuckDB
  • MetricFlow 0.212.0, dbt-core 1.12.3, dbt-duckdb 1.11.0, CPython 3.12
  • Cube Core 1.7.33, Node 22, dev mode, CUBEJS_TESSERACT_SQL_PLANNER=true, no pre-aggregations

Results

dosi MetricFlow (mf CLI) Cube
warm compile, unseen query 0.3–0.5 ms — 21–32 ms
one query, cold process (compile) 18–20 ms 2.9–3.1 s —
server cold start → first query 47–59 ms — 2.3 s
engine overhead per request, 1M rows (upper bound) 4–10 ms — 36–90 ms
SQL floor, window/offset queries, 1M rows 7–12 ms 20–62 ms 209–343 ms
SQL floor, five windows in one request, 1M rows 12 ms 202 ms 1,511 ms
throughput at 16 clients, rolling window, 1M rows 57 q/s 1.0 q/s (16 processes) 2.9 q/s
cold start with 200 datasets / 3,000 metrics 0.44 s 6.5 s (dbt parse + mf) 9.3 s
10M rows: trailing 60-day window, warm execute 129 ms 5.0 s (cold CLI) 3.4 s
10M rows: five windows in one request, warm execute 128 ms 5.0 s (cold CLI) 28.4 s
peak RSS, server after 1M-row warm loops 284 MB 172–380 MB per CLI process 1,021 MB

Across all 24 queries, dosi's warm compile is 37–78x faster than Cube's. With 1,820 rows and 1 to 64 concurrent clients, dosi serves 12–46x Cube's queries per second without errors.

What the numbers say

  • Engine overhead is about a millisecond. When the engines ask the warehouse for the same work (plain aggregates, joins, filters), the SQL each one generates runs within ±10% of the others, and what is left is engine overhead: about 1 ms for dosi, 25–35 ms for Cube, and about 3 s of process start for MetricFlow.
  • The generated SQL is the gap. For every time-window, offset and parameterized query, the engines ask the warehouse for different work. At 1M rows dosi's SQL runs 5–130x faster than theirs, and the gap grows with the data: at 10M rows a request with five windows takes 128 ms on dosi and 28.4 s on Cube.
  • Concurrency scales with cores. dosi-server keeps adding throughput as clients are added. Cube queues requests, and MetricFlow needs one process per query.
  • Model size barely matters. dosi loads 3,000 metrics in 0.44 s; the other two take 6.5–9.3 s.

Caveats

  • Single shared 4-vCPU machine. Treat the ratios, not the absolute milliseconds, as the finding.
  • The engine-overhead row is an upper bound: the timed request carries a cache-busting predicate that the SQL-floor run does not.
  • dosi's four rolling and running-total cases scan the whole table, while the other engines scan 60–90 days. That handicaps dosi in those rows.
  • Cube's pre-aggregations were not used; Cube ran in dev mode. A production-mode spot check showed dev mode is not what makes it slower here.
  • MetricFlow is measured cold only, because that is how it ships; dbt Cloud's serving layer is closed source.

Benchmarks: Dosi vs MetricFlow

The earlier two-way comparison of the dosi CLI against the legacy MetricFlow engine. The numbers below are from the 2026-07-11 run; the three-way run above supersedes it.

Methodology

Both engines process the same semantic model (MetricFlow's own simple_model test model, converted 1:1 to Apache Ossie) and the same queries against the same seeded DuckDB database. The queries are the ones CI already checks for identical result sets between the two engines, so the benchmark measures the speed of equivalent work.

measurement dosi-engine MetricFlow
cold CLI one dosi query process: spawn → load model → compile → print SQL one python subprocess: interpreter start + imports + model parse + MetricFlowClient.explain() — the same work mf query --explain does, minus click/config overhead (a bound favorable to MetricFlow)
warm API same full-process invocation (osi has no daemon mode; the bare process-spawn floor is reported separately) in-process explain() loop after a warmup call — interpreter, imports, and model parse all already paid
execute dosi query --execute --db <file> (cold, full process) in-process query() loop (warm) and cold subprocess
peak RSS max resident set size of the cold subprocess (wait4 rusage) same

Cold runs are interleaved A/B, 5 timed runs after 1 discarded warmup; warm loops are 30 iterations. Medians reported.

Environment

  • 2026-07-11, dosi-engine f850dfd, cargo build --release
  • AWS EC2, 4 vCPU Intel Xeon Platinum 8175M @ 2.50GHz, 15 GiB RAM, Linux (al2023)
  • MetricFlow: Datus fork, editable install, CPython 3.12.12 (uv venv)
  • DuckDB CLI v1.4.4, seeded with the simple_model test data

Results

osi process-spawn floor (dosi --version): 2.0 ms median.

SQL compilation — cold CLI (process spawn → SQL printed)

query dosi MetricFlow speedup
simple_agg_by_dim (bookings ⟨is_instant⟩) 10.6 ms 2,321.9 ms 220x
three_metrics_no_dim 10.1 ms 2,403.5 ms 237x
ratio_metric (booking_fees_per_booker) 10.4 ms 2,309.2 ms 222x
expr_metric (views_times_booking_value) 10.4 ms 2,338.8 ms 226x
simple_agg_by_time (bookings ⟨ds⟩) 10.5 ms 2,289.9 ms 219x

MetricFlow cold-start breakdown: ~1,004 ms imports + ~742 ms model parse + ~161 ms client init before any query compiles — ~1.9 s of fixed overhead per CLI invocation. dosi pays ~2 ms of process spawn and parses the model in the remaining ~8 ms.

SQL compilation — warm API (everything resident)

query dosi (full process) MetricFlow (in-process) speedup
simple_agg_by_dim 10.6 ms 107.6 ms 10.2x
three_metrics_no_dim 10.1 ms 218.9 ms 21.6x
ratio_metric 10.4 ms 105.3 ms 10.1x
expr_metric 10.4 ms 205.0 ms 19.8x
simple_agg_by_time 10.5 ms 111.4 ms 10.6x

This is the most generous comparison for MetricFlow — its number is pure in-process compile time with all imports and model parsing already paid, while dosi still re-reads and re-compiles everything in a fresh process. dosi remains 10–22x faster.

End-to-end execution on DuckDB (compile + run + print rows)

query dosi cold MF cold speedup MF warm (in-process) vs dosi cold
simple_agg_by_dim 34.2 ms 2,321.1 ms 68x 127.0 ms 3.7x
three_metrics_no_dim 36.2 ms 2,428.1 ms 67x 233.1 ms 6.4x
ratio_metric 35.7 ms 2,368.8 ms 66x 122.7 ms 3.4x
expr_metric 33.5 ms 2,441.2 ms 73x 235.9 ms 7.0x
simple_agg_by_time 32.9 ms 2,309.1 ms 70x 132.8 ms 4.0x

With execution, DuckDB query time (~20 ms via the CLI shell-out, identical work for both engines) becomes a shared floor; osi's end-to-end cold run is still 3–7x faster than MetricFlow's warm in-process path.

Peak memory (cold process, max RSS)

mode dosi MetricFlow
compile ~15.5 MB ~157 MB
execute ~26.5 MB ~162 MB

Caveats

  • simple_model is small (~25 datasets/metrics). MetricFlow's ~742 ms model parse scales with model size; osi's full 10 ms budget includes its parse.
  • The MetricFlow "cold" path skips the real mf CLI's click startup and Datus config resolution, so real mf query invocations are somewhat slower than measured here.
  • Group-by spelling differs for the time-dimension case (ds for MF, bookings_source.ds for osi) — same semantic query, engine-native syntax.
  • Numbers are single-machine (shared EC2 box); treat relative ratios, not absolute times, as the finding.

Arrow IPC vs JSON: result serialization

How much the columnar egress path (POST /v1/query/execute with Accept: application/vnd.apache.arrow.stream, see rest-api.md) saves over the default JSON body for the same query.

The benchmark generates a high-cardinality synthetic DuckDB dataset (5M-row fact table, 200k distinct user_id) and a matching Ossie model, launches a release dosi-server against them, and runs each result set through both Accept formats.

Methodology

One running server, one query, two Accept headers — the only thing that changes is the response encoding. For each format we measure three things:

measurement what it captures
wire MB bytes of the HTTP response body (median)
e2e ms wall time from request send to full body received — query exec + server serialize + transfer (median)
parse ms client-side cost to turn the raw body into a usable structure: json.loads vs pyarrow.ipc.open_stream(...).read_all() (median)

Query execution over the 5M-row table is identical work on both paths, so it forms a shared floor in e2e ms (isolated by the SMALL row, where the payload is tiny and serialization is negligible). wire MB and parse ms have no shared floor — they are pure format cost. Three result sizes show the advantage scaling with the result, not the scan. 2 warmup + 7 timed runs per cell, medians reported.

Environment

  • 2026-07-14, dosi-engine 2e23173 (working tree), cargo build --release
  • AWS EC2, 4 vCPU Intel Xeon Platinum 8259CL @ 2.50GHz, 15 GiB RAM, Linux (al2023)
  • DuckDB CLI v1.4.4; client CPython 3.9 with pyarrow 21.0.0

Results

result set format rows wire MB e2e ms parse ms
SMALL — 365 rows × 3 cols JSON 365 0.03 1483.5 0.3
Arrow 365 0.01 1463.1 0.2
LARGE — 200k rows × 3 cols JSON 200,000 10.63 2051.8 198.4
Arrow 200,000 6.50 1684.0 1.4
WIDE — 200k rows × 4 cols JSON 200,000 13.99 2749.3 242.7
Arrow 200,000 8.90 1707.9 2.0

Reading the large result sets (Arrow vs JSON):

LARGE (200k × 3) WIDE (200k × 4)
wire — bytes on the network 1.6x smaller 1.6x smaller
parse — client decode ~140x faster (1.4 vs 198 ms) ~120x faster (2.0 vs 243 ms)
e2e — including the shared exec floor 1.2x faster 1.6x faster
e2e minus the ~1.46 s exec floor (serialize + transfer + parse only) ~2.7x faster (221 vs 589 ms) ~5.2x faster (245 vs 1286 ms)

Interpretation

  • Parse is the headline. JSON makes the client re-parse every value out of a string; Arrow hands over typed column buffers with essentially zero decode (1–2 ms for 200k rows vs 200–240 ms). This gap does not shrink in a release build — it is the client's, and it is structural.
  • Serialization is the server-side win. Once the exec floor is removed, Arrow spends ~2.7–5.2x less time on the format itself: no per-row materialization and no per-value JSON string encoding — record batches stream straight from the executor through the IPC writer into the body. The wider the row, the bigger the gap (WIDE > LARGE).
  • Benefit scales with result size. At 365 rows there is no measurable difference — the payload is a rounding error against query execution. The columnar path is worth opting into precisely when the result is large.
  • Wire size is a steady ~1.6x. Real deployments can layer HTTP compression on top; the parse and serialize wins are independent of it.

Caveats

  • Single-machine, loopback transport (no real network RTT or bandwidth limit), so wire MB understates the transfer-time advantage a remote client would see. Treat relative ratios as the finding.
  • The synthetic dataset is numeric/low-width-string; JSON's disadvantage grows with wider rows and more string columns, shrinks for a few wide numeric columns.
  • e2e ms carries the DuckDB scan of all 5M rows on every request (no result cache); the isolated "minus exec floor" row is the cleaner read of format cost, but the floor is only approximately constant across group-by cardinalities.