Warehouse connectors¶

Dosi executes by SQL pushdown: it compiles a metric query to dialect SQL and the warehouse runs it. A connector handles connection setup, statement submission, and normalizing results into a ResultSet — columns plus rows of Null | Bool | Int | Float | Str, dates as ISO-8601 strings — so results compare equal across engines.

This section is the setup guide. Start here for the connection-profile format that every connector shares, then follow the routing table below to your database's own page. Arrow-native result transfer (DuckDB, ClickHouse, StarRocks/Doris, Databricks) is described in Arrow.

Connection profiles¶

Connection profiles use the Datus agent.yml datasources: vocabulary — dosi-exec keeps its own minimal QueryExecutor interface (deliberately not aligned with datus-db-adapters), but the configuration is shared, so a Datus setup needs no translation.

--connections <path> (env DOSI_CONNECTIONS) accepts either a full agent.yml (profiles under services.datasources) or a standalone file with the same map under a top-level datasources:.

datasources:
  local-duck:
    type: duckdb
    uri: duckdb:///warehouse.duckdb   # omit for in-memory
  prod-sr:
    type: starrocks
    host: sr.internal
    port: 9030
    username: dosi
    password: ${SR_PASSWORD}          # ${VAR} / ${VAR:-fallback} from the environment
    database: analytics
    default: true                     # used by --execute without --connection

Run it:

$ dosi query --metrics revenue --group-by orders.status --execute --connection prod-sr

--execute without --connection uses the profile marked default: true if there is one, else local DuckDB (--db <file>, or in-memory).

Where the file is discovered¶

Without --connections, the first path that exists wins:

#	Path
1	`$DOSI_CONNECTIONS`
2	`./dosi-connections.yaml`
3	`./osi-connections.yaml` (pre-rename fallback)
4	`~/.config/dosi/connections.yaml`
5	`~/.config/osi/connections.yaml` (pre-rename fallback)
6	`./conf/agent.yml`
7	`~/.datus/conf/agent.yml`

An existing Datus install therefore works with zero configuration. Note that each legacy osi- path sits directly after its dosi- counterpart, not at the end of the list.

Keys every connector shares¶

Key	Type	Required	Default	Notes
`type`	string	yes	—	Selects the connector. One of the values in the routing table below; matched exactly, lower-case, no aliases (`postgresql`, `pg` and `opengauss` are all errors).
`default`	bool	no	`false`	At most one profile in the file may set it. Used by `--execute` without `--connection`.

Every other key is per-connector — its page lists the ones that connector reads, and the ones it parses and ignores.

Secrets and `${VAR}`¶

Any string value may reference the environment as ${VAR} or ${VAR:-fallback}. For password: and private_key_file_pwd: the exact form matters:

A value written exactly as ${VAR} resolves lazily — the file loads with the variable unset, and the error (carrying an export hint) fires only when that connection is used.
A value that merely contains ${VAR} resolves when the file loads.
A plaintext secret works, and warns on stderr: warning: connection "x" has a plaintext password in the connections file; prefer password: ${VAR}.

What a broken entry does¶

A profile that fails to parse — an unset ${VAR}, an unknown type: — never takes the file down. Its error is stored and replayed only when that name is used, so one bad entry cannot stop the profiles you are actually running.

Unknown keys are ignored: Datus adapters keep private settings in the same file. The pre-1.0 spellings are the exception — they are rejected with a migration hint rather than silently doing nothing:

Rejected	Use instead
`dialect:`	`type:`
`user:`	`username:`
`password_env:`	`password: ${VAR}`
`private_key_passphrase:`	`private_key_file_pwd:`
`private_key_passphrase_env:`	`private_key_file_pwd: ${VAR}`
`path:`	`uri: duckdb:///<path>`
`url:`	`uri:`
a top-level `connections:` list	a `datasources:` map keyed by name

Connectors¶

`type:`	Page	Cargo feature	How it talks to the engine	Arrow-native
`duckdb`	DuckDB	`exec-duckdb` (default)	in-process, shipped library	yes
`sqlite`	SQLite	`exec-duckdb`	local file, read-only, through DuckDB	yes
`mysql`	MySQL	`exec-mysql`	MySQL wire protocol	no
`tidb`	TiDB	`exec-mysql`	MySQL wire protocol	no
`starrocks`	StarRocks	`exec-mysql`, `exec-flightsql`	MySQL wire, or Arrow Flight SQL	opt-in
`doris`	Apache Doris	`exec-mysql`, `exec-flightsql`	MySQL wire, or Arrow Flight SQL	opt-in
`postgres`	PostgreSQL	`exec-postgres`	Postgres wire protocol	no
`hologres`	Hologres	`exec-hologres`	Postgres wire protocol	no
`gaussdb`	GaussDB / openGauss	`exec-gaussdb`	Postgres wire + SHA256 auth	no
`oracle`	Oracle Database	`exec-oracle`	OCI through ODPI-C	no
`clickhouse`	ClickHouse	`exec-http`, `exec-http-arrow`	HTTP interface	opt-in
`trino`	Trino	`exec-http`	REST `/v1/statement`	no
`snowflake`	Snowflake	`exec-snowflake`	SQL API v2 over HTTPS	no
`bigquery`	BigQuery	`exec-bigquery`	`jobs.query` REST over HTTPS	no
`databricks`	Databricks	`exec-databricks`, `exec-databricks-arrow`	Statement Execution API	opt-in

redshift is accepted as a type: value and compiles to SQL, but has no executor yet: naming it in --execute fails with no executor for redshift yet (planned).

What your build includes¶

Published binaries and the dosi-engine wheel ship every connector. A hand-built binary defaults to DuckDB only; ask for the rest with --features exec-all, or name them individually. A connector that is missing says so:

this build has no postgres executor (feature "exec-postgres" not enabled)
hint: rebuild with --features exec-postgres

Troubleshooting¶

These come from the connections file itself, before any connector is reached.

Message	Cause
`no connections file found; pass --connections <path>, set DOSI_CONNECTIONS, or create ./dosi-connections.yaml`	None of the seven discovery paths exist.
`<path> has no datasources: map (nor services.datasources)`	The file parsed, but the profiles are somewhere else in it.
`<path> uses the removed connections: list format`	Pre-1.0 file; move the entries under a `datasources:` map.
`datasource "x" is missing required field "type"`	Every profile needs `type:`.
`datasource "x": unknown dialect "postgresql"`	Use the exact spelling from the routing table (`postgres`).
`datasource "x": "user" is the legacy osi-connections spelling`	See the rejected-spellings table above.
`<path> marks 2 datasources as default: a, b`	At most one `default: true`.
`datasource "x": environment variable "PW" is not set`	Export it, or write `${PW:-fallback}`.
`no connection named "x"` — with `hint: available: …`	Typo in `--connection`; the hint lists the names the file defines.

Next¶

CLI reference — --connections, --connection, --execute
Arrow — which connectors stream Arrow, and what it buys
REST API — the same profiles, served over HTTP