Skip to content

Warehouse connectors

Dosi executes by SQL pushdown: it compiles a metric query to dialect SQL and the warehouse runs it. A connector handles connection setup, statement submission, and normalizing results into a ResultSet — columns plus rows of Null | Bool | Int | Float | Str, dates as ISO-8601 strings — so results compare equal across engines.

This section is the setup guide. Start here for the connection-profile format that every connector shares, then follow the routing table below to your database's own page. Arrow-native result transfer (DuckDB, ClickHouse, StarRocks/Doris, Databricks) is described in Arrow.

Let your agent connect

With the Dosi plugin in your coding agent, you don't write the connection yourself. Say "connect to my data", or name the source ("connect to our Snowflake, account xy12345"), and the agent:

  • finds what is already there. It reads a dbt profiles.yml or a Cube .env through dosi migrate conf, connects a local .duckdb or .sqlite file directly, or writes a warehouse entry from the connector's template, asking only for the fields that are not secret;
  • keeps the password out of the conversation. It writes a ${NAME} placeholder and tells you where the real password goes: in dosi-connections.yaml itself, which it adds to .gitignore, or as NAME=... in ~/.config/dosi/secrets.env. It never asks for a password in the chat and never opens these files;
  • tests the connection with dosi connection test and reports it in one line, or, when it fails, what happened, why, and the next step.

To change a connection later, ask the agent. It cannot read the file, so it tells you which line to edit. The quickstart shows the whole flow on a sample database.

The rest of this page is the file the agent writes: read it to set up a connection manually, or to review what the agent wrote.

Connection profiles

Connection profiles use the Datus agent.yml datasources: vocabulary. Dosi runs queries through its own connectors and shares the common configuration vocabulary with Datus. Each connector page defines its transport and authentication settings; Redshift Data API profiles use target and identity keys in place of TCP host/password fields.

--connections <path> (env DOSI_CONNECTIONS) accepts either a full agent.yml (profiles under services.datasources) or a standalone file with the same map under a top-level datasources:.

datasources:
  local-duck:
    type: duckdb
    uri: duckdb:///warehouse.duckdb   # omit for in-memory
  prod-sr:
    type: starrocks
    host: sr.internal
    port: 9030
    username: dosi
    password: ${SR_PASSWORD}          # ${VAR} / ${VAR:-fallback} from the environment
    database: analytics
    default: true                     # used by --execute without --connection

Run it:

$ dosi query --metrics revenue --group-by orders.status --execute --connection prod-sr

--execute without --connection uses the profile marked default: true if there is one, else local DuckDB (--db <file>, or in-memory).

Where the file is discovered

Without --connections, the first path that exists wins:

# Path
1 $DOSI_CONNECTIONS
2 ./dosi-connections.yaml
3 ./osi-connections.yaml (pre-rename fallback)
4 ~/.config/dosi/connections.yaml
5 ~/.config/osi/connections.yaml (pre-rename fallback)
Each legacy osi- path sits directly after its dosi- counterpart, not at
the end of the list. A Datus agent.yml (./conf/agent.yml,
~/.datus/conf/agent.yml) is not discovered; name it with --connections to
use it.

Keys every connector shares

Key Type Required Default Notes
type string yes — Selects the connector, case-insensitively. Use a value from the routing table below; postgresql and greenplum are also accepted and use the postgres connector and SQL dialect. pg and opengauss are not aliases.
default bool no false At most one profile in the file may set it. Used by --execute without --connection. Compile-only query and select also use its dialect when no explicit dialect, connection or local database is selected.

Every other key is per-connector — its page lists the ones that connector reads, and the ones it parses and ignores.

Secrets and ${VAR}

Any string value may reference the environment as ${VAR} or ${VAR:-fallback}. For password: and private_key_file_pwd: the exact form matters:

  • A value written exactly as ${VAR} resolves lazily — the file loads with the variable unset, and the error (carrying an export hint) fires only when that connection is used.
  • A value that merely contains ${VAR} resolves when the file loads.
  • A plaintext secret works, and warns on stderr: warning: connection "x" has a plaintext password in the connections file; prefer password: ${VAR}.

From dbt or Cube

If dbt (or MetricFlow) or Cube already connects to your warehouse, dosi migrate conf writes the entries for you, then dosi connection test checks one:

$ dosi migrate conf
$ dosi connection test analytics.dev

dbt. profiles.yml is found the way dbt finds it — $DBT_PROFILES_DIR, then ./profiles.yml, then ~/.dbt/profiles.yml. Every profile × target becomes one entry named <profile>.<target>; the target your dbt_project.yml's profile selects is marked default. {{ env_var('X') }} becomes ${X} and {{ env_var('X', 'd') }} becomes ${X:-d}; any other Jinja refuses the target with unsupported_config.

Cube. CUBEJS_DB_* settings are read from ./.env (never a parent directory) and the environment. CUBEJS_DATASOURCES=default,events with CUBEJS_DS_EVENTS_DB_* gives one entry per data source, named cube.<name>; cube.default is marked default. A value set in the environment is written as ${NAME}; a value only in .env is copied. A driverFactory in cube.js / cube.py builds connections in code and is reported, not migrated.

dbt type / Cube CUBEJS_DB_TYPE Becomes Refused with unsupported_auth
snowflake snowflake (key pair: private_key_path) password, inline private_key, any authenticator other than snowflake_jwt (browser SSO, OAuth, MFA, Okta)
bigquery bigquery (keyfile, keyfile_json, CUBEJS_DB_BQ_KEY_FILE / _CREDENTIALS) method: oauth and other token methods, no key at all
databricks / databricks-jdbc databricks (SQL warehouse from http_path / httpPath, token) no token (OAuth)
postgres, mysql, tidb, starrocks, doris, oracle the same type —
duckdb duckdb (path made absolute) MotherDuck (md:) is unsupported_dialect
trino trino method other than none; a password or token
athena Amazon Athena exec-athena
clickhouse clickhouse over HTTP(S) — (driver: native is unsupported_config)
anything else (redshift, sqlserver, spark, ...) not written: unsupported_dialect

Interactive sign-in is refused because nobody is at the keyboard when an agent runs dosi; switch that target to a key or token and run the command again.

Values. The report never prints them. In the file, literal values — passwords included — are copied verbatim, the file is written owner-only (0600), and when it sits inside a git work tree its path is added to that tree's .gitignore. Re-running replaces entries of the same name and leaves every other entry alone; the entry already marked default keeps the role — also when its settings are replaced — and no imported target takes it over. A Datus agent.yml is never rewritten — pass --out for a standalone file.

What a broken entry does

A profile that fails to parse — an unset ${VAR}, an unknown type: — never takes the file down. Its error is stored and replayed only when that name is used, so one bad entry cannot stop the profiles you are actually running.

Unknown keys are ignored: Datus adapters keep private settings in the same file. The pre-1.0 spellings are the exception — they are rejected with a migration hint rather than silently doing nothing:

Rejected Use instead
dialect: type:
user: username:
password_env: password: ${VAR}
private_key_passphrase: private_key_file_pwd:
private_key_passphrase_env: private_key_file_pwd: ${VAR}
path: uri: duckdb:///<path>
url: uri:
a top-level connections: list a datasources: map keyed by name

Connectors

type: Page Cargo feature How it talks to the engine Arrow-native
duckdb DuckDB exec-duckdb (default) in-process, shipped library yes
sqlite SQLite exec-duckdb local file, read-only, through DuckDB yes
mysql MySQL exec-mysql MySQL wire protocol no
tidb TiDB exec-mysql MySQL wire protocol no
starrocks StarRocks exec-mysql, exec-flightsql MySQL wire, or Arrow Flight SQL opt-in
doris Apache Doris exec-mysql, exec-flightsql MySQL wire, or Arrow Flight SQL opt-in
postgres PostgreSQL exec-postgres Postgres wire protocol no
hologres Hologres exec-hologres Postgres wire protocol no
oracle Oracle Database exec-oracle OCI through ODPI-C no
hana SAP HANA exec-hana SAP HDBODBC no
clickhouse ClickHouse exec-http, exec-http-arrow HTTP interface opt-in
trino Trino exec-http REST /v1/statement no
snowflake Snowflake exec-snowflake SQL API v2 over HTTPS no
bigquery BigQuery exec-bigquery jobs.query REST over HTTPS no
databricks Databricks exec-databricks, exec-databricks-arrow Statement Execution API opt-in
redshift Amazon Redshift exec-redshift Data API through AWS SDK no

What your build includes

Published binaries and the dosi-engine wheel ship every connector. A hand-built binary defaults to DuckDB only; ask for the rest with --features exec-all, or name them individually. A connector that is missing says so:

this build has no postgres executor (feature "exec-postgres" not enabled)
hint: rebuild with --features exec-postgres

Troubleshooting

These come from the connections file itself, before any connector is reached.

Message Cause
no connections file found; pass --connections <path>, set DOSI_CONNECTIONS, or create ./dosi-connections.yaml None of the seven discovery paths exist.
<path> has no datasources: map (nor services.datasources) The file parsed, but the profiles are somewhere else in it.
<path> uses the removed connections: list format Pre-1.0 file; move the entries under a datasources: map.
datasource "x" is missing required field "type" Every profile needs type:.
datasource "x": unknown dialect "pg" Use postgres, postgresql or greenplum; pg is not a supported alias.
datasource "x": "user" is the legacy osi-connections spelling See the rejected-spellings table above.
<path> marks 2 datasources as default: a, b At most one default: true.
datasource "x": environment variable "PW" is not set Export it, or write ${PW:-fallback}.
no connection named "x" — with hint: available: … Typo in --connection; the hint lists the names the file defines.

Next

  • CLI reference — --connections, --connection, --execute
  • Arrow — which connectors stream Arrow, and what it buys
  • REST API — the same profiles, served over HTTP