Redshift connector¶

Dosi runs Redshift SQL through the Amazon Redshift Data API, using the official AWS Rust SDK. Enable exec-redshift when building from source; exec-all and exec-all-lean include it.

Experimental

This connector supports Provisioned clusters and Serverless workgroups. Results arrive as JSON rows; Arrow output uses Dosi's shared row conversion. Live cloud acceptance is opt-in and does not run in ordinary CI.

Connection profile¶

datasources:
  redshift:
    type: redshift
    cluster_identifier: analytics          # Provisioned
    # workgroup_name: analytics            # Serverless: use this instead
    database: dev
    schema: public
    aws:
      region: us-east-1
      profile: analytics                   # optional AWS shared profile
    secret_arn: ${REDSHIFT_SECRET_ARN}       # optional; omit for IAM identity

Select exactly one of cluster_identifier and workgroup_name. The Data API uses HTTPS and does not need inbound database access from Dosi. A TCP profile containing host, port, uri, password, ssl, sslmode or sslrootcert must be converted to the profile above.

Authentication¶

AWS credentials come from refreshable SDK providers: environment credentials, shared profiles, assume-role profiles, web identity, or container/instance identity. A named aws.profile explicitly selects that profile, including its role chain. Configure SSO profiles with the AWS CLI and sign in before running Dosi.

Database authentication has two options:

Secrets Manager: set secret_arn. Its secret supplies the database username and password; omit username in the Dosi profile. The AWS identity needs permission to read the secret and use the Data API.
IAM identity: omit secret_arn. Provisioned can set username to select an existing database user, requiring redshift:GetClusterCredentials; without it, IAM-derived authentication uses redshift:GetClusterCredentialsWithIAM. Serverless derives the user from IAM and requires redshift-serverless:GetCredentials; omit username.

The AWS identity also needs redshift-data:BatchExecuteStatement, DescribeStatement, GetStatementResult, and CancelStatement, with access to the selected target. Database grants still apply. See Data API authentication.

Parameters¶

Key	Default	Meaning
`database`	required	Redshift database.
`schema`	unset	Search path for unqualified tables; qualified model table names still take precedence.
`cluster_identifier` / `workgroup_name`	exactly one required	Provisioned cluster / Serverless workgroup.
`secret_arn`	unset	Secrets Manager database credentials.
`username`	unset	Provisioned IAM database user only.
`aws.region`	SDK region chain	AWS region. Flat `region` is an alias.
`aws.profile`	SDK provider chain	AWS shared profile. Flat `profile_name` is an alias.
`aws.ca_bundle`	SDK trust roots	PEM CA bundle for AWS HTTPS. Flat `ca_bundle` is an alias.
`iam`	inferred	Optional validation hint: `false` requires `secret_arn`; `true` excludes it.
`statement_name_prefix`	`dosi`	1–128 ASCII letters, digits, `_` or `-`; Dosi appends a unique ID.
`timeout_seconds`	`30`	Budget for each AWS API call.
`query_timeout_seconds`	`600`	Submission and database execution budget.
`result_timeout_seconds`	`60`	Separate result-pagination budget.
`cancel_timeout_seconds`	`10`	Cleanup budget after cancellation or uncertain submission; zero disables cleanup.
`poll_interval_seconds`	`1`	Statement polling interval.
`max_rows`	`10000`	Maximum returned rows; exceeding it fails the query.
`max_result_bytes`	`16777216`	Maximum normalized result-value bytes; exceeding it fails the query.

Timeouts accept fractional seconds and are bounded at 86400 seconds. Conflicting nested and flat AWS settings are rejected. AWS access keys belong in credential providers, rather than datasource fields.

Execution and diagnostics¶

Each query uses an independent batch/session to set UTC, case-sensitive quoted identifiers, and its schema before executing generated SQL. Sessions are not shared between concurrent queries. Batch setup scripts are split into chunks of at most 37 application statements, and each chunk commits independently.

All result pages are read. NULL and empty strings remain distinct; decimals too precise for a float remain exact text. Dates and timestamps use the shared UTC normalizer. No partial result is returned when a result limit is exceeded.

Cancellation requests target the submitted statement ID and poll for a terminal state within the cleanup budget. A timeout error's execution_info records whether cancellation was confirmed, whether the query finished during cancellation, or whether cleanup could not be confirmed. SQL failures retain the original Redshift error and statement/substatement IDs. The corpus report retains these diagnostics alongside the generated SQL.

Value-based RANGE window frames are rejected by Redshift. Existing two-argument CORR/COVAR window limitations also apply. These failures remain visible in execution reports. Arithmetic lowering preserves fractional averages and division results across integer and decimal inputs.

Verify the connection¶

$ dosi query --model model.yaml --metrics revenue --group-by orders.status \
    --execute --connections connections.yaml --connection redshift

A missing feature produces a build-feature error. Authentication errors should be checked against the selected AWS profile and database grants. SQL rejection remains a Redshift error: SQL generation support does not guarantee that every PostgreSQL function or type exists in Redshift.