Skip to main content

Concepts

The attached catalog is three things​

ATTACH 'tresor:…' AS corp creates one name with three roles:

  1. A secret storage named corp in DuckDB's secret manager. Its secrets take part in every lookup; CREATE PERSISTENT SECRET … IN corp stores into the service and DROP PERSISTENT SECRET … FROM corp deletes there. SET default_secret_storage = 'corp' makes the service the default for persistent secrets.
  2. A catalog of functions — corp.secrets(), corp.whoami(), corp.grants(…), corp.annotate_secret(…), corp.grant_secret(…), corp.revoke_secret(…) — the view, and the management surface for administrators; corp.variable(…) and its siblings where the service holds variables (optional); and corp.act_for_sessions(…) for a duckdb-acl node.
  3. A login. The identity the attach established is what every call to the service carries.

Several services can be attached at once; each is its own storage and its own catalog.

Lookup​

DuckDB picks a secret by how well its scope matches the path in hand. tresor keeps the list of secrets your role may use (names, types, scopes — no material) and answers DuckDB's lookup from it; the material is fetched only when a secret matches, and cached until it expires. Secrets the service marks dynamic are generated per request with an expiry — short-lived credentials — and S3 secrets among them are refreshed through httpfs's own REFRESH auto when they run out.

A secret is served as its extension made it: a type the extension registered with a class of its own (BigQuery's) reaches it in that class, as one read from a persistent secret file would. Such extensions usually renew their tokens themselves. For BigQuery, prefer a workload identity federation config (external_account_json): it holds no key, and BigQuery renews its token. An access_token the service mints on every fetch suits a short-lived attach; an authorized user (refresh_token, client_id, client_secret) works too. Do not serve a service account key (service_account_json): whoever may use the secret gets the long-lived key itself.

What your role may do​

Two kinds of callers, and one rule each:

  • Users use. You use exactly the secrets an administrator granted to one of your roles or groups (role:…, group:… from your token). You do not create or share secrets in the service: keep your own in your DuckDB (CREATE SECRET). corp.secrets() shows permissions = [use].
  • Administrators manage. They create, change, annotate and delete secrets, and grant use to roles and groups. An administrative role does not imply use: to use a secret, an administrator grants it to one of their own roles, like anyone else.

Nobody but an administrator can put a secret where your lookups find it, so nobody can plant one on your paths.

A scheduled job is a caller of its own: it logs in as a service, with a role, never with a person's login (Pipelines).

Acting for users​

A server (a DuckDB node behind a gateway, a data platform) runs statements for its users. It holds a role of its own, and an administrator grants that role what the server serves: its lake, its catalogs, its databases. When a user's session opens, the server obtains a delegation grant for them. Through it:

  • the user gets exactly the server's secrets for the statements the server runs for them, and nothing more. The server never adds the user's rights to its own, nor its own to the user's beyond what an administrator granted it;
  • an administrator can manage through the server: only a user who is an administrator, and only what the service's policy lets that server pass on.

A token for the caller​

Some secrets are a token rather than a stored credential. Examples are an http API that authorises per user, or another acl node through quack:

ATTACH 'tresor:secrets.example' AS corp;
ATTACH 'quack:corp.duck' AS remote (TYPE quack); -- its secret comes from corp, with your own token

An administrator stores such a secret with a parameter the service generates on every read, a token for the downstream audience (the reference server and tresor-server write it mint+exchange://<audience>, generated parameters). You then get a fresh token of your own for it, minted by the identity provider. Through a node acting for your session, you still get a token of your own, never the node's. tresor caches it until shortly before it expires.

On a duckdb-acl node​

A node running duckdb-acl attaches the service as itself and acts for the users whose sessions it serves:

CREATE SECRET node (TYPE tresor, FLOW client_credentials, ISSUER 'https://idp.example/realms/corp',
CLIENT_ID 'acl-node', CLIENT_SECRET '…');
ATTACH 'tresor:secrets.example' AS corp (SECRET node, ACT_FOR_SESSIONS true);

Load duckdb-acl before this ATTACH: ACT_FOR_SESSIONS is refused where nothing publishes acl sessions.

A node that installs duckdb-acl from a repository whose secret the service holds attaches tresor first, and turns acting on once acl is loaded, with no re-attach:

ATTACH 'tresor:secrets.example' AS corp (SECRET node); -- the repository's secret is in the lookup now
INSTALL acl FROM '…'; LOAD acl;
CALL corp.act_for_sessions(); -- the options below, in lowercase: exchange := 'on_behalf_of', …
  • act_for_sessions() makes the same checks as ACT_FOR_SESSIONS, at the call.
  • It is one way: DETACH ends it.
  • It is idempotent: the same options again change nothing (changed = false), and other options are refused.
  • It is refused under an acl session.
  • A session opened before it gets no grant: it ends, and the next one does.

When a user's acl session opens, tresor:

  1. exchanges the session's token at the identity provider for one meant for the service;
  2. trades that token for a delegation grant;
  3. uses the grant for every statement of the session: secret lookups (the node's secrets, served for the user's statement), corp.secrets(), corp.whoami() (which answers the user, with actor = the node), and an administrator's management;
  4. revokes the grant when the session ends.

A statement under a session never runs with the node's bare identity: always through the session's grant, or not at all. If the grant is still pending, lookups wait up to SESSION_GRANT_WAIT seconds (10 by default). If the exchange or the grant failed, lookups in corp find nothing and explicit calls fail with the reason.

OptionDefault
ACT_FOR_SESSIONSfalseact for duckdb-acl's sessions
EXCHANGE'token_exchange''token_exchange' (RFC 8693: Keycloak, Okta, …) or 'on_behalf_of' (Entra)
EXCHANGE_SCOPE—the scope asked for; required for on_behalf_of (api://…/.default)
EXCHANGE_AUDIENCEthe discovery'sthe audience users' tokens are exchanged for; without it, the discovery's is accepted only if the node's own token carries it
SESSION_GRANT_WAIT10seconds a session's statement waits for its grant

The identity provider must allow the node's client to exchange tokens. In Keycloak (standard token exchange):

  • set standard.token.exchange.enabled on the node's client;
  • give the users' client an audience mapper that names the node's client;
  • give the node's client an audience mapper that names the service's client.

The test realm in server/testdata/keycloak has all three. For Entra, see Microsoft Entra ID. With Entra, the token's iss must equal the issuer the node logs in with: v1 tokens (https://sts.windows.net/<tenant>/) against a v2 issuer are refused as "from another issuer".

What tresor did: audit and traces​

tresor records what it did as events: logins and logouts, secret lookups and refreshes, writes, drops, annotations, grants and revocations, and on a duckdb-acl node each session's delegation grant (obtained, failed, revoked, rejected, expired). An event names:

  • the service and the caller;
  • the secret (never its material);
  • the outcome, and for a refusal a bounded reason code;
  • the time the service took.

It never carries a token, a session handle, a grant id or a statement's text. The events go to two places, each off unless you ask for it:

  • DuckDB's log. tresor_audit_level is off (the default), denied (refusals and failures) or all. DuckDB keeps a row only while its own logging is on for the tresor type:

    SET tresor_audit_level = 'all';
    CALL enable_logging('tresor');
    SELECT kind, outcome, secret, reason_code, duration_us FROM duckdb_logs_parsed('tresor');

    A row written for a statement names its connection and query, like any DuckDB log row. A lookup served from tresor's memory is not logged: a scan asks for its secret once per file.

  • The tresor_audit contract (duckdb-ext-common), for an exporter loaded beside tresor. On a duckdb-acl node, acl-otel turns the events into OpenTelemetry logs, spans and metrics. With no exporter loaded, no event is even composed.

Traces. Under an acl session, an event carries the statement's traceparent and correlation id, as duckdb-acl publishes them. An exporter places tresor's work in the trace of the statement that caused it. tresor also sends the traceparent to the service (protocol, Tracing), so a service that traces continues the same trace.