Skip to main content

The reference server

server/ in tresor's repository is a small implementation of the duckdb-secrets/1 protocol in Go. It exists for two reasons: tresor's end-to-end tests run against it next to a real Keycloak, and a company writing its own service can read it to see the protocol working end to end. It is not meant for production: one process, one encrypted file, no high availability. A production service is being designed separately: tresor-server.

What it implements​

  • Discovery and whoami.

  • Secrets: list, read, create/replace with preconditions (If-None-Match: *, If-Match), delete, annotate.

  • Grants.

  • Variables (optional in the protocol, capabilities.variables): the secrets' rules in a namespace of their own. It resolves no references, so none of its variables is sensitive.

  • Token verification against any number of OIDC issuers:

    • the signature against the issuer's JWKS;
    • iss (verbatim: an issuer ending in /, like Auth0 or Entra v1, works), expiry, and the audience;
    • asymmetric algorithms only.

  • Tracing: the request log names the caller's trace_id and parent_span_id from a well-formed traceparent. It does not export spans; a production service would.

  • Delegation: grant exchange, the Delegation header, and actor policy (tresor specs/009: under a grant a server uses its own grants for the user, and passes management through only for admins).

Dynamic secrets: any parameter of any secret type may be generated per caller, a token for them or a claim of their token (generated parameters, below). Issuers must be https unless they are on loopback: their signing keys are fetched from them.

Run it​

cd server
go build -o ref-server ./cmd/ref-server
REF_SERVER_KEY=$(openssl rand -base64 32) ./ref-server -config server.yaml
listen: 127.0.0.1:8443
public_url: http://127.0.0.1:8443        # what discovery calls `api`
# tls: {cert: server.crt, key: server.key}   # required unless listening on loopback
store:
  path: data/secrets.enc                 # AES-256-GCM; omit for memory only
  key_env: REF_SERVER_KEY                # 32 bytes, base64
issuers:
  - issuer: https://login.corp.example/realms/main
    audience: duckdb-secrets             # the token's `aud` must contain it
    client_id: duckdb                    # the public client people log in with
    scopes: [openid]
    human_flows: [authorization_code, device_code]
    service_flows: [client_credentials, private_key_jwt]   # also: federated, managed_identity
    # audience_parameter: true           # Auth0: clients send audience=<audience> (tresor specs/013)
    roles_claim: realm_access.roles      # Keycloak; Entra/Okta: roles
    groups_claim: groups
    service: {claim: client_id}          # what marks a client-credentials token (see below)
policy:
  admins: [role:secrets_admin, client:etl]   # the only principals that manage secrets and grant use
  actors:                                    # servers that may act for users
    - {principal: client:acl-node, verbs: [use]}      # its own grants, for its users' statements
    - {principal: client:ops-node, issuer: https://login.corp.example/realms/main,
       verbs: [use, create, update, delete, annotate, grant]}   # admins may manage through it
# cors_origins: [https://app.corp.example]   # web pages whose DuckDB-wasm calls the API (tresor specs/020)

Browsers. cors_origins lists the origins of the web applications that run tresor in DuckDB-wasm (https://host[:port], or http on loopback for a dev server; no wildcard). The server answers their preflights and marks their responses; any other origin gets no CORS header, so browsers refuse it. No credentials mode is involved: the page sends a bearer token, never a cookie.

Who may do what​

  • Principals. Every caller is subject:<issuer>|<sub>, which is its identity. Its roles become role:<name> and its groups group:<name>.
  • Services. A token is a service's only by its issuer's service rule: claim is present, and equals equals when that is set. The service then also gets client:<name>, taken from client_claim (default azp). No claim marks a service by convention. RFC 9068 puts client_id into every access token, a person's included. Without a rule, every caller is a person.
    • Keycloak: {claim: client_id}. The service_account scope sets it on client-credentials tokens only.
    • Entra: {claim: idtyp, equals: app}.
  • Verbs on a secret (tresor specs/009). use comes only from a grant to one of the caller's roles or groups; grants name role: or group: principals, with use only. Admins (policy.admins) hold the management verbs on every secret, and create, but an admin role implies no use: an admin uses a secret when one of its roles is granted it. Users create and grant nothing.
  • client: names are shared. They carry no issuer, so with several issuers a client:etl in a policy matches the etl of each.
  • Invisible secrets. A secret you hold no verb on answers 404, exactly like one that does not exist.

Delegation​

  • Actors. A server listed in policy.actors, optionally pinned to the issuer of its token, exchanges a person's token for a grant. The grant lives in memory only (8 h at most, and at most 100 000 grants), is bound to that server, and is never logged.
  • Revocation. Admins revoke by actor or subject (DELETE /v1/delegations?actor=…), and users revoke their own grants.
  • Under a grant:
    • use is the server's own (its roles' grants), for the grant's user: nothing beyond what an admin granted the server;
    • a management verb (or create) passes only for a user who is an admin, and only if the actor's verbs list it;
    • everything else is 403 actor_not_allowed.

Generated parameters​

The protocol lets a service generate a parameter on each read (dynamic secrets) and leaves how it is written to the service. The reference server and tresor-server share the whole-value syntax below. tresor-server reads more (interpolation, typed values, its own kinds), so a store moved there escapes what it must keep literal (below).

An administrator writes a generator as the whole value of a VARCHAR parameter (a JSON string) of a secret of any type, with the provider the consuming extension knows:

{"type": "mssql", "provider": "access_token", "scope": ["mssql://sql.corp.example"],
 "params": {"host": "sql.corp.example", "access_token": "mint+exchange://https://database.windows.net/?scope=user_impersonation"}}
ValueGeneratesReference server
mint+exchange://<audience>[?scope=<scope>]a token for the caller at the identity provider, for <audience> (RFC 8693, below)yes
mint+claim://<claim>the text of a claim of the caller's verified token: a string as it is, a number or a boolean as its JSON textyes
mint+login://…tresor-server's own (see its documentation)no
mint+rule://…tresor-server's own (see its documentation)no
  • The audience is everything up to the first ?, percent-decoded (%3F for a ? in it). The query is scope=<scope> and nothing else, form-encoded (read+write is read write).
  • The claim is a top-level claim, named exactly. A claim that is missing, empty, a list or an object is refused at the read (403 mint_refused), and so is one shaped like a JWT. The served value is the claim, never the token it came from. Claims that name the caller's IdP session or bind a token (sid, session_state, jti, nonce, at_hash, c_hash, s_hash) are refused at PUT. Prefer claims the user cannot change (sub, an object id) to editable ones (preferred_username, where the IdP lets users edit it) for anything that grants access.
  • For whom. For the effective user: the direct caller, or under a delegation grant the grant's user, never the server.
  • What is served. The secret is dynamic; the descriptor's provider is the stored one; each generated key is added to redact_keys; expires_at is the earliest expiry: a minted token's, a claim's token's (under a grant, the grant's).
  • Failures: 403 mint_refused for a lasting refusal, 503 for an outage, never an empty value.
  • Only administrators write a generator, as they write any secret. At PUT a generator must be well formed and of a known kind, and an exchange for this service's own audience is refused; the refusal names the parameter, never its value.
  • A literal beginning with mint+ is written with the + doubled: a stored mint++x is served as mint+x, and mint+++x as mint++x. Exactly one + after mint is a generator; two or more are a literal, with one dropped. The prefix is lowercase: MINT+x is a literal as it is.
  • Whole values only, here. tresor-server also generates a part of a string (${…} inside a value, including values inside MAP, STRUCT and LIST); a literal ${ there is written $${. The reference server reads no ${…} and no $${: it serves such a value as written. A typed value ({"type": …, "value": …}) is always a literal here, never unescaped (a typed mint++x is served as mint++x). Moving a store to tresor-server, escape every literal ${ as $${.
  • Material is a resolved copy. An administrator does not read a stored generator back: GET serves what it generated, and an escaped literal unescaped. Never write material back with PUT: a served literal mint+claim://sub written back is a generator.

A token for the caller​

mint+exchange://<audience> mints a token for the caller. The older form still works as an alias: a secret with provider: token_exchange and the downstream audience (and scope) parameters, for an http (bearer_token) or quack (token) secret, is served as if that parameter were mint+exchange://<audience>?scope=<scope>, with the consumer's provider config (the protocol never serves a name for the service's mechanism):

{"type": "quack", "provider": "token_exchange", "scope": ["quack:corp.duck"], "params": {"audience": "acl-node"}}

On every read, the service mints a token for the caller at the identity provider (RFC 8693, as its own client), and serves it as a dynamic secret:

  • a caller reading directly gets a token exchanged from the one it called with;

  • under a delegation grant, the grant's user gets a token, never the server. At the grant's exchange the service exchanges the user's token for each audience the server may use, with a refresh token kept with the grant in memory, and renews from it while the user's IdP session lives.

  • Failures: a lasting refusal is 403 mint_refused, an outage 503.

  • An outage at the grant's exchange does not spoil the session. The grant keeps the user's token for this service until that token expires (minutes), and mints from it later, as it does for a secret granted to the server after the session opened.

  • A minted token is checked: its aud must name the audience asked for, and never this service itself. At PUT, an audience equal to this service's own is refused.

  • Refresh tokens are dropped, not revoked at the IdP (RFC 7009), when a grant ends. They stay valid at the IdP until the user's SSO session ends, and renewing keeps that session from going idle.

The issuer names the service's client, whose secret comes from the environment:

issuers:
  - issuer: https://idp.example/realms/corp
    exchange: {client_id: duckdb-secrets, client_secret_env: TRESOR_EXCHANGE_SECRET}

Keycloak (standard token exchange):

  • the service's client: standard.token.exchange.enabled, standard.token.exchange.enableRefreshRequestedTokenType: SAME_SESSION, and an audience mapper per downstream audience;
  • the node's client: the same refresh attribute, because tresor's exchange asks for a refresh token so that Keycloak binds the token to the user's session. The test realm has both.

Upgrading from before tresor specs/009​

  • policy.create is gone. A config that still has it is refused with that message; list the principals who may create in policy.admins.
  • Old grants are ignored. A stored grant to a subject: or a client:, or of any verb but use, gives nothing now, and the server logs each at start. Re-grant use to a role or a group. A service account (a node) is granted through a role of its token.
  • Delegation rules are dropped when a stored file is loaded.
  • An actor's verbs changed meaning. use is now the server's own grants, used for its users. Any other verb lets admins manage through the server. Keep them to what you mean to allow; the test configuration lists every verb for its node, which is not a production default.
  • Admin status is taken at the grant's exchange. A demoted admin keeps managing through a node until the grant expires (8 h at most) or is revoked (DELETE /v1/delegations?subject=…).

Identity provider setup (Keycloak)​

server/testdata/keycloak/realm-tresor.json is the realm the tests import. What matters for a real deployment:

  • A public client for people with the standard flow, PKCE S256 and the redirect URI http://127.0.0.1/*. Keycloak accepts any port on a loopback redirect, as RFC 8252 requires. Enable the device grant if people log in over SSH.
  • An audience mapper that puts the service's audience into access tokens, on the people's client and on every service client.
  • Service clients with service accounts. A client created in the admin console gets the service_account scope, which adds the client_id claim. The imported test realm adds the same mapper by hand.

Conformance​

tresor's repository carries the conformance suite: sqllogictests in test/sql/conformance/, driven against a service URL by environment variables. Build tresor, then:

TRESOR_CONFORMANCE_HOST=secrets.corp.example \
TRESOR_CONFORMANCE_INSECURE=false \
TRESOR_CONFORMANCE_ISSUER=https://login.corp.example/realms/main \
TRESOR_CONFORMANCE_CLIENT_ID=conformance TRESOR_CONFORMANCE_CLIENT_SECRET=… \
build/release/test/unittest --skip-error-messages '' 'test/sql/conformance/*'

Set TRESOR_CONFORMANCE_PERSON=1 and a BROWSER that can complete your identity provider's login to include the person flow. To include the secrets case, seed a secret the conformance client may use and name it: TRESOR_CONFORMANCE_SECRET, _SECRET_TYPE, _SECRET_PATH (covered by its scope), _SECRET_KEY and _SECRET_VALUE (a VARCHAR parameter). scripts/ci/test_keycloak.sh runs the whole suite against the reference server and Keycloak in docker.

Entra ID​

Setting the server up behind Microsoft Entra ID is its own page: Microsoft Entra ID. It covers the app registrations, the issuer entry and the live check (scripts/dev/entra_live.sh).