Skip to main content

Deployment

The node exports to an OpenTelemetry Collector, or to any OTLP endpoint. One endpoint carries all three signals: /v1/logs, /v1/metrics and /v1/traces under the same host for HTTP, and one host:4317 for gRPC.

In a hugr platform node​

A platform node runs hugr-node's hugr_node, which has acl_otel built in (with duckdb-acl and tresor): there is nothing to LOAD, and the settings and functions on this site are the same. Until an endpoint is set it exports nothing. In a cluster the destination is the cluster's - acl_otel_endpoint, its protocol and TLS, and the lineage backend are cluster profile items (ACL CLUSTER SET), one for every node and never a group's.

Configure it the OpenTelemetry way​

Leave the settings at their defaults and set the environment:

OTEL_EXPORTER_OTLP_ENDPOINT=https://collector.example.com:4318
OTEL_EXPORTER_OTLP_HEADERS=authorization=Bearer <token>
OTEL_EXPORTER_OTLP_CERTIFICATE=/etc/ssl/collector-ca.pem

Headers carry credentials, so they come only from the environment; no setting can hold them. A Collector that faces a public network should terminate TLS on its receivers.

Application Insights​

deploy/otel-collector-azuremonitor.yaml is a reference Collector configuration. It forwards the records to Application Insights through the contrib distribution's azuremonitor exporter. The mapping needs no rewrite:

OpenTelemetryApplication Insights
service.namecloud_RoleName
service.instance.idcloud_RoleInstance
acl.* attributescustomDimensions
INFO / WARN / ERRORseverityLevel 1 / 2 / 3
W3C trace contextoperation_Id / operation_ParentId

The connection string is a secret. Keep it in the Collector's environment.

A local stack​

deploy/local-stack/ runs five containers so you can see what a node sends:

  • an OpenTelemetry Collector;
  • Loki for the logs;
  • Prometheus for the metrics;
  • Tempo for the traces;
  • Grafana to look at all three.
cd deploy/local-stack && docker compose up -d # Grafana on http://localhost:3000
LOAD acl;
LOAD acl_otel;
SET GLOBAL acl_otel_endpoint = 'http://127.0.0.1:4318';
SET GLOBAL acl_otel_service_name = 'acl-node-demo';
SET GLOBAL acl_otel_traces = 'linked';
SET GLOBAL acl_audit_level = 'all';
-- ... statements under a principal, some with TRACE '<id>' PARENT '<traceparent>' ...
SELECT acl_audit_flush(); SELECT acl_otel_flush(); SELECT acl_otel_metrics_flush(); SELECT acl_otel_traces_flush();

Where to look in Grafana:

  • Explore → Loki: {service_name="acl-node-demo"} shows the decisions with their acl.* attributes.
  • Explore → Prometheus: acl_decisions_total or acl_otel_healthy.
  • Explore → Tempo: search by the trace id. The decision is a bar named acl SELECT inside the request.

A readiness probe​

With acl_otel_strict on, SELECT acl_otel_healthy() returns false while the node is losing audit events or is not attached (Metrics). An orchestrator can take such a node out of rotation.