Skip to content

feat(helm): expose OTEL headers, service name, and resource attributes#2128

Draft
nirzOps wants to merge 1 commit into
kagent-dev:mainfrom
nirzOps:feat/otel-headers-service-name-resource-attrs
Draft

feat(helm): expose OTEL headers, service name, and resource attributes#2128
nirzOps wants to merge 1 commit into
kagent-dev:mainfrom
nirzOps:feat/otel-headers-service-name-resource-attrs

Conversation

@nirzOps

@nirzOps nirzOps commented Jul 1, 2026

Copy link
Copy Markdown

Fixes #2126.
Design discussion: #2127.

Draft per CONTRIBUTING.md > Large changes — un-drafting once maintainers weigh in on the API shape in #2127. Slack ping to #kagent-dev on CNCF Slack coming from @nirzeira.

Summary

Expose the four standard OpenTelemetry SDK env vars that the controller already reads via autoexport.NewSpanExporter and resource.WithFromEnv in go/core/internal/telemetry/tracing.go, but that helm/kagent/values.yaml did not surface:

Chart key (new) Rendered env var
otel.serviceName OTEL_SERVICE_NAME
otel.resourceAttributes (map) OTEL_RESOURCE_ATTRIBUTES
otel.tracing.exporter.otlp.headers (map) OTEL_EXPORTER_OTLP_TRACES_HEADERS
otel.logging.exporter.otlp.headers (map) OTEL_EXPORTER_OTLP_LOGS_HEADERS

Maps serialize to key1=value1,key2=value2 with sorted keys (same pattern already used for DEFAULT_AGENT_POD_LABELS). Everything renders only when non-emptyhelm template output is byte-identical for existing installs. No controller code change is required.

Unblocks self-service tracing to any authenticated OTLP backend (Langfuse Cloud, Honeycomb, Datadog, Grafana Cloud, New Relic, …) without the current workaround of maintaining out-of-band ConfigMap + Secret and wiring them via controller.envFrom.

What changed

  • helm/kagent/values.yaml — new keys with inline docs pointing users to controller.envFrom for secret headers.
  • helm/kagent/templates/controller-configmap.yaml — conditional rendering of the four env vars.
  • helm/kagent/tests/controller-configmap_test.yaml — new suite (11 tests) covering:
    • backward compat: none of the new keys render with default values
    • each new field renders independently when set
    • map fields serialize with sorted keys
    • full Langfuse-Cloud style end-to-end render

Non-changes

  • No Go / Python controller change. The controller already understands these env vars via the standard OTEL SDK. This is a chart-surface fix only.
  • No new secret-injection schema. Secret OTLP headers (Basic auth for Langfuse, API keys for Honeycomb / Datadog / …) stay on the existing controller.envFrom extension point — the ConfigMap OTEL_EXPORTER_OTLP_TRACES_HEADERS is transparently overridden by the Pod-level env var. values.yaml comments document this. Happy to add headersSecretRef in a follow-up if maintainers prefer.
  • helm/tools/querydoc/templates/configmap.yaml has the same gap but is intentionally left for a separate PR to keep this reviewable.

Verification

$ helm lint helm/kagent/
==> Linting helm/kagent/
[INFO] Chart.yaml: icon is recommended
1 chart(s) linted, 0 chart(s) failed

$ helm unittest helm/kagent
Charts:      1 passed, 1 total
Test Suites: 16 passed, 16 total
Tests:       225 passed, 225 total

Rendered output with defaults (unchanged from main):

# OpenTelemetry Configuration
OTEL_TRACING_ENABLED: "false"
OTEL_LOGGING_ENABLED: "false"
# Using separate endpoints for traces and logs

Rendered output with a Langfuse-Cloud style overlay:

OTEL_TRACING_ENABLED: "true"
OTEL_SERVICE_NAME: "kagent"
OTEL_RESOURCE_ATTRIBUTES: "deployment.environment.name=production,langfuse.environment=production"
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: "https://cloud.langfuse.com/api/public/otel/v1/traces"
OTEL_EXPORTER_OTLP_TRACES_PROTOCOL: "http/protobuf"
OTEL_EXPORTER_OTLP_TRACES_INSECURE: "false"
OTEL_EXPORTER_OTLP_TRACES_HEADERS: "x-tenant-id=platform"

Downstream context

The workaround this PR obsoletes is currently deployed in elementor/elementor-cloud-external-charts@1e8b9436 + elementor/elementor-agents@13545eb. Both will be simplified to native otel.* values once this lands.

Checklist

The Go controller already reads standard OTEL SDK env vars through
`autoexport.NewSpanExporter` (headers, protocol, endpoint, insecure,
timeout) and `resource.WithFromEnv` (service name, resource
attributes), but the Helm chart's `otel.tracing` / `otel.logging`
config only surfaces endpoint/protocol/insecure/timeout. Anyone
sending traces to an authenticated OTLP backend (Langfuse Cloud,
Honeycomb, Datadog, New Relic, Grafana Cloud, ...) currently has to
bypass the chart's `otel.*` config entirely and inject env vars via
`controller.envFrom` from an out-of-band ConfigMap/Secret.

This change adds four declarative knobs, all optional and backward
compatible (nothing is rendered when unset):

  otel.serviceName                              -> OTEL_SERVICE_NAME
  otel.resourceAttributes (map)                 -> OTEL_RESOURCE_ATTRIBUTES
  otel.tracing.exporter.otlp.headers (map)      -> OTEL_EXPORTER_OTLP_TRACES_HEADERS
  otel.logging.exporter.otlp.headers (map)      -> OTEL_EXPORTER_OTLP_LOGS_HEADERS

Map values are rendered as `key1=v1,key2=v2` with sorted keys for
deterministic output. Secret headers (Basic auth, API keys) should
continue to be overlaid through `controller.envFrom` referencing a
Secret; the values.yaml comments document this pattern.

Adds `tests/controller-configmap_test.yaml` covering:
- backward compat: no new keys rendered with default values
- each new field renders independently when set
- map fields serialize in sorted key order
- an end-to-end Langfuse Cloud style render

Signed-off-by: Nir Zeira <nirz@elementor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] helm/kagent: expose OTEL_EXPORTER_OTLP_HEADERS, OTEL_SERVICE_NAME, and OTEL_RESOURCE_ATTRIBUTES

1 participant