feat(collector): add gen_ai_normalizer processor for Traceloop telemetry (#3604)

Add the gen_ai_normalizer processor to the traces pipeline to convert
OpenLLMetry/Traceloop instrumentation spans (from the agent service)
into official OpenTelemetry GenAI semantic conventions (gen_ai.*
attributes). The processor uses the built-in "openllmetry" source
mapping which handles the Traceloop SDK's attribute format.

Bump collector-contrib from v0.151.0 to v0.155.0 which includes the
gen_ai_normalizer processor (added in v0.154.0, Alpha stability).

Closes #3526

Assisted-by: Claude Opus 4.8

Signed-off-by: Shenoy Pratik Gurudatt <4348487+ps48@users.noreply.github.com>
Co-authored-by: Shenoy Pratik Gurudatt <4348487+ps48@users.noreply.github.com>

Author: ps48

.env

@@ -8,7 +8,7 @@ OTEL_JAVA_AGENT_VERSION=2.28.1
 OPENTELEMETRY_CPP_VERSION=1.27.0
 
 # Dependent images
-COLLECTOR_CONTRIB_IMAGE=ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib:0.151.0
+COLLECTOR_CONTRIB_IMAGE=ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib:0.155.0
 FLAGD_IMAGE=ghcr.io/open-feature/flagd:v0.14.2
 GRAFANA_IMAGE=grafana/grafana:13.0.1
 JAEGERTRACING_IMAGE=quay.io/jaegertracing/jaeger:2.14.1

CHANGELOG.md

@@ -15,6 +15,11 @@ the release.
 * [accounting] Run the Kafka consumer as a hosted background service so process
   shutdown can stop the consumer cleanly
   ([#3608](https://github.com/open-telemetry/opentelemetry-demo/pull/3608))
+* [collector] Add `gen_ai_normalizer` processor to the traces pipeline to
+  convert OpenLLMetry/Traceloop instrumentation telemetry (from the agent
+  service) into official GenAI semantic conventions (`gen_ai.*` attributes).
+  Bump collector-contrib to v0.155.0 which includes the processor
+  ([#3526](https://github.com/open-telemetry/opentelemetry-demo/issues/3526))
 * [llm] Increase `llm` service memory limit from 50M to 100M to prevent a
   startup restart loop caused by the container exceeding its memory limit
   ([#2944](https://github.com/open-telemetry/opentelemetry-demo/issues/2944))

src/otel-collector/otelcol-config.yml

@@ -187,6 +187,10 @@ processors:
           # Upstream Issue: https://github.com/open-telemetry/opentelemetry-collector/issues/15221
           - set(attributes["otelcol_signal"], attributes["otelcol.signal"]) where attributes["otelcol.signal"] != nil and not IsMap(attributes["otelcol.signal"])
           - delete_key(attributes, "otelcol.signal") where attributes["otelcol_signal"] != nil
+  gen_ai_normalizer:
+    sources:
+      - name: openllmetry
+
   transform/sanitize_spans:
     error_mode: ignore
     trace_statements:
@@ -240,7 +244,7 @@ service:
   pipelines:
     traces:
       receivers: [otlp]
-      processors: [resourcedetection, memory_limiter, transform/sanitize_spans]
+      processors: [resourcedetection, memory_limiter, transform/sanitize_spans, gen_ai_normalizer]
       exporters: [debug, span_metrics]
     metrics:
       receivers: [docker_stats, http_check/frontend-proxy, host_metrics, nginx, otlp, redis, postgresql, prometheus/ad, span_metrics]