feat: add kafka metrics exporter (#3158)

* feat: add kafka metrics exporter

Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>

* chore: update lint

Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>

* chore: update CHANGELOG

Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>

* feat: switch to using kafkametricsreceiver

Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>

---------

Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>
Co-authored-by: Juliano Costa <julianocosta89@outlook.com>

Author: Red-GV

CHANGELOG.md

@@ -7,9 +7,6 @@ the release.
 
 ## Unreleased
 
-* [product-catalog] Enrich DB spans and metrics with `server.address` and `server.port`
-  attributes extracted from the DSN via `otelsql.AttributesFromDSN`
-  ([#3154](https://github.com/open-telemetry/opentelemetry-demo/pull/3154))
 * [accounting] fix memory leak with dbcontext
   ([#2876](https://github.com/open-telemetry/opentelemetry-demo/pull/2876))
 * [chore] Upgrade OTel Collector to v0.145.0 with :warning: breaking change:
@@ -34,6 +31,11 @@ the release.
   180m for fraud-detection to prevent large heap size that causes
   OOMKills with k8s.
   ([#3105](https://github.com/open-telemetry/opentelemetry-demo/pull/3105))
+* [product-catalog] Enrich DB spans and metrics with `server.address` and `server.port`
+  attributes extracted from the DSN via `otelsql.AttributesFromDSN`
+  ([#3154](https://github.com/open-telemetry/opentelemetry-demo/pull/3154))
+* [otelcollector] add kafkametricsreceiver
+  ([#3158](https://github.com/open-telemetry/opentelemetry-demo/pull/3158))
 * [load-generator] Wait for Roof Binoculars image to load in web tasks, and fix
   task failures due to missing `tracer` attribute
   ([#3171](https://github.com/open-telemetry/opentelemetry-demo/pull/3171))

src/otel-collector/otelcol-config.yml

@@ -22,6 +22,14 @@ receivers:
     endpoint: unix:///var/run/docker.sock
     # https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/44511
     api_version: "1.44"
+  kafkametrics:
+    scrapers:
+      - brokers
+      - topics
+      - consumers
+    brokers:
+      - ${KAFKA_ADDR}
+    collection_interval: 10s
   postgresql:
     endpoint: ${POSTGRES_HOST}:${POSTGRES_PORT}
     username: root
@@ -208,7 +216,7 @@ service:
       processors: [resourcedetection, memory_limiter, transform/sanitize_spans]
       exporters: [otlp_grpc/jaeger, debug, spanmetrics]
     metrics:
-      receivers: [docker_stats, httpcheck/frontend-proxy, hostmetrics, nginx, otlp, postgresql, redis, spanmetrics]
+      receivers: [docker_stats, httpcheck/frontend-proxy, hostmetrics, nginx, otlp, postgresql, redis, spanmetrics, kafkametrics]
       processors: [resourcedetection, memory_limiter]
       exporters: [otlp_http/prometheus, debug]
     logs: