Skip to content

Commit 6004654

Browse files
authored
[extension/pebbletailstorage] Promote pebble_tail_storage extension to alpha stability (#48877)
#### Description Promote pebble tail storage extension to alpha stability <!-- Issue number (e.g. #1234) or full URL to issue, if applicable. --> #### Link to tracking issue Fixes #47916 Fixes #42326 <!--Describe what testing was performed and which tests were added.--> #### Testing <!--Describe the documentation added.--> #### Documentation <!--Authorship attestation. See AGENTS.md for details. AI agents must not check this box on behalf of the user; the human author must check it themselves before the PR is ready for review.--> #### Authorship - [x] I, a human, wrote this pull request description myself. <!--Please delete paragraphs that you did not use before submitting.-->
1 parent e97e96b commit 6004654

10 files changed

Lines changed: 72 additions & 23 deletions

File tree

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Use this changelog template to create an entry for release notes.
2+
3+
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
4+
change_type: enhancement
5+
6+
# The name of the component, or a single word describing the area of concern, (e.g. receiver/filelog)
7+
component: extension/pebble_tail_storage
8+
9+
# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
10+
note: Promote `pebble_tail_storage` extension to alpha stability
11+
12+
# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
13+
issues: [47916]
14+
15+
# (Optional) One or more lines of additional information to render under the primary note.
16+
# These lines will be padded with 2 spaces and then inserted directly into the document.
17+
# Use pipe (|) for multiline entries.
18+
subtext:
19+
20+
# If your change doesn't affect end users or the exported elements of any package,
21+
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
22+
# Optional: The change log or logs in which this entry should be included.
23+
# e.g. '[user]' or '[user, api]'
24+
# Include 'user' if the change is relevant to end users.
25+
# Include 'api' if there is a change to a library API.
26+
# Default: '[user]'
27+
change_logs: [user]

cmd/otelcontribcol/builder-config.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ extensions:
4545
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/extension/storage/dbstorage v0.153.0
4646
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/extension/storage/filestorage v0.153.0
4747
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/extension/storage/redisstorageextension v0.153.0
48+
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/extension/tailstorage/pebbletailstorageextension v0.153.0
4849
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/extension/sumologicextension v0.153.0
4950
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/extension/encoding/awscloudwatchmetricstreamsencodingextension v0.153.0
5051
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/extension/encoding/awslogsencodingextension v0.153.0

extension/tailstorage/pebbletailstorageextension/README.md

Lines changed: 28 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -5,13 +5,14 @@ Stores pending trace data for tail sampling in a local Pebble database.
55

66
| Status | |
77
| ------------- |-----------|
8-
| Stability | [development] |
9-
| Distributions | [] |
8+
| Stability | [alpha] |
9+
| Distributions | [contrib] |
1010
| Issues | [![Open issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aopen%20label%3Aextension%2Fpebbletailstorage%20&label=open&color=orange&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aopen+is%3Aissue+label%3Aextension%2Fpebbletailstorage) [![Closed issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aclosed%20label%3Aextension%2Fpebbletailstorage%20&label=closed&color=blue&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aclosed+is%3Aissue+label%3Aextension%2Fpebbletailstorage) |
1111
| Code coverage | [![codecov](https://codecov.io/github/open-telemetry/opentelemetry-collector-contrib/graph/main/badge.svg?component=extension_pebbletailstorage)](https://app.codecov.io/gh/open-telemetry/opentelemetry-collector-contrib/tree/main/?components%5B0%5D=extension_pebbletailstorage&displayType=list) |
1212
| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner) | [@carsonip](https://www.github.com/carsonip), [@jmacd](https://www.github.com/jmacd), [@axw](https://www.github.com/axw) |
1313

14-
[development]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/component-stability.md#development
14+
[alpha]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/component-stability.md#alpha
15+
[contrib]: https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-contrib
1516
<!-- end autogenerated section -->
1617

1718
# Pebble Tail Storage Extension
@@ -22,10 +23,6 @@ Tail Sampling processor.
2223
This extension is intended to be used with the Tail Sampling processor `tail_storage`
2324
setting and is useful when in-memory pending-trace state would otherwise be too large.
2425

25-
## Status
26-
27-
This extension is in development and may change.
28-
2926
## Limitations
3027

3128
Persistence across collector restarts is **not yet supported**. The extension
@@ -39,17 +36,40 @@ treat the directory as ephemeral.
3936

4037
## Example
4138

39+
The following example uses the `span-ingest` sampling strategy with a decision
40+
cache and Pebble tail storage. The `span-ingest` strategy evaluates each
41+
incoming span batch immediately on ingest, which avoids accumulating complete
42+
traces in memory before making a decision. Combined with `tail_storage`, spans
43+
for traces that are still pending a decision are written to disk rather than
44+
held in memory, keeping collector memory usage bounded under high load.
45+
46+
The `decision_cache` keeps a record of already-decided trace IDs so that late
47+
spans for sampled or dropped traces are handled correctly without re-evaluating
48+
policies.
49+
4250
```yaml
4351
extensions:
4452
pebble_tail_storage:
4553
directory: /var/lib/otelcol/pebble-tail-storage
4654

55+
receivers:
56+
otlp:
57+
protocols:
58+
grpc:
59+
60+
exporters:
61+
debug:
62+
4763
processors:
4864
tail_sampling:
49-
decision_wait: 30s
65+
sampling_strategy: span-ingest
66+
decision_wait: 2m
5067
num_traces: 50000
5168
expected_new_traces_per_sec: 1000
5269
tail_storage: pebble_tail_storage
70+
decision_cache:
71+
sampled_cache_size: 100000
72+
non_sampled_cache_size: 100000
5373
policies:
5474
- name: errors
5575
type: status_code

extension/tailstorage/pebbletailstorageextension/go.mod

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,13 +22,13 @@ require (
2222
github.com/beorn7/perks v1.0.1 // indirect
2323
github.com/cespare/xxhash/v2 v2.3.0 // indirect
2424
github.com/cockroachdb/crlib v0.0.0-20241112164430-1264a2edc35b // indirect
25-
github.com/cockroachdb/errors v1.11.3 // indirect
25+
github.com/cockroachdb/errors v1.13.0 // indirect
2626
github.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b // indirect
2727
github.com/cockroachdb/redact v1.1.5 // indirect
2828
github.com/cockroachdb/swiss v0.0.0-20251224182025-b0f6560f979b // indirect
2929
github.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06 // indirect
3030
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
31-
github.com/getsentry/sentry-go v0.27.0 // indirect
31+
github.com/getsentry/sentry-go v0.46.0 // indirect
3232
github.com/go-logr/logr v1.4.3 // indirect
3333
github.com/go-logr/stdr v1.2.2 // indirect
3434
github.com/go-viper/mapstructure/v2 v2.5.0 // indirect

extension/tailstorage/pebbletailstorageextension/go.sum

Lines changed: 4 additions & 4 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

extension/tailstorage/pebbletailstorageextension/integrationtest/go.mod

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ require (
3838
github.com/cenkalti/backoff/v5 v5.0.3 // indirect
3939
github.com/cespare/xxhash/v2 v2.3.0 // indirect
4040
github.com/cockroachdb/crlib v0.0.0-20241112164430-1264a2edc35b // indirect
41-
github.com/cockroachdb/errors v1.11.3 // indirect
41+
github.com/cockroachdb/errors v1.13.0 // indirect
4242
github.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b // indirect
4343
github.com/cockroachdb/pebble/v2 v2.1.6 // indirect
4444
github.com/cockroachdb/redact v1.1.5 // indirect
@@ -51,7 +51,7 @@ require (
5151
github.com/felixge/httpsnoop v1.0.4 // indirect
5252
github.com/foxboron/go-tpm-keyfiles v0.0.0-20251226215517-609e4778396f // indirect
5353
github.com/fsnotify/fsnotify v1.10.1 // indirect
54-
github.com/getsentry/sentry-go v0.27.0 // indirect
54+
github.com/getsentry/sentry-go v0.46.0 // indirect
5555
github.com/go-logr/logr v1.4.3 // indirect
5656
github.com/go-logr/stdr v1.2.2 // indirect
5757
github.com/go-ole/go-ole v1.2.6 // indirect

extension/tailstorage/pebbletailstorageextension/integrationtest/go.sum

Lines changed: 4 additions & 4 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

extension/tailstorage/pebbletailstorageextension/internal/metadata/generated_status.go

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

extension/tailstorage/pebbletailstorageextension/metadata.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ description: Stores pending trace data for tail sampling in a local Pebble datab
66
status:
77
class: extension
88
stability:
9-
development: [extension]
10-
distributions: []
9+
alpha: [extension]
10+
distributions: [contrib]
1111
codeowners:
1212
active: [carsonip, jmacd, axw]

reports/distributions/contrib.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,7 @@ components:
9393
- oidc
9494
- opamp
9595
- otlp_encoding
96+
- pebble_tail_storage
9697
- pprof
9798
- redis_storage
9899
- sigv4auth

0 commit comments

Comments
 (0)