[receiver/sqlquery] Support static attributes, timestamp, and event name in log collection

[atom-mfeldkamp]

Component(s)

receiver/sqlquery

Is your feature request related to a problem? Please describe.

I'd like to additional support for logs in the sqlquery receiver that allows me to statically assign attributes to logs (similar to metrics), assign the log timestamp from a database column (similar to metrics), and set the event name on a LogRecord based on the contents of a database column. The absence of the ability to set the timestamp and event name in particular are preventing reporting of particular details in our log system. The static attributes are a really nice quality of life feature on metrics in this receiver that would be great to mirror in logs.

Describe the solution you'd like

I'd like to add configuration items static_attributes, ts_column, and event_name_column configuration in the logs portion of the sqlquery receiver.

Example config:

receivers:
  sqlquery:
    driver: postgres
    datasource: "host=localhost port=5432 user=postgres password=s3cr3t sslmode=disable"
    storage: file_storage
    queries:
      - sql: "select * from my_logs where log_id > $$1"
        tracking_start_value: "10000"
        tracking_column: log_id
        logs:
          - body_column: log_body
            attribute_columns: ["log_attribute_1", "log_attribute_2"]
            static_attributes:
              dbinstance: mydbinstance
            ts_column: log_ts
            event_name_column: log_event_name

I would like static_attributes and ts_column configuration provided in this way to work exactly the same way as their companion settings in the metrics section: setting log attributes statically, and setting the log timestamp from the provided column. I'd imagine leaving how the observed timestamp is currently set would capture the difference in actual log creation time vs log scrape time.

I would like event_name_column to set the LogRecord's event name based on the content of what's in the row at that column. Additionally, I'd like to relax the requirement that body_column be set when event_name_column is set: an empty string body is apparently fine on an event (even expected, if I correctly understand event semantics).

Describe alternatives you've considered

For static attributes, I can just inject static values into my SQL query and get those with attribute_columns. Awkward, but doable.

I don't think there's any way to get timestamps or event names into the logs produced from the sqlquery receiver. I suppose I could add them to attributes on the LogRecord, then use a transform processor to re-shape the log how I want. That's a bit tedious.

Additional context

I might look into implementing this myself if there's no issues with the request. Seems pretty simple to implement.

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[github-actions]

Pinging code owners:

• receiver/sqlquery: @dmitryax @crobert-1

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[me-saurabhkohli]

Hi @atom-mfeldkamp , I'm a first time contributor to this repo and would like to contribute on this ask. Even though the scope looks clear and I agree with the approach to mirror static_attributes and ts_column from MetricCfg into LogsCfg, plus adding event_name_column.

Before I start, a few questions for maintainers:

• Timestamp parsing format: The current metrics ts_column parses the column value as a nanosecond-epoch int64 (strconv.ParseInt). Should the logs ts_column follow the same approach, or should it also attempt RFC3339 parsing (since the row scanner already formats time.Time values as RFC3339Nano strings)?

• event_name_column API: Should this use plog.LogRecord.SetEventName() (the dedicated OTLP 1.5 field), or set the event.name semconv attribute? I'm leaning toward SetEventName() since it's the modern approach.

• ts_column format mismatch: The row scanner converts native DB time.Time values to RFC3339Nano strings, but rowToMetric parses ts_column as a nanosecond-epoch int64 (strconv.ParseInt). This means ts_column only works with integer columns today, not native timestamp columns. For the logs implementation, should I add RFC3339 fallback parsing (try int64 first, then time.Parse)? This could later be back-ported to metrics too.

• PR scope: Would you prefer one PR covering all three features, or smaller separate PRs (e.g., static_attributes + ts_column parity in one, event_name_column + body_column validation relaxation in another)?

I see this is still labeled needs triage, I would be happy to wait for a green light before putting up a PR.

[atom-mfeldkamp]

@me-saurabhkohli Thanks for the interest!

• Timestamp parsing: I personally dislike having to provide nanos on the metrics side of things, so I really like your suggestion of attempting RFC3339 parsing so I don't have to do goofy epoch extraction / conversion in my SQL queries when I already have a timestamp column. I think falling back to ParseInt when it's clearly an integer also makes sense. It's a big of a bigger scope than what I originally suggested, but if you want to do it, I'm completely for it!
• event_name_column: Definitely agree on using SetEventName()! The dedicated OTLP 1.5 field was what I had in mind as well.
• ts_column behavior in metrics: I love the idea of back-porting any work you do to support parsing a timestamp column to metrics.
• PR scope: I'm happy with however you want to split it. I imagine none of these features are that big, so one PR could work, or two/three could also work. I'd guess back-porting any timestamp enhancements to metrics probably merits its own PR.

Let me know if you want a sample SQL query / sqlquery config block where I have to extract an epoch from a timestamp column on metrics so you can see what I'm talking about. Can also probably provide the table schema for that sort of thing.

Otherwise... thanks! I'm definitely all for it.

[me-saurabhkohli]

Awesome, thanks for the quick reply @atom-mfeldkamp and for being on board with the RFC3339 idea.

I'll keep it as a single PR since none of these are huge on their own. Here's roughly what I'm thinking:

• Add static_attributes, ts_column, and event_name_column to the logs config
• For the timestamp piece, I'll pull out a small shared parseTimestamp() helper in internal/sqlquery/ that tries RFC3339Nano first and falls back to ParseInt for epoch nanos. This way it stays backward-compatible but native DB timestamp columns should just work without any SQL gymnastics
• I'll wire that same helper into the existing metrics ts_column / start_ts_column too, as the change is small and shared so I believe one PR would help adding test coverage as well, I hope you’re ok with that
• For event name, I'll go with LogRecord.SetEventName() as we discussed
• And I'll relax the body_column validation so it's only required when event_name_column isn't set

Please share the SQL config block and if possible the table schema as you mentioned. It would be super helpful to see the epoch flow. I can use it for test coverage and context in the PR as well.

I'll start working on this and will have a PR up soon! Thanks again and appreciate your feedback !!