feat(tools): add "webhook" toolset for outbound notifications

[dwin-gharibi]

Closes #3640

What this is

A toolset that lets an agent send a notification to a chat service, without knowing the URL or the secret.

The agent gets one tool - send_webhook(message). Where that message goes is decided by config, not by the model:

agents:
  root:
    model: openai/gpt-4o-mini
    description: Runs the nightly build and reports the result.
    instruction: Report the build outcome with send_webhook.
    toolsets:
      - type: webhook
        webhook_config:
          provider: slack
          url: ${env.SLACK_WEBHOOK_URL}

Agent: send_webhook(message: "nightly build passed ✅")
  →  POST https://hooks.slack.com/services/…   {"text": "nightly build passed ✅"}
  →  "Queued delivery to the slack webhook. You will only be notified if it ultimately fails."

The model wrote the message. It never saw the URL.

Why not just use the api or fetch tool?

You can already POST to Slack with api. The difference is what the model has to know, and what happens when the network misbehaves:

	api / fetch	webhook
Destination	model supplies the URL	fixed in config
Secret	in the model's context	never in the model's context
Payload shape	model must know Slack ≠ Discord ≠ Telegram	built from provider
429 / 5xx	model sees the error and improvises	retried with backoff, honours Retry-After
Duplicate sends	model may retry on its own	suppressed for 30s
Blocking	model waits for the HTTP round-trip	returns immediately

The bottom four rows are the substance. A model that gets an HTTP 429 will happily retry in a loop, or give up and report failure when nothing was wrong. Retry policy is not something a language model should be improvising.

How it works

A send_webhook call passes two cheap guards before any network I/O, then delivers - in the background when the runtime allows it:

1. Fire-and-forget

send_webhook returns as soon as the message is accepted - the agent isn't blocked on a network round-trip and doesn't stall its turn behind a slow Slack. Delivery continues on a background context (context.WithoutCancel) so it survives the tool call returning. The agent is interrupted only if delivery ultimately fails, via Recall; success is silent.

A notification tool that makes the agent babysit every send is worse than no tool. When the runtime doesn't support Recall, it falls back to delivering synchronously and returning the real result - the semantics degrade to "blocking but correct" rather than silently dropping failures.

2. Retries are classified, not blind

Every response is sorted into delivered / transient / permanent, and only transient failures are retried - up to 4 attempts, honouring the server's Retry-After:

Response	Verdict	Action
2xx	delivered	stop
429, 5xx	transient	retry - honour Retry-After, else exponential backoff
other 4xx	permanent	stop immediately, report
network error	transient	retry
bad URL / bad payload	permanent	stop, report

Retry-After is capped at backoff.MaxRetryAfterWait, so a hostile server can't park a delivery forever. Retrying a 401 forever is pointless; not retrying a 503 is careless - hence the split.

3. Duplicate suppression and rate limiting

An identical (message, value2, value3) within 30s is suppressed, and sends are spaced at least 1s apart (the two amber guards in the flow above). Models retry on their own when they think something failed; without a guard, one flaky send becomes ten Slack messages.

Providers

provider selects the payload shape. Same send_webhook(message) call, different body:

provider	Body
slack, mattermost, rocketchat, googlechat, teams	{"text": "..."}
discord	{"content": "..."}
telegram	{"chat_id": "...", "text": "..."}
ifttt	{"value1": "...", "value2": "...", "value3": "..."}
generic (default)	{"text": "..."}

value2 / value3 are optional and exist for IFTTT's three-value contract; other providers ignore them.

More examples

Discord:

toolsets:
  - type: webhook
    webhook_config:
      provider: discord
      url: ${env.DISCORD_WEBHOOK_URL}

Telegram - needs a destination chat as well as the token URL:

toolsets:
  - type: webhook
    webhook_config:
      provider: telegram
      url: https://api.telegram.org/bot${env.TELEGRAM_BOT_TOKEN}/sendMessage
      chat_id: ${env.TELEGRAM_CHAT_ID}

A generic endpoint that authenticates with a header instead of a secret URL:

toolsets:
  - type: webhook
    webhook_config:
      provider: generic
      url: https://alerts.internal.example.com/hook
      headers:
        Authorization: Bearer ${env.ALERTS_TOKEN}

url and headers expand through the standard ${env.VAR} expander - the same one api and open_url use - so secrets stay in the environment rather than in the config file or the model's context.

Notes for review

• The destination is deliberately not a tool parameter. Making it one would put the secret in the model's context and let a prompt injection retarget the notification. (This is the change from the first revision of this PR.)
• webhook_config is required for type: webhook, enforced by a dedicated anyOf branch in the schema, so a misconfigured toolset fails at parse time rather than at send time.
• Response bodies are read through a 64 KiB LimitReader before being echoed into an error, so a misbehaving endpoint can't flood the transcript.

Verification

• go test ./pkg/tools/builtin/webhook/ ./pkg/teamloader/toolsets/ → passing
• golangci-lint run (repo's own .golangci.yml) → 0 issues
• Schema drift guards (TestSchemaMatchesGoTypes, TestJsonSchemaWorksForExamples, TestDocYAMLSnippetsAreValid) → passing
• markdownlint-cli2 from docs/ → 0 errors

[dwin-gharibi]

@Sayt-0 Problems solved.

[Sayt-0]

New webhook toolset fills a real gap (agents currently have no way to notify anyone) with a well-chosen scope: message-shaped payloads only, SSRF-safe client, pure payload builder with an injectable HTTP doer. Registration, catalog entry, docs layout and tests follow the existing patterns.

Two blocking items, both cheap to fix:

#	Issue	Impact
1	Docs reference the scheduler toolset, which does not exist on main (#3632 is still open): overview prose, example config, closing tip	Copy-pasting the example fails at load time with unknown toolset type: scheduler (pkg/teamloader/registry.go)
2	agent-schema.json toolset type enum not updated	Editor validation (yaml-language-server, recommended in docs/configuration/overview) flags type: webhook as invalid. Precedent: the rag toolset change updated the schema in the same PR

Item 1: inline suggestions below remove the references. If #3632 is expected to merge first, rebasing on top of it works too; as long as merge order is undefined, published docs should not reference a toolset that fails to load.

Item 2: add "webhook" in two places:

• definitions/Toolset/properties/type/enum
• definitions/Toolset/anyOf[1]/properties/type/enum (variant listing types with no extra required field; no dedicated anyOf entry is needed since the toolset takes no options)

Non-blocking inline suggestions: reuse httpclient.DefaultToolHTTPTimeout, set the docker-agent User-Agent on outbound requests, list all providers in the tool description, consider honoring timeout: / allowed_domains: from the toolset config.

Context note: the Go workflow (ci.yml) currently fails repo-wide in 0s ("workflow file issue", setup-go with: indentation, also failing on main pushes), so task lint / task test did not run for this PR. Pre-existing breakage, not caused by this change, but Go tests were not executed by CI.

[rumpl]

I like the idea but as is this toolset is just an http call, and we already have a toolset for this. Let's try and figure out how this toolset can really be a webhook

[aheritier]

👋 This PR has merge conflicts with the base branch. Please rebase or merge the latest base branch and resolve them. I've moved it to draft and added status/needs-rebase; it'll be picked back up automatically once the conflicts are cleared.

[dwin-gharibi]

I like the idea but as is this toolset is just an http call, and we already have a toolset for this. Let's try and figure out how this toolset can really be a webhook

Yes, you’re right @rumpl. I’m currently working on the architecture to make this something more flexible that we can build on and extend, rather than just another HTTP call wrapper.

I’m also thinking about capabilities like outbound webhooks and how we can make the toolset a more proper webhook-based integration point with better extensibility.

I’ll push the new implementation soon so we can review the direction and iterate on it.

[dwin-gharibi]

@rumpl @Sayt-0 Thank you both - the review changed my mind on the design, so this is a rework rather than a patch. Heads-up: the tool's interface changed (send_webhook no longer takes a url).

@rumpl - "the agent can't know the URL" / "we need at least some config"

You were right, and it is true for every provider, not just Slack: the webhook URL is the credential (Slack and Mattermost embed a secret path, Discord a token, IFTTT a key, Telegram a bot token). Having the model supply it only ever worked if a user pasted a secret into the prompt.

The destination is now deployment configuration, following the api / open_url precedent (${env.VAR}, expanded at call time, never stored):

toolsets:
  - type: webhook
    webhook_config:
      provider: slack
      url: ${env.SLACK_WEBHOOK_URL}

Token-based endpoints are covered by headers (expanded the same way), and Telegram takes its chat_id:

      provider: telegram
      url: https://api.telegram.org/bot${env.TELEGRAM_BOT_TOKEN}/sendMessage
      chat_id: "123456789"

      provider: generic
      url: https://alerts.example.com/notify
      headers:
        Authorization: Bearer ${env.ALERTS_TOKEN}

send_webhook(message) is all the model sees; the URL is never returned to it, nor echoed into error messages. Added WebhookToolConfig to latest.Toolset plus the matching agent-schema.json definition, property and anyOf variant requiring webhook_config.

@rumpl - "as is this is just an http call, we already have a toolset for this"

Also correct, and the sharpest point: once the URL moves into config, what was left was api_config plus a payload lookup table. So the toolset now owns delivery, not the request. Verified against api.go: it fires exactly one request and contains no retry, backoff, 429 or Retry-After handling.

1. At-least-once delivery - transient failures (429, 5xx, network) are retried with exponential backoff via pkg/backoff, honouring Retry-After (capped by backoff.MaxRetryAfterWait so a misbehaving server can't stall the agent). 4xx is classified permanent and fails immediately instead of burning retries.
2. Non-blocking + failure callback - the structural difference. api is request/response: it cannot return before the call completes, nor tell you anything afterwards. send_webhook returns once queued, delivers in the background, and wakes the agent via Runtime.Recall only if delivery ultimately fails. When the host has no recall support it degrades to inline delivery so a verdict is never lost.
3. Storm protection - identical message to the same destination within a window is suppressed, and deliveries are rate limited, so a looping agent can't flood a channel or burn the provider's rate limit. api has no memory between calls.
4. Provider semantics - payload shaping per service, and the result normalised to delivered / transient / permanent rather than handing the model a raw body to interpret.

I also considered making it a real inbound webhook (receive events → wake the agent), which is the truest reading of "webhook" and the one gap api and the http_post hook don't cover. I dropped it: agents run on a user's machine with no public endpoint, so there is nothing for a provider to call back into.

There is a deliberate behaviour trade-off worth flagging: delivery is fire-and-forget, so on success the agent gets no confirmation - only failures call back. That is correct webhook semantics, but happy to make async opt-in if you'd rather the agent always receive a verdict.

@Sayt-0 - your points

1. Docs referencing scheduler (not on main): removed from the overview, the example and the closing tip.
2. agent-schema.json enum: added - and since webhook now requires webhook_config, it moved out of the "no extra required field" anyOf variant into its own, mirroring api.
3. httpclient.DefaultToolHTTPTimeout: reused (local requestTimeout dropped).
4. useragent.SetIdentity(req): added, as fetch/api do.
5. Tool description: now lists all nine providers.
6. timeout / allowed_domains: timeout: on the toolset is now honoured. allowed_domains / blocked_domains are not - with the destination fixed in config the model can no longer choose where data goes, so the exfiltration surface that motivated it is closed by construction. Happy to add them anyway if you still want them; I've written up the reasoning either way.

CI note

Since the Go jobs still don't run, I installed golangci-lint v2.12.2 and ran the repo's .golangci.yml locally - it surfaced four real blockers that were invisible here (3× forbidigo context.Background() in tests, 1× perfsprint fmt.Errorf without verbs), all fixed. Current state:

• golangci-lint run ./pkg/tools/builtin/webhook/... ./pkg/teamloader/toolsets/... ./pkg/config/latest/... → 0 issues
• go test ./pkg/tools/builtin/webhook/ ./pkg/teamloader/toolsets/ ./pkg/config/ → passing, and -race clean (delivery runs on a goroutine)
• markdownlint-cli2@0.22.1 from docs/ → 0 errors

Do let me know if any change is required please.

[dwin-gharibi]

Why webhook is not "just an HTTP call"

Answering the review: "as is this toolset is just an http call, and we already have a toolset for this. Let's try and figure out how this toolset can really be a webhook."

That was correct about the original design. The toolset has been reworked so the thing it owns is delivery, not the request.

The reframing

	api toolset	webhook toolset
Question it answers	"Call this endpoint and give me the response."	"Make sure this notification arrives."
Model supplies	request args	a message
Execution	one request, synchronous	queued, retried, out-of-band
Failure	returns an error, agent must cope	retried; agent is called back only if it ultimately fails

What api structurally cannot do

Verified against pkg/tools/builtin/api/api.go: it fires exactly one request and returns the body - it contains no retry, backoff, 429 or Retry-After handling.

1. At-least-once delivery

Transient failures (429, 5xx, network errors) are retried with exponential backoff (pkg/backoff), honouring the server's Retry-After (capped by backoff.MaxRetryAfterWait, so a misbehaving server can't stall the agent). A 4xx is classified permanent and fails immediately instead of burning retries.

With api, a transient blip means the notification is simply lost unless the model happens to notice and retry by hand - costing turns and doing it inconsistently.

2. Non-blocking delivery + failure callback - the structural difference

api is request/response: it cannot return before the call completes, and it cannot tell you anything afterwards. send_webhook returns as soon as the message is queued and delivers in the background, so a slow or retrying endpoint never stalls a turn. If delivery ultimately fails, the agent is woken via Runtime.Recall - the same primitive background_jobs uses.

This is not expressible in api_config at any setting. It is a different execution model, and it is what "webhook" means operationally: hand it over, it gets delivered, you hear about it only if it doesn't.

3. Storm protection

Agents loop. An identical message to the same destination within a short window is suppressed, and deliveries are rate limited, so a retry loop cannot flood a channel or burn the provider's rate limit. api has no memory between calls and no throttle.

4. Provider semantics

Each service's wire format is applied for the agent (Slack/Mattermost/Rocket.Chat/Google Chat/Teams {"text"}, Discord {"content"}, IFTTT {"value1..3"}, Telegram {"chat_id","text"}), and the result is normalised to delivered / transient / permanent. With api the user hand-writes each payload template and the model has to guess from a raw body whether it worked.

5. Credential shape

The destination is deployment config, not a model choice: every provider's webhook URL is the credential (Slack/Mattermost a secret path, Discord a token, IFTTT a key, Telegram a bot token). The model supplies only the message, never sees the URL, and the URL is never echoed into results or error messages. api would work here, but it leaves the endpoint choice in the tool surface.

Summary

api = "make an HTTP request." webhook = "deliver this notification reliably to a destination I configured." The overlap is the byte on the wire; everything that matters - retries, backoff, Retry-After, dedupe, rate limiting, async hand-off, failure callback, provider payloads - lives outside what api_config can express.

Every one of these is covered by unit tests with a fake HTTP doer, a fake runtime, and an injected clock - no network, no sleeping.

[rumpl]

@dwin-gharibi can you please fix your git history, rebase with main instead of merge commits.

And also, can you please give us a human readable explanation of what this is, how it works, with examples in the pr description, there's a lot of text here but not a lot of info

[dwin-gharibi]

@dwin-gharibi can you please fix your git history, rebase with main instead of merge commits.

And also, can you please give us a human readable explanation of what this is, how it works, with examples in the pr description, there's a lot of text here but not a lot of info

Done. Rebased and also I've provided new PR desc with some amazing diagrams and better flow. Would you please check it out and tell me if we still have any problems again @rumpl @Sayt-0