feat: add Cascadia provider for distributed on-prem Intel AI PC inference (#1147)

## Description

Adds `cascadia` as a provider (closes #1146). Cascadia
(https://cascadia.to) is a
distributed inference runtime that shards open-weights models (INT4
OpenVINO) across
fleets of Intel AI PCs behind an OpenAI-compatible coordinator, keeping
inference
on-premises.

**How**

- `BaseOpenAIProvider` subclass following the vLLM keyless pattern,
extended to honor
`CASCADIA_API_KEY` from the environment when a fleet is keyed (an
explicit `api_key`
wins; otherwise it falls back to `no-key-required` so trusted-LAN
coordinators work
  keyless). Env-configurable base via `CASCADIA_API_BASE` (default
`http://localhost:9090/v1`); no new dependencies (`cascadia = []` extra,
spliced
  into `all`).
- Capability flags scoped to v1: completion, streaming, reasoning, and
list-models
  on; responses, embeddings, moderation, image, PDF, and batch off.
- Registered in `LLMProvider` (`src/any_llm/constants.py`), the
`tests/conftest.py`
  fixture maps, and `LOCAL_PROVIDERS` + `CI_EXCLUDED_PROVIDERS` in
`tests/constants.py` (self-hosted backend; CI cannot reach a fleet,
mirroring vllm).

Files: `src/any_llm/providers/cascadia/{cascadia,__init__}.py`,
`src/any_llm/constants.py`, `pyproject.toml`, `tests/constants.py`,
`tests/conftest.py`, and
`tests/unit/providers/test_cascadia_provider.py`.

**Testing**

- `tests/unit/providers/test_cascadia_provider.py`: provider metadata,
keyless auth,
explicit-key passthrough, `CASCADIA_API_KEY` env resolution, env/default
base
resolution, capability flags, and enum/loader registration. 8 passed,
100%
  line+branch coverage of `cascadia.py`.
- Full local unit suite green (1419 passed, 64 skipped), no regressions.
Developed
against current `main` (latest release `any-llm-sdk` 1.18.0); rebased on
the default
  branch, where no `cascadia` provider previously existed.
- `pre-commit run` on the change set: ruff, ruff-format, mypy (strict),
and codespell
  all pass.
- Manual smoke via `acompletion("cascadia:<model>", ...)` against a
Cascadia
coordinator implementing the OpenAI-compatible contract: non-streaming
and streaming
both return content and a populated `usage` object. Production Cascadia
runs the same
  contract across a physical Intel AI PC fleet.

<details><summary>Smoke transcript (real coordinator)</summary>

```
$ export CASCADIA_API_BASE=https://<coordinator>/api/v1
$ export CASCADIA_API_KEY=<bearer token>     # keyed fleet; trusted-LAN fleets run keyless
$ python smoke_cascadia.py

models: ['llama-8b-2stage', 'qwen3-8b']
using: cascadia:qwen3-8b

# non-streaming: content + usage object returned
non-stream: '<think> [reasoning trimmed] </think>\n\nCASCADIA-OK'
            usage: CompletionUsage(completion_tokens=1, prompt_tokens=0, total_tokens=1)

# streaming: reassembled deltas
stream:     '<think> [reasoning trimmed] </think>\n\nOne, two, three, four, five.'
```

qwen3-8b is a reasoning model, so it emits a `<think>` block before the
answer; that
text is trimmed here for readability. Token counts on this hosted demo
coordinator are
approximate; the provider faithfully surfaces whatever the coordinator
reports in
`usage`.
</details>

## PR Type

- 🆕 New Feature

## Relevant issues

Closes #1146

## Checklist

- [x] I understand the code I am submitting.
- [x] I have added unit tests that prove my fix/feature works
- [x] I have run this code locally and verified it fixes the issue.
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the contribution guidelines
- [x] **AI Usage:**
    - [ ] No AI was used.
    - [x] AI was used for drafting/refactoring.
    - [ ] This is fully AI-generated.

## AI Usage Information

- AI Model used: Claude Opus 4.8
- AI Developer Tool used: Claude Code
- Any other info you'd like to share: The provider, tests, and docs were
drafted with
AI assistance under my direction and review. I verified the change
locally (full unit
suite, ruff/mypy-strict/codespell, and a live smoke against a real
Cascadia
  coordinator) and will respond to review comments myself.

- [ ] I am an AI Agent filling out this form (check box if true)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added Cascadia as a new supported language model provider with an
OpenAI-compatible coordinator endpoint.
* Enabled completion with streaming and reasoning-style capabilities,
plus model listing.
* Added flexible API key handling using `CASCADIA_API_KEY` (with safe
non-failing fallback when unset).
* **Optional Dependencies**
* Introduced a new `cascadia` extras option (and included it in `all`).
* **Tests**
* Added unit test coverage for Cascadia provider registration,
configuration, capability flags, and API-base/API-key precedence.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Tate Berenbaum <t8@users.noreply.github.com>

Author: t8

pyproject.toml

@@ -21,7 +21,7 @@ dependencies = [
 [project.optional-dependencies]
 
 all = [
-  "any-llm-sdk[mistral,anthropic,huggingface,gemini,vertexai,vertexaianthropic,cohere,cerebras,fireworks,groq,bedrock,azure,azureanthropic,azureopenai,watsonx,together,sambanova,ollama,moonshot,neosantara,nebius,xai,databricks,deepseek,inception,openai,otari,openrouter,portkey,qiniu,requesty,lmstudio,llama,voyage,perplexity,platform,llamafile,llamacpp,sagemaker,github,zai,minimax,mzai,vllm,dashscope,deepinfra]"
+  "any-llm-sdk[mistral,anthropic,huggingface,gemini,vertexai,vertexaianthropic,cohere,cerebras,fireworks,groq,bedrock,azure,azureanthropic,azureopenai,cascadia,watsonx,together,sambanova,ollama,moonshot,neosantara,nebius,xai,databricks,deepseek,inception,openai,otari,openrouter,portkey,qiniu,requesty,lmstudio,llama,voyage,perplexity,platform,llamafile,llamacpp,sagemaker,github,zai,minimax,mzai,vllm,dashscope,deepinfra]"
 ]
 
 platform = [
@@ -110,6 +110,7 @@ lmstudio = [
 
 # These providers don't require any additional dependencies, but are included for completeness.
 azureopenai = []
+cascadia = []
 databricks = []
 deepseek = []
 fireworks = []

src/any_llm/constants.py

@@ -21,6 +21,7 @@ class LLMProvider(StrEnum):
     AZURE = "azure"
     AZUREANTHROPIC = "azureanthropic"
     AZUREOPENAI = "azureopenai"
+    CASCADIA = "cascadia"
     CEREBRAS = "cerebras"
     COHERE = "cohere"
     DATABRICKS = "databricks"

src/any_llm/providers/cascadia/__init__.py

@@ -0,0 +1,3 @@
+from any_llm.providers.cascadia.cascadia import CascadiaProvider
+
+__all__ = ["CascadiaProvider"]

src/any_llm/providers/cascadia/cascadia.py

@@ -0,0 +1,42 @@
+import os
+
+from typing_extensions import override
+
+from any_llm.providers.openai.base import BaseOpenAIProvider
+
+
+class CascadiaProvider(BaseOpenAIProvider):
+    """Provider for Cascadia distributed on-prem inference clusters.
+
+    Cascadia (https://cascadia.to) is a distributed LLM inference runtime that
+    splits open-weights transformer models into INT4 OpenVINO shards and runs
+    them across fleets of Intel AI PCs, keeping inference fully on-premises.
+    A Cascadia coordinator exposes an OpenAI-compatible API; this provider
+    targets that endpoint.
+    """
+
+    API_BASE = "http://localhost:9090/v1"
+    ENV_API_KEY_NAME = "CASCADIA_API_KEY"
+    ENV_API_BASE_NAME = "CASCADIA_API_BASE"
+    PROVIDER_NAME = "cascadia"
+    PROVIDER_DOCUMENTATION_URL = "https://cascadia.to"
+
+    SUPPORTS_COMPLETION = True
+    SUPPORTS_COMPLETION_STREAMING = True
+    SUPPORTS_COMPLETION_REASONING = True
+    SUPPORTS_COMPLETION_IMAGE = False
+    SUPPORTS_COMPLETION_PDF = False
+    SUPPORTS_EMBEDDING = False
+    SUPPORTS_MODERATION = False
+    SUPPORTS_LIST_MODELS = True
+    SUPPORTS_BATCH = False
+    SUPPORTS_IMAGE_GENERATION = False
+    SUPPORTS_RERANK = False
+    SUPPORTS_RESPONSES = False
+
+    @override
+    def _verify_and_set_api_key(self, api_key: str | None = None) -> str:
+        # Honor an explicit key or CASCADIA_API_KEY from the environment. Cascadia
+        # coordinators on trusted LANs may run keyless, so fall back to a placeholder
+        # instead of hard-failing when no key is set (vLLM convention).
+        return api_key or os.getenv(self.ENV_API_KEY_NAME) or "no-key-required"

tests/conftest.py

@@ -52,6 +52,7 @@ def provider_reasoning_model_map() -> dict[LLMProvider, str]:
         LLMProvider.LLAMAFILE: "N/A",
         LLMProvider.LLAMACPP: "N/A",
         LLMProvider.VLLM: "N/A",
+        LLMProvider.CASCADIA: "N/A",
         LLMProvider.LMSTUDIO: "qwen3-0.6b",
         LLMProvider.AZUREOPENAI: "gpt-4.1-nano",
         LLMProvider.CEREBRAS: "gpt-oss-120b",
@@ -97,6 +98,7 @@ def provider_model_map() -> dict[LLMProvider, str]:
         LLMProvider.LLAMAFILE: "N/A",
         LLMProvider.LMSTUDIO: "qwen/qwen3-1.7b",  # small model keeps the LM Studio cache under the 10GB Actions limit; you must have LM Studio running and the server enabled
         LLMProvider.VLLM: "Qwen/Qwen2.5-0.5B-Instruct",
+        LLMProvider.CASCADIA: "Qwen/Qwen2.5-0.5B-Instruct",
         LLMProvider.COHERE: "command-a-03-2025",
         LLMProvider.CEREBRAS: "gpt-oss-120b",
         LLMProvider.HUGGINGFACE: "Qwen/Qwen2.5-72B-Instruct",
@@ -180,6 +182,7 @@ def provider_client_config() -> dict[LLMProvider, dict[str, Any]]:
         LLMProvider.GROQ: {"timeout": 10},
         LLMProvider.LLAMACPP: {"api_base": "http://127.0.0.1:8090/v1"},
         LLMProvider.VLLM: {"api_base": "http://127.0.0.1:8080/v1"},
+        LLMProvider.CASCADIA: {"api_base": "http://localhost:9090/v1"},
         LLMProvider.MISTRAL: {"timeout_ms": 100000},
         LLMProvider.NEOSANTARA: {"timeout": 10},
         LLMProvider.NEBIUS: {"api_base": "https://api.studio.nebius.com/v1/"},

tests/constants.py

@@ -7,13 +7,15 @@
     LLMProvider.OLLAMA,
     LLMProvider.LMSTUDIO,
     LLMProvider.LLAMAFILE,
+    LLMProvider.CASCADIA,
 ]
 
 # Providers that should never run in CI (only for local development)
 CI_EXCLUDED_PROVIDERS = [
     LLMProvider.AZUREANTHROPIC,
     LLMProvider.VERTEXAIANTHROPIC,
     LLMProvider.VLLM,
+    LLMProvider.CASCADIA,
 ]
 
 # Strip whitespace and drop empties so values like "anthropic, otari" or an unset env var

tests/unit/providers/test_cascadia_provider.py

@@ -0,0 +1,75 @@
+import pytest
+
+from any_llm.any_llm import AnyLLM
+from any_llm.constants import LLMProvider
+from any_llm.providers.cascadia.cascadia import CascadiaProvider
+
+
+def test_provider_metadata() -> None:
+    provider = CascadiaProvider()
+    assert provider.PROVIDER_NAME == "cascadia"
+    assert provider.API_BASE == "http://localhost:9090/v1"
+    assert provider.ENV_API_KEY_NAME == "CASCADIA_API_KEY"
+    assert provider.ENV_API_BASE_NAME == "CASCADIA_API_BASE"
+    assert provider.PROVIDER_DOCUMENTATION_URL == "https://cascadia.to"
+
+
+def test_provider_without_api_key(monkeypatch: pytest.MonkeyPatch) -> None:
+    # Keyless coordinators on trusted networks must work (vLLM convention).
+    monkeypatch.delenv("CASCADIA_API_KEY", raising=False)
+    provider = CascadiaProvider()
+    assert provider._verify_and_set_api_key(None) == "no-key-required"
+
+
+def test_provider_with_api_key() -> None:
+    provider = CascadiaProvider(api_key="test-api-key")
+    assert provider._verify_and_set_api_key("test-api-key") == "test-api-key"
+
+
+def test_api_key_from_env(monkeypatch: pytest.MonkeyPatch) -> None:
+    # A keyed fleet's token set via CASCADIA_API_KEY must be honored.
+    monkeypatch.setenv("CASCADIA_API_KEY", "env-key")
+    provider = CascadiaProvider()
+    assert provider._verify_and_set_api_key(None) == "env-key"
+
+
+def test_explicit_api_key_takes_precedence_over_env(monkeypatch: pytest.MonkeyPatch) -> None:
+    # An explicit key must win over CASCADIA_API_KEY in the environment.
+    monkeypatch.setenv("CASCADIA_API_KEY", "env-key")
+    provider = CascadiaProvider(api_key="explicit-key")
+    assert provider._verify_and_set_api_key("explicit-key") == "explicit-key"
+
+
+def test_api_base_from_env(monkeypatch: pytest.MonkeyPatch) -> None:
+    monkeypatch.setenv("CASCADIA_API_BASE", "http://fleet.internal:9090/v1")
+    provider = CascadiaProvider()
+    assert provider._resolve_api_base(None) == "http://fleet.internal:9090/v1"
+
+
+def test_api_base_default_when_unset(monkeypatch: pytest.MonkeyPatch) -> None:
+    monkeypatch.delenv("CASCADIA_API_BASE", raising=False)
+    provider = CascadiaProvider()
+    # No env var, no explicit base: falls through to the class default.
+    assert provider._resolve_api_base(None) is None
+    assert str(provider.client.base_url).rstrip("/") == "http://localhost:9090/v1"
+
+
+def test_capability_flags() -> None:
+    assert CascadiaProvider.SUPPORTS_COMPLETION
+    assert CascadiaProvider.SUPPORTS_COMPLETION_STREAMING
+    assert CascadiaProvider.SUPPORTS_COMPLETION_REASONING
+    assert CascadiaProvider.SUPPORTS_LIST_MODELS
+    assert not CascadiaProvider.SUPPORTS_EMBEDDING
+    assert not CascadiaProvider.SUPPORTS_MODERATION
+    assert not CascadiaProvider.SUPPORTS_COMPLETION_IMAGE
+    assert not CascadiaProvider.SUPPORTS_COMPLETION_PDF
+    assert not CascadiaProvider.SUPPORTS_BATCH
+    assert not CascadiaProvider.SUPPORTS_IMAGE_GENERATION
+    assert not CascadiaProvider.SUPPORTS_RERANK
+    assert not CascadiaProvider.SUPPORTS_RESPONSES
+
+
+def test_registered_in_enum_and_loader() -> None:
+    assert LLMProvider.from_string("cascadia") is LLMProvider.CASCADIA
+    assert AnyLLM.get_provider_class("cascadia") is CascadiaProvider
+    assert "cascadia" in AnyLLM.get_supported_providers()