Skip to content

Commit d1fe7a1

Browse files
prplxprplx
authored andcommitted
fix: allow manual fallback models when provider catalog fails
1 parent cf7d252 commit d1fe7a1

4 files changed

Lines changed: 118 additions & 28 deletions

File tree

README.md

Lines changed: 38 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,19 @@
33
<p align="center">
44
<img width="1439" height="854" alt="image" src="https://github.com/user-attachments/assets/b4f68d1a-1c12-4abc-b810-1280f3ef49cb" />
55
</p>
6-
<p align="center"><strong>Desktop AI chat, RP, writing, lorebook, RAG, and plugin workbench.</strong></p>
6+
<p align="center"><strong>Desktop AI chat, RP, writing, RAG, agent, and plugin workbench.</strong></p>
77

88
Desktop app built with Electron, React, a local Express API, and SQLite.
99

1010
<img width="1440" height="857" alt="image" src="https://github.com/user-attachments/assets/03e75de3-5b39-4012-98f8-4c959eb1fc80" />
1111

12+
## Current Release
13+
14+
- Latest release: [`v0.9.7`](https://github.com/tg-prplx/vellium/releases/tag/v0.9.7)
15+
- Desktop builds: macOS (`arm64`, `x64`), Windows (`x64`), Linux (`x64` AppImage).
16+
- Release builds are unsigned. macOS and Windows may require manual confirmation on first launch.
17+
- The app is usable day to day, but still moving quickly. Expect active iteration around Agents, tool calling, and provider compatibility.
18+
1219
## User Documentation
1320

1421
- Detailed user guide: [`docs/vellium/README.md`](./docs/vellium/README.md)
@@ -17,9 +24,9 @@ Desktop app built with Electron, React, a local Express API, and SQLite.
1724
## Important
1825
- Use `npm run dev` for day-to-day development.
1926
- Use `npm run dev:electron` when testing the real desktop shell.
20-
- Use `npm run dist:mac` / `npm run dist:win` for desktop bundles.
21-
- CI desktop builds are unsigned. macOS and Windows may require manual confirmation.
22-
- Desktop packaging works, but it still has rough edges. It is usable, not polished.
27+
- Use `npm run dist:mac`, `npm run dist:win`, or `npm run dist:linux` for platform bundles.
28+
- CI publishes GitHub Release assets when a `v*` tag is pushed.
29+
- Local data is stored in `data/` during development and in the Electron user-data directory in packaged builds.
2330

2431
## Stack
2532
- Electron
@@ -30,14 +37,23 @@ Desktop app built with Electron, React, a local Express API, and SQLite.
3037

3138
## Core Features
3239

40+
### Agents
41+
- Dedicated `Agents` workspace with ask, build, and research modes.
42+
- Workspace tools for listing, reading, searching, editing, moving, deleting, and diffing files.
43+
- Optional command execution for tests/builds, with separate security gates for shell-like commands, network commands, destructive file operations, and git writes.
44+
- OpenAI-compatible structured planning with JSON-schema responses when supported.
45+
- Mid-run corrections, abort/resume/retry, event traces, reasoning traces, and partial-response recovery.
46+
- Context management for long agent threads, including auto-compaction, continuation cues, duplicate read-only call guards, and stale-run cleanup after edits/deletes.
47+
3348
### Chat / RP
3449
- Branching chat history.
3550
- Edit, delete, resend, regenerate.
3651
- Multi-character chats with auto-turns.
3752
- RP controls: prompt stack, author note, scene state, presets, personas.
3853
- LoreBook / World Info support, including SillyTavern-compatible world info import/export.
39-
- Reasoning support, including `<think>...</think>` parsing.
54+
- Reasoning support, including streamed reasoning fields and `<think>...</think>` parsing.
4055
- Vision attachments and chat attachments.
56+
- MCP tool calling for OpenAI-compatible chat/completions providers, with text-tool-call fallback parsing for providers that do not emit native tool calls cleanly.
4157

4258
### Writing
4359
- Projects, chapters, scenes, outlines.
@@ -56,7 +72,10 @@ Desktop app built with Electron, React, a local Express API, and SQLite.
5672
- OpenAI-compatible providers.
5773
- KoboldCpp support.
5874
- Custom endpoint adapters for non-OpenAI / non-Kobold backends.
75+
- Presets for OpenAI, LM Studio, Ollama, KoboldCpp, OpenRouter, and custom OpenAI-compatible endpoints.
76+
- Manual fallback models for providers whose `/models` endpoint is missing, empty, or provider-specific.
5977
- Separate models for translate / compress / TTS / RAG.
78+
- API parameter forwarding controls for providers that reject unsupported sampling fields.
6079

6180
### Plugins / Extensions
6281
- Toolbar tabs from plugins.
@@ -72,7 +91,7 @@ Desktop app built with Electron, React, a local Express API, and SQLite.
7291

7392

7493
## Requirements
75-
- Node.js + npm.
94+
- Node.js + npm. Node.js 20+ is recommended because CI builds with Node 20.
7695
- Python 3 + Pillow for icon generation:
7796

7897
```bash
@@ -148,6 +167,12 @@ Windows only:
148167
npm run dist:win
149168
```
150169

170+
Linux AppImage only:
171+
172+
```bash
173+
npm run dist:linux
174+
```
175+
151176
Build output goes to `release/`.
152177

153178
## GitHub Actions
@@ -156,8 +181,8 @@ Workflow:
156181
- `.github/workflows/build-desktop.yml`
157182

158183
What it does:
159-
- builds macOS (`x64`, `arm64`) and Windows (`x64`) bundles,
160-
- uploads artifacts,
184+
- builds macOS (`x64`, `arm64`), Windows (`x64`), and Linux (`x64` AppImage) bundles,
185+
- uploads workflow artifacts,
161186
- publishes GitHub Release assets on `v*` tag pushes.
162187

163188
## Plugins
@@ -174,11 +199,11 @@ Plugin capabilities:
174199
- `Pluginfile` import/export.
175200

176201
Useful docs:
177-
- `/Users/prplx/Documents/slv/docs/plugins/README.md`
202+
- [`docs/plugins/README.md`](./docs/plugins/README.md)
178203

179204
Runtime plugin locations:
180-
- user plugins: `/Users/prplx/Documents/slv/data/plugins`
181-
- bundled plugins: `/Users/prplx/Documents/slv/data/bundled-plugins`
205+
- user plugins: `data/plugins`
206+
- bundled plugins: `data/bundled-plugins`
182207

183208
Important:
184209
- plugins are local extensions, not a trusted public plugin marketplace model,
@@ -251,6 +276,8 @@ Generated files:
251276
- `npm run build` — frontend production build.
252277
- `npm run build:server` — bundled server build.
253278
- `npm run build:desktop` — full desktop build pipeline without publishing.
279+
- `npm run dist` — package all desktop targets supported by the current host/CI runner.
280+
- `npm run dist:mac` / `npm run dist:win` / `npm run dist:linux` — package a specific desktop target.
254281
- `npm run rebuild:native` — rebuild `better-sqlite3`.
255282
- `npm run test` — Vitest.
256283

docs/vellium/README.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,11 @@ Vellium is a local-first desktop/workbench app for:
88
- long-form writing workflows
99
- characters and LoreBooks
1010
- knowledge collections and RAG
11+
- autonomous agent workflows over a selected workspace
1112
- MCP / tool calling
1213
- local plugins and themes
1314

14-
This guide documents the current UI and is based on the real app areas: `Welcome`, `Chat`, `Writing`, `Characters`, `LoreBooks`, `Knowledge`, `Settings`, and plugin-powered surfaces.
15+
This guide documents the current UI and is based on the real app areas: `Welcome`, `Chat`, `Writing`, `Agents`, `Characters`, `LoreBooks`, `Knowledge`, `Settings`, and plugin-powered surfaces.
1516

1617
The screenshots in this guide are local captures from the current app build. Where it makes onboarding clearer, they use `Simple Mode` so the first-run flow matches what many users will actually see.
1718

@@ -41,9 +42,11 @@ flowchart LR
4142
F["LoreBooks"] --> C
4243
G["Knowledge"] --> C
4344
G --> D
45+
B --> I["Agents"]
4446
B --> H["Plugins / Themes / MCP"]
4547
H --> C
4648
H --> D
49+
H --> I
4750
```
4851

4952
## Workspaces
@@ -52,6 +55,7 @@ flowchart LR
5255
| --- | --- | --- |
5356
| `Chat` | Dialogues, RP, tool calling, translation, TTS | `Characters`, `LoreBooks`, `Knowledge`, `Settings` |
5457
| `Writing` | Books, chapters, scenes, drafts, summaries, lenses | `Characters`, `Knowledge`, `Settings` |
58+
| `Agents` | Ask/build/research workflows over a workspace with tools, traces, and resumable runs | `Settings`, provider profiles, workspace/tool security |
5559
| `Characters` | Importing and editing character cards | `Chat`, `Writing` |
5660
| `LoreBooks` | World facts, trigger keys, scripted prompt injections | `Chat` |
5761
| `Knowledge` | Retrieval collections for RAG | `Chat`, `Writing`, `Settings` |
@@ -66,13 +70,15 @@ flowchart LR
6670
4. Add or import a character in `Characters`.
6771
5. If your workflow needs world facts, create a LoreBook.
6872
6. If your workflow needs retrieval, create a knowledge collection in `Knowledge`.
69-
7. Only after that move on to multi-character scenes, writer workflows, plugins, and MCP.
73+
7. For workspace automation, create an `Agents` thread after providers and tool/security settings are configured.
74+
8. Only after that move on to multi-character scenes, writer workflows, plugins, and MCP.
7075

7176
## Important Things to Know Up Front
7277

7378
- Vellium is not tied to a single backend. Chat, translation, compression, TTS, and RAG can all use different models.
7479
- `Local-only mode` limits the app to localhost or private-network endpoints.
7580
- Tool calling through MCP only works with OpenAI-compatible chat/completions providers, not with KoboldCpp.
81+
- Agents can use first-party workspace tools when enabled. Command execution, network commands, destructive file operations, and git writes are separately gated in settings.
7682
- `Knowledge` and `LoreBooks` solve different problems: one is retrieval-based, the other is trigger-based scripted context.
7783
- Plugins in Vellium are local extensions. Treat their permissions the same way you would treat shell tools or third-party scripts.
7884

server/app/createApp.integration.test.ts

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3287,6 +3287,42 @@ process.stdin.on("data", (chunk) => {
32873287
});
32883288
});
32893289

3290+
it("uses manual fallback models when a provider model endpoint cannot be loaded", async () => {
3291+
const fallbackPayload = {
3292+
baseUrl: `${mockProviderBaseUrl}/missing-catalog`,
3293+
apiKey: "test-key",
3294+
fullLocalOnly: false,
3295+
providerType: "openai",
3296+
adapterId: null,
3297+
manualModels: ["featherless/manual-model"]
3298+
};
3299+
3300+
const previewModels = await postJson("/api/providers/preview/models", fallbackPayload);
3301+
expect(previewModels).toEqual([{ id: "featherless/manual-model" }]);
3302+
3303+
const previewTest = await postJson("/api/providers/preview/test", fallbackPayload);
3304+
expect(previewTest).toEqual({ ok: true });
3305+
3306+
const savedProvider = await postJson("/api/providers", {
3307+
id: "manual-fallback-provider",
3308+
name: "Manual Fallback Provider",
3309+
baseUrl: fallbackPayload.baseUrl,
3310+
apiKey: fallbackPayload.apiKey,
3311+
proxyUrl: null,
3312+
fullLocalOnly: false,
3313+
providerType: "openai",
3314+
adapterId: null,
3315+
manualModels: fallbackPayload.manualModels
3316+
});
3317+
expect(savedProvider.manualModels).toEqual(["featherless/manual-model"]);
3318+
3319+
const savedModels = await parseJsonResponse(
3320+
"/api/providers/manual-fallback-provider/models",
3321+
await fetch(`${baseUrl}/api/providers/manual-fallback-provider/models`)
3322+
);
3323+
expect(savedModels).toEqual([{ id: "featherless/manual-model" }]);
3324+
});
3325+
32903326
it("streams tool-calling turns through an MCP server and persists tool traces", async () => {
32913327
await updateSettings({
32923328
activeProviderId: "mock-openai",

server/routes/providers.ts

Lines changed: 36 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,10 @@ async function fetchOpenAiCompatibleModels(baseUrlRaw: string, apiKeyRaw: string
7878

7979
const apiKey = String(apiKeyRaw || "").trim();
8080
const response = await fetch(`${baseUrl}/models`, {
81-
headers: apiKey ? { Authorization: `Bearer ${apiKey}` } : undefined
81+
headers: {
82+
Accept: "application/json",
83+
...(apiKey ? { Authorization: `Bearer ${apiKey}` } : {})
84+
}
8285
});
8386
if (!response.ok) {
8487
const text = await response.text().catch(() => "");
@@ -109,6 +112,26 @@ async function fetchOpenAiCompatibleModels(baseUrlRaw: string, apiKeyRaw: string
109112
return Array.from(uniq.values());
110113
}
111114

115+
function mergeManualModels(models: Array<{ id: string }>, manualModels: Array<{ id: string }>) {
116+
if (models.length === 0) return manualModels;
117+
return [
118+
...models,
119+
...manualModels.filter((item) => !models.some((model) => model.id === item.id))
120+
];
121+
}
122+
123+
async function resolveWithManualFallback(
124+
manualModels: Array<{ id: string }>,
125+
fetchModels: () => Promise<Array<{ id: string }>>
126+
) {
127+
try {
128+
return mergeManualModels(await fetchModels(), manualModels);
129+
} catch (error) {
130+
if (manualModels.length > 0) return manualModels;
131+
throw error;
132+
}
133+
}
134+
112135
function assertProviderAllowed(baseUrl: string, fullLocalOnly: boolean) {
113136
const settings = getSettings();
114137
if (settings.fullLocalMode && !isLocalhostUrl(baseUrl)) {
@@ -141,25 +164,23 @@ async function resolveProviderModels(row: Pick<ProviderRow, "base_url" | "api_ke
141164

142165
const providerType = normalizeProviderType(row.provider_type);
143166
if (providerType === "koboldcpp") {
144-
const koboldModels = await fetchKoboldModels(row);
145-
const fetched = koboldModels.map((id) => ({ id }));
146-
return fetched.length > 0
147-
? [...fetched, ...manualModels.filter((item) => !fetched.some((model) => model.id === item.id))]
148-
: manualModels;
167+
return resolveWithManualFallback(manualModels, async () => {
168+
const koboldModels = await fetchKoboldModels(row);
169+
return koboldModels.map((id) => ({ id }));
170+
});
149171
}
150172

151173
if (providerType === "custom") {
152-
const customModels = await fetchCustomAdapterModels(row);
153-
const fetched = customModels.map((id) => ({ id }));
154-
return fetched.length > 0
155-
? [...fetched, ...manualModels.filter((item) => !fetched.some((model) => model.id === item.id))]
156-
: manualModels;
174+
return resolveWithManualFallback(manualModels, async () => {
175+
const customModels = await fetchCustomAdapterModels(row);
176+
return customModels.map((id) => ({ id }));
177+
});
157178
}
158179

159-
const models = await fetchOpenAiCompatibleModels(row.base_url, row.api_key_cipher);
160-
return models.length > 0
161-
? [...models, ...manualModels.filter((item) => !models.some((model) => model.id === item.id))]
162-
: manualModels;
180+
return resolveWithManualFallback(
181+
manualModels,
182+
() => fetchOpenAiCompatibleModels(row.base_url, row.api_key_cipher)
183+
);
163184
}
164185

165186
router.post("/", (req, res) => {

0 commit comments

Comments
 (0)