Skip to content

[Enhancement] Auto-save streaming responses to survive crashes #374

@Ganeshship

Description

@Ganeshship

Summary

Streaming responses live only in RAM until generation completes. If the app or model crashes mid-generation, the partial output is lost even though the user could see it on screen. Even a partial response is better than losing everything and regenerating from scratch.

Problem

  1. User chats with a model
  2. Model generates a long response (streaming, token by token)
  3. App crashes mid-generation (background kill, model crash, OOM)
  4. User reopens the app
  5. The partial response that was already visible is gone
  6. User has to regenerate from scratch

Root cause: streaming text is stored in Zustand state (RAM). It only persists to AsyncStorage when finalizeStreamingMessage() runs after generation completes. If the app crashes before that, the function never fires and the output is lost.

Proposed Solution

Auto-save the streaming buffer to AsyncStorage every ~100 tokens during generation. On next app launch, detect the partial response and offer recovery.

During streaming:

  • Every 100 tokens, write current streaming content to a partial_response key in AsyncStorage (async, non-blocking)
  • On successful generation complete, delete partial_response

On app launch:

  • Check for partial_response in AsyncStorage
  • If found, prompt: "A response was interrupted. Recover partial output?"
  • Recover appends the partial message to the conversation
  • Discard deletes it

Files involved

  • src/stores/chatStore.ts - streamingMessage and appendStreamingToken
  • App entry point - recovery check on launch

Reported by

Community member on Slack. Original feedback: partial output should auto-save to storage instead of staying solely in RAM. Even a partial output is better than having to regenerate everything.

Platform

  • Android
  • iOS
  • Both

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions