Skip to content

Clean up treating MessageDelta in ChatModels.ChatGoogleAI#353

Merged
brainlid merged 2 commits into
brainlid:mainfrom
nallwhy:clean_up_chat_google_ai_message_delta
Aug 17, 2025
Merged

Clean up treating MessageDelta in ChatModels.ChatGoogleAI#353
brainlid merged 2 commits into
brainlid:mainfrom
nallwhy:clean_up_chat_google_ai_message_delta

Conversation

@nallwhy

@nallwhy nallwhy commented Aug 5, 2025

Copy link
Copy Markdown
Contributor

No description provided.

@brainlid

Copy link
Copy Markdown
Owner

Hi @nallwhy!

These look like some significant changes. Is this a case of the Google model API changing? Or did it not work correctly before?

And can you confirm that these changes are working correctly for you against a live server?

@nallwhy

nallwhy commented Aug 15, 2025

Copy link
Copy Markdown
Contributor Author

This wasn’t a Google API change. It looks like some incorrect code had lingered, and this PR cleans that up.

What changed and why:

  • role mapping — Updated per Google’s Content docs (https://ai.google.dev/api/caching#Content) so we’re setting roles correctly.
  • Removed do_process_response/3 — That clause was never reachable; all callers match the functions above it, so the extra function could never be invoked.
  • Finish reason handling — We were always marking MessageDelta as incomplete due to using a wrong key (complete). Now we correctly map Google’s finishReason and set completion state as intended.

I’ve already applied these changes in production code using both streaming and non-streaming flows, and they work correctly in both cases.

In the previous code, even when finishReason was present, the status of MessageDelta was set to incomplete. I’m curious what the intention behind that was. In ChatOpenAI, it seems that the status is set to complete based on the finishReason.

@nallwhy
nallwhy force-pushed the clean_up_chat_google_ai_message_delta branch from 54b658c to e0989eb Compare August 15, 2025 05:02
@nallwhy

nallwhy commented Aug 15, 2025

Copy link
Copy Markdown
Contributor Author

Sample stream

DECODED STREAM DATA: {
  [
   % {
      "candidates" => [
       % {
          "content" => % {
            "parts" => [
             % {
                "functionCall" => % { "args" => % {}, "name" => "current_datetime" },
                "thoughtSignature" => "CiQBVKhc7n3VmT2ZrGFTpttOeSFQJRKz82lAZ/hrQPU0Jze8ifoKZgFUqFzuVVfE3aYh3N/CrOTZ2wITKfi/CF0oAyo+ESh1hecgQNsyikC7fZwMs4bfw4c9J0nHWEDmgCg0t78ZmLxKepvaMRyXDjhn1eQgZ+x3nkGMkM4DOhccoboZ4kpjauJ+0UUS2wqhAQFUqFzuunrV3v4M420/JEhr2nNxS0/TgwkGUz4uI5ED4mdoXLl0kNW1iexW5UN+bzVA3034G5n6ZQwwkyf/ul5w4UMszw2+oZ+xOiZJHVUWuNcRCPsPaTcrk3KFsIchtASvxnSJtF8EZkZByiY6xgTRsxQRvIx+zbEVNp0RR06TKlRTUV6Gu4U4dVpOAdIAzRw7Gy87+6m5dkT7G1clkI7x"
              }
            ],
            "role" => "model"
          },
          "finishReason" => "STOP",
          "index" => 0
        }
      ],
      "modelVersion" => "gemini-2.5-flash",
      "responseId" => "ykyRaNTaE7WcqtsP2vHd6QU",
      "usageMetadata" => % {
        "candidatesTokenCount" => 10,
        "promptTokenCount" => 41,
        "promptTokensDetails" => [% { "modality" => "TEXT", "tokenCount" => 41 }],
        "thoughtsTokenCount" => 46,
        "totalTokenCount" => 97
      }
    }
  ], ""
}

DECODED STREAM DATA: {
  [
   % {
      "candidates" => [
       % {
          "content" => % {
            "parts" => [% { "text" => "The current time is" }],
            "role" => "model"
          },
          "index" => 0
        }
      ],
      "modelVersion" => "gemini-2.5-flash",
      "responseId" => "y0yRaIytCaf6qtsP8e7n-Q8",
      "usageMetadata" => % {
        "candidatesTokenCount" => 4,
        "promptTokenCount" => 103,
        "promptTokensDetails" => [% { "modality" => "TEXT", "tokenCount" => 103 }],
        "totalTokenCount" => 107
      }
    }
  ], ""
}
DECODED STREAM DATA: {
  [
   % {
      "candidates" => [
       % {
          "content" => % {
            "parts" => [% { "text" => " 00:14:02 UTC on August 5, 20" }],
            "role" => "model"
          },
          "index" => 0
        }
      ],
      "modelVersion" => "gemini-2.5-flash",
      "responseId" => "y0yRaIytCaf6qtsP8e7n-Q8",
      "usageMetadata" => % {
        "candidatesTokenCount" => 21,
        "promptTokenCount" => 103,
        "promptTokensDetails" => [% { "modality" => "TEXT", "tokenCount" => 103 }],
        "totalTokenCount" => 124
      }
    }
  ], ""
}
DECODED STREAM DATA: {
  [
   % {
      "candidates" => [
       % {
          "content" => % { "parts" => [% { "text" => "25." }], "role" => "model" },
          "finishReason" => "STOP",
          "index" => 0
        }
      ],
      "modelVersion" => "gemini-2.5-flash",
      "responseId" => "y0yRaIytCaf6qtsP8e7n-Q8",
      "usageMetadata" => % {
        "candidatesTokenCount" => 24,
        "promptTokenCount" => 103,
        "promptTokensDetails" => [% { "modality" => "TEXT", "tokenCount" => 103 }],
        "totalTokenCount" => 127
      }
    }
  ], ""
}

@brainlid

Copy link
Copy Markdown
Owner

Thanks for the clarification @nallwhy. This PR has a slight conflict with another PR of yours I just merged. After you resolve it (because you can test it and verify it works as expected), I'll merge this.

@nallwhy
nallwhy force-pushed the clean_up_chat_google_ai_message_delta branch from e0989eb to 0540c5b Compare August 16, 2025 23:45
@nallwhy

nallwhy commented Aug 16, 2025

Copy link
Copy Markdown
Contributor Author

@brainlid I've rebased the commits.

@brainlid

Copy link
Copy Markdown
Owner

Thanks @nallwhy!
❤️💛💙💜

@brainlid
brainlid merged commit 528de25 into brainlid:main Aug 17, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants