Support thinking to ChatGoogleAI

[nallwhy]

No description provided.

[nallwhy]

@brainlid

I’m almost done, but I’m currently facing an issue when merging MessageDelta in the case of stream: true.
The following code should succeed, but there’s a problem when merging content parts of different types.

deltas =
  [
    %LangChain.MessageDelta{
      content: %LangChain.Message.ContentPart{
        type: :thinking,
        content:
          "**Refining the Questions**\n\nI'm now zeroing in on the core philosophical questions. I'm expanding my initial list to include a deeper dive into the implications for ethics and agency in AI systems. The structure is taking form. I am exploring consciousness, ethics, agency, and personhood, as these are the main aspects.\n\n\n",
        options: []
      },
      merged_content: [],
      status: :incomplete,
      index: 0,
      role: :assistant,
      tool_calls: nil,
    },
    %LangChain.MessageDelta{
      content: %LangChain.Message.ContentPart{
        type: :text,
        content: "Okay, here is a deeply",
        options: []
      },
      merged_content: [],
      status: :incomplete,
      index: 0,
      role: :assistant,
      tool_calls: nil,
    }
  ]

LangChain.MessageDelta.merge_deltas(deltas)

11:58:33.281 [warning] Cannot merge content parts of different types: ...

Looking at MessageDelta.merge_content_part_at_index/3, it seems the position is determined via the index in the parts.
Does this mean that we don’t mix different content part types (e.g., thinking, text) in a single Message, but instead separate them into different Messages as a rule?

If that’s the case, when a MessageDelta with a different content type is streamed, I may need to manually mark the current message as status: :complete in ChatGoogleAI.

I’d like to hear your thoughts on the best way to handle this.
Let me know what you think.

[brainlid]

Hi @nallwhy! Thanks for your work on this, it's looking good!

To answer your question, yes, we do not merge a thinking and text content part. It could also be an image or file.

Other APIs like ChatOpenAI and ChatAnthropic both return an index in the delta for a content part to indicate which content part (one or more) it is connected to.

So a single API response can return a single message with multiple content parts where some are thinking related (Anthropic includes thinking signatures to prevent and detect tampering with it) which is separate from a displayed text response.

Without having looked into it, I would expect some type of index for the content part to come from Google API.

[nallwhy]

Implemented message delta separation so that the output matches the behavior observed in ChatAnthropic’s results.

[brainlid]

Thanks @nallwhy!
❤️💛💙💜