feat: Add Gmail REST API backend for native CJK search support

[metrovoc]

Problem

IMAP SEARCH on Gmail servers cannot handle CJK (Chinese/Japanese/Korean) characters. This is a known limitation of Gmail's IMAP implementation — search queries containing CJK text return empty results. This affects a significant number of users in East Asia.

Proposed Solution

Add a Gmail REST API backend that uses the Gmail API directly instead of going through IMAP. The Gmail API's q parameter natively supports CJK full-text search.

This backend would implement all existing feature traits (folders, envelopes, messages, flags) by mapping them to Gmail API endpoints:

• Gmail labels → folders
• users.messages.list with q param → ListEnvelopes (CJK search works natively)
• users.messages.get(format=RAW) → GetMessages (returns standard RFC 2822)
• users.messages.send → SendMessage
• users.messages.modify → flag/copy/move operations
• OAuth2 authentication (reusing existing OAuth2 infrastructure)

Implementation

I have a working implementation across all three repos (core, tui, himalaya):

• core: https://github.com/metrovoc/pimalaya-core/tree/himalaya-v1.2-compat — GmailContext, all trait implementations (~1250 lines), config, error handling
• tui: https://github.com/metrovoc/pimalaya-tui/tree/himalaya-v1.2-compat — Backend/Config enum integration, setup wizard
• himalaya: https://github.com/metrovoc/himalaya/tree/master — feature flag wiring, doctor command, sample config

All changes compile cleanly with cargo check --features gmail. The implementation follows the same patterns as the existing IMAP/Maildir/Notmuch backends.

I'd be happy to open PRs if you're interested in merging this upstream. Would appreciate feedback on the approach before doing so.

[soywod]

Thank you for the initiative, much appreciated. Sad you did not ask me first, because the Himalaya v2 is on its way, with a huge Pimalaya libraries refactoring. Would you be able to make a working version based on v2 branch? It should be much easier to integrate, but the API is not stabilized yet.

[metrovoc]

Thanks for the steer toward v2. I took a shot at the v2 version:

• https://github.com/metrovoc/io-gmail (sans-io, same shape as io-jmap / io-oauth)
• https://github.com/metrovoc/himalaya/tree/v2-gmail (CLI on top)

Covers the usual read/write paths, and CJK search works end to end against my own account, which was the original motivation. Still early, but wanted to share a pointer. Feedback welcome whenever you have a moment to look.

[soywod]

The v2 starts to stabilize on master, could you pull latest so I could try?

EDIT: many things change, the migration is not obvious at all. Let me give it a try in a fork.

[soywod]

I forked your io-gmail and update the code with new Pimalaya libs and standards:

https://github.com/pimalaya/io-gmail

Thanks a lot for your initiative, it definitely served as a solid basement. How would you like to proceed? Would you like to be the maintainer of io-gmail, and Pimalaya as a co-contributor? Or do you prefer Pimalaya to maintain it? We adjust to your needs, you initiated the work. Let us know.

It is now available on Himalaya, master branch, curious to have your opinion. I close the issue as it is resolved, but we can continue the discussion about modalities.

[metrovoc]

Hi @soywod, thanks a lot for the kind words and for doing the heavy lifting on the rebase. Much appreciated.

I pulled the latest and tried it against my own account. The core works great: CJK search through the Gmail q parameter returns correct results (e.g. searching Chinese or Japanese terms), listing and envelope list render CJK subjects correctly too.

One small issue: reading a message whose body uses a legacy CJK charset (e.g. Content-Type: text/plain; charset=ISO-2022-JP, still very common for Japanese bank / utility / corporate mail) comes out as mojibake in message read. Both the body and the encoded-word subject show raw escape sequences:

Subject: Text("\u{1b}$B$4MxMQ$N$*CN$i$;...")
body:    ...$B!Z$/$i$7(BTEPCO web$B![%]%$%s%H...

Interesting detail: envelope list / gmail messages list render the same subject fine, because those use the header values already decoded by the Gmail API. message read fetches format=RAW and parses the full RFC 2822 with mail-parser, which is where it breaks.

It seems the cause is that mail-parser is built without the full_encoding feature, so the non-UTF-8 charset decoders (ISO-2022-JP, Shift-JIS, EUC-JP, GBK, Big5, …) aren't compiled in and the content passes through undecoded. I confirmed that adding full_encoding to the mail-parser dependency in himalaya's Cargo.toml fixes it. The same messages then render correctly:

mail-parser = { version = "0.11", features = ["full_encoding", "serde"], optional = true }

I just opened a PR for this #706 .

On maintenance: I'd prefer Pimalaya to own io-gmail. I can't really commit to actively maintaining a project in production, but I'll keep using it daily and I'm glad to report issues and give feedback going forward.

Thanks again for the great work!

[soywod]

Great, happy to know that it works as expected for you!

On maintenance: I'd prefer Pimalaya to own io-gmail. I can't really commit to actively maintaining a project in production, but I'll keep using it daily and I'm glad to report issues and give feedback going forward.

Alright, let's go for it. I propose you the following: since you initiated the work, it could be great that you remain part of contributors. What about I propose a PR on your repo, you merge it, then you transfer the ownership to Pimalaya?

[metrovoc]

Sounds good, let's go with that.
One practical thing: since pimalaya/io-gmail already exists as a fork of my repo, a same-name transfer into the org will probably be blocked. Do you want to delete/rename that fork first so I can transfer the original over, or would you rather just keep the existing pimalaya/io-gmail as the canonical one and add me as a contributor there? Either works for me