We currently have translate method for translate single text, and translateBatch method to translate many texts in one request.
The problem with translateBatch is we have to wait for translating whole request. That is critical for services that generates a lot of errors, like LLM translation API.
In case with LLM translators, we could parse a response and in case it is partially correct, resolve the part of translated texts, and ask LLM to translate only invalid part. It would speed up the translation.
Currently we just ask the LLM to translate all texts again and user wait for the response. It may take a dozens seconds to handle one batch of texts.
As side effect, the translators could implement the concurrent translation itself. That is related to #143
We currently have
translatemethod for translate single text, andtranslateBatchmethod to translate many texts in one request.The problem with
translateBatchis we have to wait for translating whole request. That is critical for services that generates a lot of errors, like LLM translation API.In case with LLM translators, we could parse a response and in case it is partially correct, resolve the part of translated texts, and ask LLM to translate only invalid part. It would speed up the translation.
Currently we just ask the LLM to translate all texts again and user wait for the response. It may take a dozens seconds to handle one batch of texts.
As side effect, the translators could implement the concurrent translation itself. That is related to #143