Currently we handle texts in one thread:
|
const free = await this.semafor.take(); |
That make sense, because free services have strong rate limits.
However it works very bad with LLM translators. They are error prone, they generate a lot of errors and wait for 3-5 retries.
We could run at least 10 threads in parallel and show the translations faster.
Let's add the optional method like getConcurrency(): number and in case it is larger than 1 we will run translation in multiple threads.
Currently we handle texts in one thread:
core/src/scheduling/Scheduler.ts
Line 433 in 3fcfd2a
That make sense, because free services have strong rate limits.
However it works very bad with LLM translators. They are error prone, they generate a lot of errors and wait for 3-5 retries.
We could run at least 10 threads in parallel and show the translations faster.
Let's add the optional method like
getConcurrency(): numberand in case it is larger than 1 we will run translation in multiple threads.