Add CLI test scripts for Parakeet and Whisper-Tiny ASR models#27
Conversation
| # run_case "Run MediaTek compiled Parakeet-TDT" \ | ||
| # litert run "$MODEL_DIR/parakeet_tdt/parakeet_tdt_0.6b_v3_MediaTek_MT6993.tflite" --android --npu | ||
| # run_case "Benchmark MediaTek compiled Parakeet-TDT" \ | ||
| # litert benchmark "$MODEL_DIR/parakeet_tdt/parakeet_tdt_0.6b_v3_MediaTek_MT6993.tflite" --android --npu |
There was a problem hiding this comment.
Will those compile comamnds work for those machines?
If yes, good to put here; If not, we can remove them for now.
Also, the benchmarking or running might be not be acculate, because there are both encoder and decoder， and we need to specify which signature to benchmark.
Fine to submit first, and I will change after this.
There was a problem hiding this comment.
Got it. Thanks for pointing that out and for the review. I haven't tested the MediaTek commands yet, but I will test them tomorrow and confirm. Thank you!
There was a problem hiding this comment.
I tested the MediaTek - The compilation works, but the run command fails because libLiteRtDispatch_MediaTek.so is currently missing. I think it is best to remove those commands for now as you suggested.
|
I have submit a new change to clean up the tests, and can you adjust your PR? |
…licit benchmark signatures, and desktop benchmarks
…licit benchmark signatures, and desktop benchmarks
|
Thanks for pointing out setup_test_env! I modified accordingly for both Parakeet and Whisper-Tiny. I also made a few tweaks while testing: I added the --signature-key flags so we actually hit the encode and decode paths separately, as you mentioned. I also added desktop benchmarks since they are working now. Lastly, I kept the compile command in, but removed the run and benchmark commands for now. Tested everything locally and all 19 cases passed. Kindly let me know if anything else needs to be modified. Thank you! |
smilingday
left a comment
smilingday
left a comment
There was a problem hiding this comment.
Thanks for catching up latest changes
2 participants
Adding CLI test scripts for the whisper_tiny and parakeet models.