Skip to content

Commit b16e7a9

Browse files
Merge pull request #52 from google-ai-edge:smilingday-patch-1
PiperOrigin-RevId: 918466013
2 parents 0d8565c + c9743bf commit b16e7a9

1 file changed

Lines changed: 35 additions & 36 deletions

File tree

README.md

Lines changed: 35 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,10 @@ platforms (desktop, mobile, or cloud).
1010
[Common commands](#-common-commands) | 📓 [Try Colab](#-try-colab) | 🌟
1111
[Quick demos](#-quick-demos) | 🤖 [Use in coding agent](#-use-in-coding-agent)
1212

13+
> [!NOTE] It's still an early preview under active development, thus has limited
14+
> platform and feature support, plus possible bugs. We appreciate your patience
15+
> and feedback to help us improve it. Welcome issues and PRs!
16+
1317
LiteRT CLI is built on top of [Google AI Edge](https://ai.google.dev/edge)
1418
stacks, including [LiteRT](https://github.com/google-ai-edge/LiteRT),
1519
[LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM),
@@ -18,10 +22,6 @@ stacks, including [LiteRT](https://github.com/google-ai-edge/LiteRT),
1822
[AI Edge Portal](https://ai.google.dev/edge/ai-edge-portal), and
1923
[Model Explorer](https://ai.google.dev/edge/model-explorer).
2024

21-
> [!NOTE] It's still an early preview under active development, thus has limited
22-
> platform and feature support, plus possible bugs. We appreciate your patience
23-
> and feedback to help us improve it. Welcome issues and PRs!
24-
2525
--------------------------------------------------------------------------------
2626

2727
## 🚀 Installation
@@ -343,7 +343,34 @@ litert benchmark model.tflite --gcp --device "pixel 7" --gcp-project "your-gcp-p
343343
litert benchmark model.tflite --gcp --devices "pixel 7, sm-s931u1" --gpu
344344
```
345345

346-
### 7. Visualize a model's architecture
346+
### 7. Run and benchmark a generative LLM model using LiteRT-LM CLI
347+
348+
`litert lm` command will utlitize `litert-lm`, and you can use the same command
349+
with `litert-lm`, for example, both `litert lm run` and `litert-lm run` or
350+
`litert lm benchmark` and `litert-lm benchmark` achieve the same results.
351+
352+
Please follow the
353+
[LiteRT-LM CLI guide](https://ai.google.dev/edge/litert-lm/cli) for detailed
354+
instructions.
355+
356+
```bash
357+
# Run a generative LLM model, and load from hugging face
358+
litert lm run \
359+
--from-huggingface-repo=litert-community/gemma-4-E2B-it-litert-lm \
360+
gemma-4-E2B-it.litertlm \
361+
--prompt="What is the capital of France?"
362+
363+
# Or load from a local LLM model file
364+
litert lm run ./my_model.litertlm
365+
366+
# Example with a custom prompt
367+
litert lm run ./my_model.litertlm --prompt "Hello, how are you?"
368+
369+
# Benchmark a generative LLM model
370+
litert lm benchmark ./my_model.litertlm
371+
```
372+
373+
### 8. Visualize a model's architecture
347374

348375
```bash
349376
# Open in Model Explorer graph
@@ -353,7 +380,7 @@ litert visualize model.tflite
353380
litert visualize --stop-all
354381
```
355382

356-
### 8. Import a local model
383+
### 9. Import a local model
357384

358385
```bash
359386
# Import a local file into the centralized cache
@@ -363,7 +390,7 @@ litert import my_model.tflite --model-ref my_model
363390
litert import ./my_model_dir --model-ref my_model --hf-id my_org_name/my_model
364391
```
365392

366-
### 9. List managed models
393+
### 10. List managed models
367394

368395
```bash
369396
# List all managed models
@@ -373,40 +400,12 @@ litert list
373400
litert list my_model
374401
```
375402

376-
### 10. Delete a managed model
403+
### 11. Delete a managed model
377404

378405
```bash
379406
# Delete a model from cache
380407
litert delete my_model
381408
```
382-
383-
### 11. Run and benchmark a generative LLM model using LiteRT-LM CLI
384-
385-
`litert lm` command will utlitize `litert-lm`, and you can use the same command
386-
with `litert-lm`, for example, both `litert lm run` and `litert-lm run` or
387-
`litert lm benchmark` and `litert-lm benchmark` achieve the same results.
388-
389-
Please follow the
390-
[LiteRT-LM CLI guide](https://ai.google.dev/edge/litert-lm/cli) for detailed
391-
instructions.
392-
393-
```bash
394-
# Run a generative LLM model, and load from hugging face
395-
litert lm run \
396-
--from-huggingface-repo=litert-community/gemma-4-E2B-it-litert-lm \
397-
gemma-4-E2B-it.litertlm \
398-
--prompt="What is the capital of France?"
399-
400-
# Or load from a local LLM model file
401-
litert lm run ./my_model.litertlm
402-
403-
# Example with a custom prompt
404-
litert lm run ./my_model.litertlm --prompt "Hello, how are you?"
405-
406-
# Benchmark a generative LLM model
407-
litert lm benchmark ./my_model.litertlm
408-
```
409-
410409
### 12. Clean up all caches
411410

412411
```bash

0 commit comments

Comments
 (0)