|
1 | 1 | # LiteRT CLI (Preview) |
2 | 2 |
|
3 | | -A convenient command-line toolkit to streamline LiteRT development workflow, |
| 3 | +A convenient command-line toolkit to streamline |
| 4 | +[LiteRT](https://ai.google.dev/edge/litert) related development workflow, |
4 | 5 | including converting, quantizing, compiling, managing, running, and benchmarking |
5 | 6 | LiteRT (TFLite) models on various hardware (CPU / GPU / NPU) across platforms |
6 | 7 | (desktop, mobile, or cloud). |
7 | 8 |
|
8 | | -## 🤖 Use in Coding Agent |
9 | | - |
10 | | -Add the LiteRT CLI skill |
11 | | -[`SKILL.md`](file:///.agents/skills/litert_cli/SKILL.md) into your AI coding |
12 | | -agent (like Antigravity or Gemini CLI) to enable prompts such as: |
13 | | - |
14 | | -* "Download LiteRT model `litert-community/efficientnet_b1` and run it on CPU" |
15 | | -* "Benchmark LiteRT model `litert-community/efficientnet_b1` on my Android |
16 | | - GPU" |
17 | | -* "Compile LiteRT model `litert-community/efficientnet_b1` for NPU target |
18 | | - `sm8750`" |
19 | | -* "Visualize LiteRT model `litert-community/efficientnet_b1`" |
20 | | - |
21 | | -The agent will automatically install the necessary tools, including Python |
22 | | -virtual environments, `litert-cli`, and all required dependencies. |
| 9 | +> [!NOTE] It's a still early preview release under active development, thus has |
| 10 | +> limited platform and feature support, plus possible bugs. We appreciate your |
| 11 | +> patience and feedback to help us improve it. |
23 | 12 |
|
24 | 13 | -------------------------------------------------------------------------------- |
25 | 14 |
|
26 | 15 | ## 🚀 Installation |
27 | 16 |
|
28 | | -We support installation using either **`uv`** (recommended for ultra-fast |
29 | | -dependency resolution) or standard **`pip`** within a virtual environment. |
| 17 | +You can install `litert-cli-nightly` from PyPI or from local clone. LiteRT CLI |
| 18 | +will install the dependencies on-demands, based on which commands to run, to |
| 19 | +speed up initial installation. |
30 | 20 |
|
31 | | -### Option 1: Use UV (Recommended) |
| 21 | +We support installation using either |
| 22 | +**[uv](https://docs.astral.sh/uv/getting-started/installation/)** (recommended |
| 23 | +for ultra-fast dependency resolution) or standard |
| 24 | +**[pip](https://pip.pypa.io/)** within a Python virtual environment. |
32 | 25 |
|
33 | | -`uv` is an extremely fast Python package manager written in Rust. |
| 26 | +#### Option 1: Use UV (Recommended) |
34 | 27 |
|
35 | | -#### 1. Create and Activate Virtual Environment |
| 28 | +`uv` is an extremely fast Python package manager written in Rust. |
36 | 29 |
|
37 | 30 | ```bash |
38 | | -# Create a virtual environment with Python 3.13 in the current directory. |
39 | | -# When meeting dependency resolution error, try to set environment variable: |
40 | | -# UV_INDEX_URL=https://pypi.org/simple |
| 31 | +# 1. Create a virtual environment with Python 3.13. |
| 32 | +# TIP: When meeting dependency resolution error, try to set environment variable: |
| 33 | +# export UV_INDEX_URL=https://pypi.org/simple |
41 | 34 | uv venv --clear --python=3.13 --seed |
42 | 35 | source .venv/bin/activate |
43 | | -``` |
44 | 36 |
|
45 | | -#### 2. Install `litert-cli` |
| 37 | +# 2. Install the package into the active virtual environment |
| 38 | +uv pip install litert-cli-nightly |
46 | 39 |
|
47 | | -##### 2a. Install from PyPI |
| 40 | +# 3. Run help command |
| 41 | +litert --help |
| 42 | +``` |
| 43 | + |
| 44 | +### Option 2: Use Standard Pip |
48 | 45 |
|
49 | 46 | ```bash |
50 | | -# Install the package into the active virtual environment |
51 | | -uv pip install -q litert-cli-nightly |
| 47 | +python3 -m venv .venv |
| 48 | +source .venv/bin/activate |
| 49 | +pip install -q litert-cli-nightly |
| 50 | +litert --help |
52 | 51 | ``` |
53 | 52 |
|
54 | | -##### 2b. Or Install from Local Clone (Recommended for Development) |
| 53 | +#### Option 3. Install from Local Clone (for development) |
55 | 54 |
|
56 | 55 | ```bash |
57 | | -# Clone the repository via SSH |
| 56 | +uv venv --clear --python=3.13 --seed |
| 57 | +source .venv/bin/activate |
58 | 58 | git clone git@github.com:google-ai-edge/LiteRT-CLI.git |
59 | | -# Or clone using your Personal Access Token (PAT) |
60 | | -git clone https://<your-access-token>@github.com/google-ai-edge/LiteRT-CLI.git |
61 | 59 | cd LiteRT-CLI |
62 | | - |
63 | | -# Install in editable mode inside the active virtual environment |
64 | 60 | uv pip install -e . |
65 | 61 | ``` |
66 | 62 |
|
67 | | -#### 3. Run Commands |
| 63 | +-------------------------------------------------------------------------------- |
| 64 | + |
| 65 | +## Quick Start |
| 66 | + |
| 67 | +### Try colab |
| 68 | + |
| 69 | +Try |
| 70 | +[LiteRT CLI Colab](https://github.com/google-ai-edge/LiteRT-CLI/blob/main/examples/litert_cli.ipynb) |
| 71 | +to explore different features quickly. |
68 | 72 |
|
69 | | -Check more comprehensive usage examples under the `examples/` directory. You can |
70 | | -run our automated demo drivers: - **Run all command demos**: |
71 | | -`./examples/run_commands.sh` - **Run all model demos**: |
72 | | -`./examples/run_models.sh` - **Run a specific model demo**: |
73 | | -`./examples/run_models.sh efficientnet` |
| 73 | +### Follow command help |
| 74 | + |
| 75 | +You can always follow `litert --help` or `litert {command} --help` to find how |
| 76 | +to use the CLI tool. Check detailed instructions for each command below. |
74 | 77 |
|
75 | 78 | ```bash |
76 | 79 | # Run help command |
77 | 80 | litert --help |
78 | 81 |
|
79 | 82 | # Download a LiteRT model |
80 | | -litert download litert-community/MobileNet-v3-large --file "*.tflite" --output mobilenet |
| 83 | +litert download --help |
| 84 | +litert download litert-community/efficientnet_b1 --file "*.tflite" --output efficientnet |
| 85 | + |
| 86 | +# Run and benchmark a LiteRT model on your devices |
| 87 | +litert run --help |
| 88 | +litert run efficientnet/efficientnet_b1.tflite --desktop --cpu |
| 89 | +litert benchmark --help |
| 90 | +litert benchmark efficientnet/efficientnet_b1.tflite --android --gpu |
81 | 91 | ``` |
82 | 92 |
|
83 | | --------------------------------------------------------------------------------- |
| 93 | +### Quick Demos |
84 | 94 |
|
85 | | -### Option 2: Use Standard Pip |
| 95 | +Check comprehensive usage examples under the `examples/` directory, which |
| 96 | +contains per-command demos and model-specific demos. |
86 | 97 |
|
87 | | -#### 1. Create and Activate Virtual Environment |
| 98 | +If you have cloned the repo, you can run the following commands to see the |
| 99 | +demos: |
88 | 100 |
|
89 | 101 | ```bash |
90 | | -# Create and activate a Python virtual environment |
91 | | -python3 -m venv .venv |
92 | | -source .venv/bin/activate |
93 | | -``` |
| 102 | +# Run all command demos |
| 103 | +./examples/run_commands.sh |
94 | 104 |
|
95 | | -#### 2. Install `litert-cli` |
| 105 | +# Run all model demos |
| 106 | +./examples/run_models.sh |
96 | 107 |
|
97 | | -##### 2a. Install from PyPI |
98 | | - |
99 | | -```bash |
100 | | -pip install -q litert-cli-nightly |
| 108 | +# Run a specific model demo |
| 109 | +./examples/run_models.sh efficientnet |
101 | 110 | ``` |
102 | 111 |
|
103 | | -##### 2b. Or Install from Local Clone |
| 112 | +## 🤖 Use in Coding Agent |
104 | 113 |
|
105 | | -```bash |
106 | | -# Clone the repository |
107 | | -git clone git@github.com:google-ai-edge/LiteRT-CLI.git |
108 | | -cd LiteRT-CLI |
| 114 | +Add the LiteRT CLI skill |
| 115 | +[`SKILL.md`]([file:///.agents/skills/litert_cli/SKILL.md]\(https://github.com/google-ai-edge/LiteRT-CLI/blob/main/.agents/skills/litert_cli/SKILL.md\)) |
| 116 | +into your AI coding agent (like Google Antigravity) and try prompts such as: |
109 | 117 |
|
110 | | -# Install in editable mode |
111 | | -pip install -e . |
112 | | -``` |
| 118 | +* "Download LiteRT model `litert-community/efficientnet_b1` and run it on CPU" |
| 119 | +* "Benchmark LiteRT model `litert-community/efficientnet_b1` on my Android |
| 120 | + GPU" |
| 121 | +* "Compile LiteRT model `litert-community/efficientnet_b1` for NPU target |
| 122 | + `sm8750`" |
| 123 | +* "Visualize LiteRT model `litert-community/efficientnet_b1`" |
113 | 124 |
|
114 | | -#### 3. Run Commands |
| 125 | +The agent will automatically install the necessary tools, including Python |
| 126 | +virtual environments, `litert-cli-nightly`, and all required dependencies. |
115 | 127 |
|
116 | | -```bash |
117 | | -# Run help command |
118 | | -litert --help |
| 128 | +-------------------------------------------------------------------------------- |
119 | 129 |
|
120 | | -# Download a LiteRT model |
121 | | -litert download litert-community/MobileNet-v3-large --file "*.tflite" --output mobilenet |
122 | | -``` |
| 130 | +### Verified Platforms |
123 | 131 |
|
124 | | -### Tested Platforms |
| 132 | +Verified in Python 3.13. |
125 | 133 |
|
126 | 134 | * **Host Machines**: |
127 | | - * Linux (Ubuntu) with Python 3.13 |
128 | | - * macOS (Apple Silicon) with Python 3.13 |
129 | | -* **Android Devices**: |
130 | | - * Qualcomm Snapdragon 8750 |
| 135 | + * Linux (Ubuntu) |
| 136 | + * macOS (Apple Silicon): don't support `litert compile` |
| 137 | + * Windows: partially supported |
| 138 | +* **Android**: |
| 139 | + * CPU, GPU |
| 140 | + * NPU: Qualcomm (supported), MediaTek (soon), Google Tensor (soon) |
131 | 141 |
|
132 | 142 | -------------------------------------------------------------------------------- |
133 | 143 |
|
@@ -316,4 +326,4 @@ litert lm run gemma-4-E2B-it.litertlm --prompt "Hello, how are you?" |
316 | 326 | ```bash |
317 | 327 | # Clean up model cache, etc. |
318 | 328 | litert clean |
319 | | -``` |
| 329 | +``` |
0 commit comments