Skip to content

Commit 0e56732

Browse files
smilingdaysmilingday
committed
Update Colab demo notebook and test scripts
1 parent e330921 commit 0e56732

3 files changed

Lines changed: 244 additions & 5 deletions

File tree

‎test_scripts/litert_cli.ipynb‎

Lines changed: 240 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,240 @@
1+
{
2+
"cells": [
3+
{
4+
"id": "90c3deda",
5+
"cell_type": "markdown",
6+
"source": [
7+
"# LiteRT CLI Demo\n",
8+
"\n",
9+
"This notebook demonstrates how to use the `litert-cli` tool to convert a PyTorch model, quantize it and run it."
10+
],
11+
"metadata": {},
12+
"execution_count": null
13+
},
14+
{
15+
"id": "19270446",
16+
"cell_type": "markdown",
17+
"source": [
18+
"## 🛠️ 1. Environment Setup \u0026 Installation"
19+
],
20+
"metadata": {},
21+
"execution_count": null
22+
},
23+
{
24+
"id": "d34739df",
25+
"cell_type": "code",
26+
"source": [
27+
"# Install required packages\n",
28+
"!pip install torch torchvision\n",
29+
"\n",
30+
"# Install litert-cli\n",
31+
"!pip install -q -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple litert-cli==0.1.1.dev23\n",
32+
"\n",
33+
"# 'litert compile' depends on Clang, and below make sure your Clang has version `18.x.x` or above\n",
34+
"!wget https://apt.llvm.org/llvm.sh\n",
35+
"!chmod +x llvm.sh\n",
36+
"!sudo ./llvm.sh 18 all"
37+
],
38+
"metadata": {},
39+
"execution_count": null
40+
},
41+
{
42+
"id": "1dd1eb34",
43+
"cell_type": "markdown",
44+
"source": [
45+
"## 📝 2. Prepare PyTorch Model Script"
46+
],
47+
"metadata": {},
48+
"execution_count": null
49+
},
50+
{
51+
"id": "5e40f66c",
52+
"cell_type": "code",
53+
"source": [
54+
"%%writefile resnet18.py\n",
55+
"import torch\n",
56+
"import torchvision\n",
57+
"\n",
58+
"def get_model() -\u003e torch.nn.Module:\n",
59+
" model = torchvision.models.resnet18(\n",
60+
" weights=torchvision.models.ResNet18_Weights.IMAGENET1K_V1\n",
61+
" )\n",
62+
" model.eval()\n",
63+
" return model\n",
64+
"\n",
65+
"def get_args() -\u003e tuple[torch.Tensor, ...]:\n",
66+
" return (torch.randn(1, 3, 224, 224),)"
67+
],
68+
"metadata": {},
69+
"execution_count": null
70+
},
71+
{
72+
"id": "427dca8a",
73+
"cell_type": "markdown",
74+
"source": [
75+
"## 🔄 3. Model Conversion (PyTorch -\u003e LiteRT)"
76+
],
77+
"metadata": {},
78+
"execution_count": null
79+
},
80+
{
81+
"id": "578cc653",
82+
"cell_type": "code",
83+
"source": [
84+
"# Convert PyTorch ResNet18 model to LiteRT\n",
85+
"!litert convert resnet18.py --output resnet18"
86+
],
87+
"metadata": {},
88+
"execution_count": null
89+
},
90+
{
91+
"id": "606efd4f",
92+
"cell_type": "markdown",
93+
"source": [
94+
"## 📉 4. Model Quantization"
95+
],
96+
"metadata": {},
97+
"execution_count": null
98+
},
99+
{
100+
"id": "c589c711",
101+
"cell_type": "code",
102+
"source": [
103+
"# Quantize the ResNet18 model\n",
104+
"!litert quantize resnet18/resnet18.tflite --type int8_dynamic --output resnet18/resnet18_int8_dynamic.tflite\n",
105+
"!litert quantize resnet18/resnet18.tflite --type int8_weight_only --output resnet18/resnet18_int8_weight_only.tflite"
106+
],
107+
"metadata": {},
108+
"execution_count": null
109+
},
110+
{
111+
"id": "b3ae9ddc",
112+
"cell_type": "markdown",
113+
"source": [
114+
"## 🚀 5. Run Inference\n",
115+
"### 🖥️ 5.1 CPU Inference"
116+
],
117+
"metadata": {},
118+
"execution_count": null
119+
},
120+
{
121+
"id": "ea93a1db",
122+
"cell_type": "code",
123+
"source": [
124+
"# Run Inference on Desktop (Colab VM CPU)\n",
125+
"!litert run resnet18/resnet18.tflite --desktop --cpu --iterations 1\n",
126+
"!litert run resnet18/resnet18_int8_dynamic.tflite --desktop --cpu --iterations 1"
127+
],
128+
"metadata": {},
129+
"execution_count": null
130+
},
131+
{
132+
"id": "9d748475",
133+
"cell_type": "markdown",
134+
"source": [
135+
"### 🎮 5.2 GPU Inference"
136+
],
137+
"metadata": {},
138+
"execution_count": null
139+
},
140+
{
141+
"id": "00b026b2",
142+
"cell_type": "code",
143+
"source": [
144+
"# Run Inference on Desktop (Colab VM GPU)\n",
145+
"!litert run resnet18/resnet18.tflite --desktop --gpu --iterations 1\n",
146+
"!litert run resnet18/resnet18_int8_dynamic.tflite --desktop --gpu --iterations 1"
147+
],
148+
"metadata": {},
149+
"execution_count": null
150+
},
151+
{
152+
"id": "eaa972bc",
153+
"cell_type": "markdown",
154+
"source": [
155+
"## ⚙️ 6. NPU Offline Compilation (AOT)"
156+
],
157+
"metadata": {},
158+
"execution_count": null
159+
},
160+
{
161+
"id": "d0a61181",
162+
"cell_type": "code",
163+
"source": [
164+
"# Compile the model for qualcomm NPU: SM8750\n",
165+
"# TIP: Only support running on Linux and it might take a few minutes, given download large SDKs from SOCs.\n",
166+
"# TIP: It depends on Clang, and please make sure your Clang has version `18.x.x` or above when running\n",
167+
"# `clang --version`. To update, choose one of below: \n",
168+
"# a) `sudo apt update \u0026\u0026 sudo apt upgrade -y`\n",
169+
"# b) wget https://apt.llvm.org/llvm.sh\n",
170+
"# chmod +x llvm.sh\n",
171+
"# sudo ./llvm.sh 18 all\n",
172+
"#\n",
173+
"!litert compile resnet18/resnet18.tflite --target sm8750"
174+
],
175+
"metadata": {},
176+
"execution_count": null
177+
},
178+
{
179+
"id": "2da953c9",
180+
"cell_type": "markdown",
181+
"source": [
182+
"## 🏁 7. Benchmark in Google AI Edge Portal"
183+
],
184+
"metadata": {},
185+
"execution_count": null
186+
},
187+
{
188+
"id": "a9a19e48",
189+
"cell_type": "code",
190+
"source": [
191+
"# Please login into Google Cloud and make sure you have joined the EAP for Google AI Edge Portal\n",
192+
"# Check: https://ai.google.dev/edge/ai-edge-portal\n",
193+
"!pip install gcloud\n",
194+
"!gcloud auth login\n",
195+
"\n",
196+
"# Benchmark on Google AI Edge Portal\n",
197+
"# Please specify you own GCP project: --gcp-project \u003cyour-own-gcp-project-id\u003e\n",
198+
"# Or set a default environment variable: LITERT_GCP_PROJECT\n",
199+
"!litert benchmark model.tflite --gcp --device \"pixel 7\" --cpu --gcp-project aep-e2e-test\n",
200+
"!litert benchmark model.tflite --gcp --device \"pixel 7\" --gpu --gcp-project aep-e2e-test\n",
201+
"!litert benchmark model.tflite --gcp --devices \"pixel 7, sm-s931u1\" --gpu"
202+
],
203+
"metadata": {},
204+
"execution_count": null
205+
},
206+
{
207+
"id": "2685b076",
208+
"cell_type": "markdown",
209+
"source": [
210+
"## Advanced Usage\n",
211+
"\n",
212+
"### Android Deployment\n",
213+
"To run or benchmark on Android, you need an ADB connected device. These commands typically look like:\n",
214+
"```bash\n",
215+
"# Run on Android\n",
216+
"!litert run resnet18/resnet18.tflite --android --cpu\n",
217+
"\n",
218+
"# Benchmark on Android\n",
219+
"!litert benchmark resnet18/resnet18.tflite --android\n",
220+
"```\n",
221+
"These are not executed in this notebook as Colab does not have access to a physical Android device by default."
222+
],
223+
"metadata": {},
224+
"execution_count": null
225+
}
226+
],
227+
"metadata": {
228+
"kernelspec": {
229+
"display_name": ".venv",
230+
"language": "python",
231+
"name": "python3"
232+
},
233+
"language_info": {
234+
"name": "python",
235+
"version": "3.13.12"
236+
}
237+
},
238+
"nbformat_minor": 5,
239+
"nbformat": 4
240+
}

‎test_scripts/models/efficientnet.sh‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -115,9 +115,8 @@ fi
115115

116116

117117
# --- 5. Compile (AOT Compilation) ---
118-
# TODO: Add this back when we fix the NPU compile issue.
119-
# run_case "Compile: EfficientNet FP32 for Qualcomm sm8750 NPU" \
120-
# litert compile "$EFFICIENTNET_TFLITE" --target sm8750 --output-dir "$MODEL_DIR/efficientnet"
118+
run_case "Compile: EfficientNet FP32 for Qualcomm sm8750 NPU" \
119+
litert compile "$EFFICIENTNET_TFLITE" --target sm8750 --output-dir "$MODEL_DIR/efficientnet"
121120

122121
# --- 6. Visualize (Model Explorer) ---
123122
run_case "Visualize: Launch Model Explorer in the background" \

‎test_scripts/models/yamnet.sh‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -121,8 +121,8 @@ fi
121121

122122
# --- 5. Compile (AOT Compilation) ---
123123
# TODO: Add this back when we fix the NPU compile issue.
124-
# run_case "Compile: YamNet FP32 for Qualcomm sm8750 NPU" \
125-
# litert compile "$YAMNET_TFLITE" --target sm8750 --output-dir "$MODEL_DIR/yamnet"
124+
#run_case "Compile: YamNet FP32 for Qualcomm sm8750 NPU" \
125+
# litert compile "$YAMNET_TFLITE" --target sm8750 --output-dir "$MODEL_DIR/yamnet"
126126

127127
# --- 6. Visualize (Model Explorer) ---
128128
run_case "Visualize: Launch Model Explorer in the background" \

0 commit comments

Comments
 (0)