Skip to content

feat(smollm2): expand SmolLM2 demo script with advanced commands.#44

Merged
copybara-service[bot] merged 1 commit into
mainfrom
add-smollm2
May 19, 2026
Merged

feat(smollm2): expand SmolLM2 demo script with advanced commands.#44
copybara-service[bot] merged 1 commit into
mainfrom
add-smollm2

Conversation

@hvt4

@hvt4 hvt4 commented May 19, 2026

Copy link
Copy Markdown
Collaborator

Expand SmolLM2 demo script with advanced run, quantization, and registry commands

  • Adds dynamic weight-only INT4 quantization (weight_only_wi4_afp32) to SmolLM2 conversion.
  • Integrates advanced litert-lm run parameters (sampler configurations, speculative decoding).
  • Adds raw prompt inference (--no-template) with custom chat control tokens.
  • Demonstrates local model registry management (import, list, delete).
  • Integrates system instruction presets using a newly added default.py preset file.


# --- 6. Import Model to Registry ---
run_case "Import SmolLM2: Register model alias" \
litert-lm import "$MODEL_PATH" smollm2-alias

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for all litert-lm commands,
"litert lm" works as well. You can try it:
litert lm import
and verify.

@copybara-service copybara-service Bot merged commit 0bab170 into main May 19, 2026
15 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants