Skip to content

how_to_build.md

Latest commit

 

History

History
30 lines (22 loc) · 892 Bytes

how_to_build.md

File metadata and controls

30 lines (22 loc) · 892 Bytes

Build & Deploy llama-cli to RK3588S

From the workspace root:

bash build_and_load_cpu.sh [model_name]

model_name filters which GGUF is uploaded (default: picks the most recently converted *.gguf).

That single command:

  1. Cross-compiles llama-cli for aarch64 using aarch64-linux-gnu-gcc
  2. Installs the binary and shared libs to llama.cpp/build-aarch64/install/
  3. Uploads to the board (khadas@192.168.1.58) under ~/programs/llama_cpu/:
    • llama-cli
    • libllama.so / libggml*.so
    • <model_name>-<QUANT>.gguf (from llama.cpp/models/, if present)

Examples

# Upload the most recently converted model
bash build_and_load_cpu.sh

# Upload a specific model
bash build_and_load_cpu.sh Qwen3-0.6B
bash build_and_load_cpu.sh Qwen2.5-0.5B-Instruct

Note: run convert_to_gguf.sh first to produce the GGUF model before deploying.