Is the RTX 4090 compatible?

[Halbert-l]

Dual RTX 4090 48GB graphics cards with a total VRAM of 96GB — can it boot up normally?

[AEON-7]

Hi @Halbert-l — short answer: the image won't run as-published on dual RTX 4090.

The image is built for TORCH_CUDA_ARCH_LIST=12.0+PTX (Blackwell sm_120/121a). RTX 4090 is Ada Lovelace sm_89. The kernels in this image are not in that architecture's PTX fallback path:

• FlashInfer's NVFP4 b12x backend is sm_121a only — RTX 4090 can't run NVFP4 natively (no FP4 tensor cores).
• The DFlash drafter runs as a separate vLLM model and would work on Ada, but the main body's NVFP4 kernels would fail on launch.

For dual RTX 4090 48GB (96GB total VRAM), your better paths are:

1. Use the BF16 model variant — AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored (~51 GB) fits across two 48 GB cards with --tensor-parallel-size 2. Run with stock vLLM v0.20.0+ (no need for our purpose-built image; the sm_121a patches are no-ops on Ada).
2. Use AWQ-Int4 quant — these load via vLLM's AWQ kernel which has Ada support. Not currently published by us (see AWQ 4bit model #4) but plenty of community AWQ Qwen3.6 builds exist.
3. DFlash on Ada is supported by upstream — the z-lab/Qwen3.6-27B-DFlash drafter works on any vLLM v0.20+ that supports the dflash speculative method, which Ada does.

If you want a one-shot config for dual RTX 4090, our other-hardware/rtx6000pro/docker-compose.yml is the closest reference (RTX PRO 6000 SE is sm_120; for sm_89 you'd swap to a stock vLLM image and the BF16 model). Happy to provide a tested compose file if you'd like — comment back with which model variant you'd prefer (BF16 vs AWQ) and I'll spin one up.

[Halbert-l]

First of all, thank you very much for your great help amid your busy schedule.
The vLLM on my server was upgraded to version 0.20.0 last night.
I plan to deploy the BF16 version model, and I have already downloaded AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16 locally on the machine.