Full fine-tuned model is not working, which LoRA fine-tuned model works fine.

[jingzhaoou]

System Info

Versions of relevant libraries:
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.26.4
[pip3] onnx==1.17.0
[pip3] onnxruntime==1.20.1
[pip3] optree==0.13.1
[pip3] pytorch-triton==3.0.0+72734f086
[pip3] torch==2.6.0a0+df5bbc09d1.nv24.11
[pip3] torch_tensorrt==2.6.0a0
[pip3] torchao==0.7.0
[pip3] torchaudio==2.5.0a0+265bc5c
[pip3] torchaudio==2.5.0a0+265bc5c
[pip3] torchprofile==0.0.4
[pip3] torchtune==0.0.0
[pip3] torchvision==0.20.0a0
[pip3] triton==3.1.0

Information

• The official example scripts
• My own modified scripts

🐛 Describe the bug

I have the same set-up as in the previous issue #831. I have

        finetuning.main(
            model_name=model_name,
            dataset='llamaguard_toxicchat_dataset',
            batch_size_training=1,
            batching_strategy='padding',
            enable_fsdp=self.enable_fsdp,
            use_peft=False,
            quantization=None,
            checkpoint_type=StateDictType.FULL_STATE_DICT,
            output_dir=self.output_dir.as_posix(),
            dist_checkpoint_root_folder='fsdp_models',
        )

This generated 3 pt files. I converted the last pt file to safetensor file and put it back to the original model and then ran things with vLLM. There was no crash. But the model outputspart of the inputs and then randomly listed S10, S111, etc. at the end. Apparently, the full fine-tuned model is not working.

In comparison, if I set use_peft=True, the LoRA fine-tuned model works fine in vLLM. Wonder if I miss anything in my setup. Appreciate your help!

Error logs

No error. As mentioned, just incorrect output.

Expected behavior

The model should output safe or unsafe with error code.