Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps
huggingface ·
This article presents a low-cost, public recipe for improving a small language model's structured-output compliance by fine-tuning LFM2.5-350M with Group Relative Policy Optimization (GRPO) using the TRL library. The entire process uses roughly 500 training samples and 100 steps, suitable for a free-tier GPU. Evaluation on the IFStruct benchmark before training gave a 22.6% pass rate, close to the reported 21.1%. After GRPO fine-tuning with LoRA adapters and three reward functions measuring JSON formatting, field count, and schema validation, the model's IFStruct score rose to 29.7%. The pipeline includes data augmentation to better match the evaluation distribution, then merges and converts the model to GGUF for local serving via llama.cpp. The authors emphasize that this shows small models can be task-specifically tuned to approach larger models' structured-output performance.
huggingface