Preparing data for supervised fine-tuning Part 1: Formatting and quality
aws ·
This article, first in a two-part series on supervised fine-tuning (SFT) data preparation, explains how data quality and formatting shape model behavior. It distinguishes SFT from continued pre-training and reinforcement fine-tuning, noting that SFT teaches a model how to respond without adding new knowledge. The author emphasizes that accurate, diverse, and consistent examples matter more than quantity, citing studies showing small curated datasets can outperform larger ones. Practical quality steps include auditing for correctness, semantic coverage, difficulty range, edge cases, consistency, deduplication, and safety screening. The formatting section covers using system prompts, structuring data as conversational JSONL with strict role alternation, including reasoning traces for reasoning models, and supporting tool-calling and multimodal formats. The post advises matching training-time formatting to inference usage to avoid distribution mismatches and warns that enabling reasoning during training on non-reasoning data can harm capabilities. A second post will address advanced strategies like readiness evaluation, subset selection, augmentation, and data mixing.
aws