Build efficient agents with knowledge distillation on Amazon SageMaker AI
aws ·
This article demonstrates how to build efficient AI agents using sequence-level knowledge distillation (SeqKD) on Amazon SageMaker AI. The authors address the high costs of agentic workflows, which often require multiple API calls to large frontier models, by training a smaller 'student' model to mimic a larger 'teacher' model's behavior. Using GLM-5.3 as the teacher and Qwen3.5-4B as the student, they generate multi-step trajectories for a shopping assistant task and filter them for success using the ShoppingBench scorer. The student is fine-tuned via serverless supervised fine-tuning on SageMaker AI, preserving the teacher's reasoning steps alongside tool calls. Evaluation results show the distilled 4B model achieves an Absolute Success Rate comparable to the frontier teacher, significantly outperforming the base untrained model. This approach allows organizations to reduce inference costs by hosting a smaller model while maintaining high-quality agent performance. The post concludes that SeqKD serves as an effective first step, though harder tasks may eventually require on-policy reinforcement learning methods.
aws