Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers
huggingface ·
Sentence Transformers v6.0 introduces MultiVectorEncoder, enabling training and fine-tuning of ColBERT-style late-interaction models. The blog post walks through the required components, including model initialization, datasets, loss functions, training arguments, evaluators, and the trainer. It explains how to either fine-tune an existing multi-vector checkpoint or build a new one from a base transformer. A key finding is that using pre-supervised 'unsupervised' checkpoints as starting points leads to better domain adaptation than fully fine-tuned checkpoints, which may regress. The post also demonstrates handling long documents by adjusting length limits and adding a punctuation skiplist. The author's fine-tuned model, trained for 14.5 hours on a single RTX 3090, outperformed all general-purpose dense, sparse, lexical, and multi-vector models on a medical retrieval evaluation.
huggingface