Do we support late interaction models like ColBERT? Or Sparse models like BM25?

[S1LV3RJ1NX]

[akshayballal95]

ColBert is in the works and will be added soon.Current model that is late interaction based is ColPali.

For Sparse models you can use the Splade models as shown in the readme. These models will give you sparse vectors which you can then use for BM25 search.

[nauyan]

@akshayballal95 any update on ColBert?

[akshayballal95]

This has taken a while, but I have been waiting to pull the Colbert implementation to Candle. Until then, I will add the ONNX implementation for the next release, which is scheduled for this weekend. Is there any preference on which models you would like to have?

[akshayballal95]

@nauyan @S1LV3RJ1NX

Colbert support has been added. You can use most of the colbert models as long as they have ONNX weights. It currently supports text_batch embedding. In the later versions I will integrate it with the ingestion pipeline.

model:ColbertModel = ColbertModel.from_pretrained_onnx(
    hf_model_id="jinaai/jina-colbert-v2",
    path_in_repo="onnx/model.onnx",
)

model:ColbertModel = ColbertModel.from_pretrained_onnx(
     hf_model_id="answerdotai/answerai-colbert-small-v1",
    path_in_repo="onnx/model_fp16.onnx",
 )