Relational Embedding - cross encoders

[wassfila]

Problem Description

• Is Relational Embedding possible between two sentences with independent llm embedding of each ? Or does every sentence pair need an relation embedding pass from an llm ?
• underlying assemption that embeddings differentiation can withhold relational information (king - man + woman = queen) requires higher dimentions than what the original embedding space can offer. e.g. there are tons of ways to break a car all of which can relate two sentences.
• For a db with N sentences, if differenciation was possible, it will reduce the problem complexity to N llm embeddings calls, the rest is vectors comparision, while if differentiation is not possible, N! calls are required to derive relations between all db sentences.

Relational Sentence Embedding

It is possible to use ML to learn embeddings differentiation meanings, but ML with supervised learning cannot take advantage from the generic llm capabilities.

• https://ar5iv.labs.arxiv.org/html/2212.08802

Massive Text Embedding Benchmark

• https://github.com/embeddings-benchmark/mteb/tree/main
• https://huggingface.co/spaces/mteb/leaderboard

[wassfila]

this is mainly the concept of cross encoders

Cross encoders

https://www.sbert.netdocs/cross_encoder/usage/usage.html

contradiction, entailment,... https://www.sbert.netdocs/cross_encoder/pretrained_models.html