Embedding Models Measure in Peculiar Ways

Juri Opitz, Andrianos Michail1 min readarXivSep 17, 2026

Key Takeaway for Students

Embedding spaces define notions of semantic similarity and distance. We study whether those embeddings reflect physical measurements of mass, distance, time and volume, which admit a unique, objective notion of semantic equivalence and distance. We find that physical measurement is only weakly modeled in the embedding space, and that instead quite peculiar measurement patterns

Reading in Plain English mode — toggle below to switch

Read original on arXiv

Career Insight

medium demand

Why it matters

This research highlights a fundamental limitation in how current embedding models represent objective, physical concepts. Understanding these peculiarities is crucial for ML engineers building systems that rely on accurate semantic understanding, especially in scientific or engineering domains. It suggests that relying solely on embedding similarity for such concepts can lead to flawed interpretations and model behavior.

Skills to learn

Evaluating embedding space properties beyond common benchmarksDesigning experiments to probe semantic representation in embeddingsTechniques for debiasing embedding spacesAdvanced vector space analysisUnderstanding limitations of current embedding models

Interview questions

Describe methods to evaluate the semantic fidelity of an embedding space for objective concepts like physical measurements.
If an embedding model shows strong influence from superficial string similarity, what architectural or training modifications could you propose to mitigate this?
Design a system to continuously monitor and recalibrate the semantic alignment of embeddings used in a critical application, specifically for concepts where objective ground truth exists.

Relevant for ~18 months · generated by the enrichment pipeline