Peculiarities of Embedding Space for Physical Measurements
This paper investigates how well embedding models represent physical measurements like mass, distance, time, and volume. It finds that these embeddings poorly reflect objective physical properties, instead showing peculiar patterns influenced by superficial string similarity.
Reading in Plain English mode — toggle below to switch
This paper investigates how well embedding models represent physical measurements like mass, distance, time, and volume. It finds that these embeddings poorly reflect objective physical properties, instead showing peculiar patterns influenced by superficial string similarity.
Imagine you have a map (embedding space) that's supposed to represent a real-world territory (physical measurements). This paper suggests that the map often distorts the territory, making things that are physically far apart seem close, or vice versa, based on how their names are spelled rather than their actual distances.
How Embeddings Represent Physical Concepts
- 11. Embeddings are generated for terms representing physical measurements (e.g., 'kilogram', 'meter', 'second').
- 22. The semantic similarity or distance between these embeddings is then analyzed.
- 33. This similarity is compared against the objective, physical relationships between the measurements.
- 44. The study observes that embedding similarity often aligns poorly with physical reality, being more influenced by textual features.
d(e₁, e₂)
This represents the distance between two embedding vectors, $e_1$ and $e_2$, in the embedding space. A smaller distance implies higher semantic similarity according to the model.
If an embedding model places 'meter' and 'kilometer' very close together because they share 'meter' as a substring, but places 'meter' and 'inch' far apart despite both being units of length, it demonstrates the superficial string similarity influence over true physical equivalence.
Embedding spaces struggle to capture objective physical measurement relationships, often prioritizing superficial textual features over true semantic equivalence.