An embedding model returns hundreds of numbers for a piece of text. It is tempting to label the first number “happiness,” the second “formality” and the third “importance.” Ordinary learned embeddings do not arrive with that kind of human dictionary.
TensorFlow’s introduction to word embeddings describes dense numerical vectors whose values are learned during training. Their length is a design choice. The representation can support useful relationships without assigning a simple named concept to every coordinate.
A coordinate is part of a representation
Think of a toy two-number vector such as [0.2, -0.7]. Without knowing the model and how it was trained, those values do not tell you that a sentence is 20 percent positive or 70 percent negative. They are coordinates in that model’s space, not calibrated percentages.
Similarity is usually computed from vectors as a whole using an appropriate measure. Even then, a high similarity score does not establish that two statements agree or are factually correct. The model and task determine what kinds of relationships are useful.
Keep vectors from compatible systems
Two models can output the same number of dimensions without using a shared coordinate system. Do not mix their vectors in one similarity index merely because the lengths match. Record the model identity, version and preprocessing used to produce them.
For an invented search system, documents embedded with one model and queries embedded with an unrelated model may produce meaningless comparisons. A migration needs a planned re-embedding and evaluation process, not just a change to a model name in configuration.
Test the use, not the mystique
Evaluate the retrieval or classification task with examples that matter to your users. Include close-looking texts that differ in an important detail. Inspect failures rather than treating an impressive dimensional count as a quality guarantee.
A visual plot can help explore a model, but reducing hundreds of coordinates to two also loses information. Use it as a view of the representation, not a complete map of meaning or a direct picture of how a person thinks.