Gradivex
  • Home
  • Academy
  • Arena
  • Visualizer
  • Models
Log in
NLP & Transformers/Embeddings/Lesson 4

The semantic space

Module 2 · Lesson 4 of 5 · 6 min read
Theory

Stop thinking words, start thinking geography

By now you have seen points and neighbors and arrows. Zoom out: the embedding space as a whole is a geography of meaning. Regions are topics. Distances are relatedness. Directions are relationships. Everything a corpus knows about how words relate, drawn as one enormous map.

This reframe matters because modern AI systems navigate this map as their core operation. Semantic search does not match keywords; it embeds your query and finds the nearest documents. Recommendation engines find items near your taste. RAG systems (a later protocol) retrieve knowledge by distance in exactly this kind of space.

In a large share of production AI, the map is the product. The model generates the coordinates; the business logic is "measure distances, return the closest".

Interactive

Four districts and one troublemaker

A richer map: emotions, animals, technology, food. Explore the neighborhoods, then click "mouse" and look closely at its neighbor list.

joyangerfearsadnesscatdogwolfhamsterlaptopserverkeyboardsoftwaremousepizzapastacheesebread
emotionsanimalstechnologyfood

Click any word to measure who lives nearby.

Theory

How distance is actually measured

The standard ruler in these spaces is cosine similarity: the angle between two vectors, ignoring their lengths. Two vectors pointing the same way score 1, unrelated directions score near 0. Length tends to track word frequency more than meaning, so the angle is the cleaner signal.

You do not need the formula in your pocket; you need the operational meaning:

  • cosine ≈ 0.9: same neighborhood, near-synonyms or tight topic partners.
  • cosine ≈ 0.5: same city, related domain, would co-occur in an article.
  • cosine ≈ 0.0: different planets.

One practical warning from real systems: "close" means RELATED, not SAME, and not even compatible. "love" and "hate" sit close (both emotions, shared contexts) while being opposites. A search system that treats proximity as agreement will cheerfully retrieve the exact opposite of what the user meant. The map tells you the district; it does not read the street signs for you.

Interactive

Check your understanding

"love" and "hate" score a high cosine similarity in most embedding spaces. Why, and what does it teach about the space?

Takeaway

What to remember

  • ✓The semantic space is a geography: regions = topics, distance = relatedness, direction = relationship.
  • ✓Semantic search, recommendations, and RAG are all "embed, then find nearest": the map is the product.
  • ✓Cosine similarity (the angle) is the standard ruler; length mostly reflects frequency.
  • ✓Close means related, NOT interchangeable: "love" and "hate" are neighbors.
  • ✓Ambiguous words like "mouse" expose the static space's flaw: one address per word, however many meanings it has.
Mentor-0

Stuck on something in this lesson? Ask.

0/1000

Every lesson is free to read. Log in to save your progress, complete lessons, and earn Compute and XP.

Log in free
Word2Vec and GloVeContextual embeddings
Gradivex

Understand AI from the inside, read it, see it, prove it.

Platform

AcademyVisualizerRoadmapsModels

Community

DiscordLeaderboard

Company

PrivacyTerms

© 2026 Gradivex. All rights reserved.