How AI Finds Meaning: Embeddings Explained

How does AI know "puppy" and "dog" mean almost the same thing? It measures distance. It's how search and "chat with your files" tools find the right page. This 500x Explained, a one-minute explainer drawn entirely in code, builds the map step by step: text becomes numbers, numbers become spots on a map, and search hands the AI whatever lands closest to your question.

Text becomes a list of numbers

An embedding model turns a piece of text into a vector: a long list of numbers. OpenAI's guide puts it plainly: "An embedding is a vector (list) of floating point numbers." By default, text-embedding-3-small gives 1,536 numbers and text-embedding-3-large gives 3,072.

The numbers shown for "cat" in the video are made up for the drawing. Real values are small decimals, and you'd never read them one by one.

Each list is a point

Think of each vector as a point in a space with 1,536 dimensions. We can only draw three, so the video shows a floor map with poles: two coordinates on the floor, one as height. The idea carries over. In OpenAI's words, "The distance between two vectors measures their relatedness," and small distances mean high relatedness. So animals end up near animals, vehicles near vehicles, fruit near fruit. The positions in the video are placed by hand to show that idea; a real model's map would look messier.

Your question becomes a point too

When you search, or "chat with your docs," your question is embedded the same way. The system finds the chunks of your documents whose vectors sit closest to it. OpenAI recommends cosine similarity for this, which compares the angle between two vectors: a small angle means a close meaning. Then, as Anthropic's write-up on retrieval describes it, "the most relevant chunks are added to the prompt sent to the generative model." That's retrieval-augmented generation, or RAG.

This is why "puppy" can find "dog." OpenAI's retrieval guide says semantic search "surfaces semantically similar results," even when they match few or no keywords. Different words, same neighborhood.

Try this

When an answer from your files looks wrong, the problem is often retrieval: the model answered from the wrong chunks. The tip from the video comes first, then two more:

1. Check the sources it cites. Many tools show the sources or passages they used. OpenAI's file search, for example, can return the search results with the response. If the right passage isn't there, the answer is unlikely to be right.
2. Use the names your documents use. Embeddings capture meaning well, and they can still miss exact terms. Anthropic's example is an error code like "TS-999": an embedding search might find content about error codes in general and miss the exact match.
3. Split big files into focused sections. Documents are cut into chunks before they're embedded (OpenAI's vector stores default to 800-token chunks). A chunk that's about one thing lands near the questions about that thing.

So a wrong answer often starts with the wrong neighbors.