Semantic Search That Understands What People Meant

The clearest sign a product needs semantic search is a support inbox full of "I could not find X" for things that are definitely there. Users searched for what they meant; the search matched on what they typed; the two did not overlap.

Semantic search closes that gap by matching on meaning rather than characters. It is one of the cheapest AI features to add, one of the most immediately noticeable to users, and one of the most frequently over-bought — plenty of products would be better served by fixing their existing keyword search first.

This is the decision framing rather than the implementation. If you have already decided and want the technical version, that is the pgvector post.

Where does keyword search break down?#

Four specific failures, and they account for most "I could not find it" reports.

Different words, same meaning#

A user searches "cancel my subscription"; the help article says "end your plan". No word overlaps. Keyword search returns nothing, or worse, returns something irrelevant that happens to contain "cancel".

Questions rather than terms#

People increasingly type whole questions, because that is what they have learned to do everywhere else. "Why is my invoice higher this month" tokenises into common words that match nearly everything, so relevance collapses.

Concepts with no canonical name#

A user wants to know about "the thing where it charges you again if you forget to cancel". There is no keyword for that. There is a document about automatic renewal, and only meaning-based matching connects the two.

Long-tail phrasing#

The same intent arrives in fifty different phrasings, and keyword search rewards whichever one happens to match your documentation's vocabulary. That is a lottery your users did not know they entered.

What does semantic search actually do?#

It converts text into a numerical representation of its meaning, then finds stored text whose representation is close by. Two passages about the same idea sit near each other even when they share no vocabulary.

That is the entire mechanism, and its consequence is that "cancel my subscription" finds "end your plan" without anyone having written a synonym list. It also means the system handles phrasings nobody anticipated, which is where the maintenance saving comes from.

The important limitation is the mirror image: it is fuzzy where you sometimes need exact. A vector has no special respect for the literal string INV-40219, so a search for that invoice number can return five documents about invoicing and not the one you wanted.

It is worth being clear that this is not the model understanding your content in any deep sense. It is a similarity measure over a representation learned from a great deal of text, and it is very good at the specific job of noticing that two passages are about the same thing. That narrowness is a feature — there is no generation step, nothing is being invented, and the failure mode is returning a less relevant document rather than a confidently wrong statement.

Should you use semantic or keyword search?#

Both, nearly always. They fail in opposite directions, which is why running them together outperforms either.

The practical answer is to run both, merge the candidate sets, and rank the union. On Postgres this is one extra column and one extra index rather than a second system, which is what makes hybrid the sensible default rather than an advanced option.

Pure semantic search is worth considering only when your content genuinely has no exact identifiers — a blog, a knowledge base of prose. The moment SKUs, order numbers or error codes are involved, dropping keyword matching is a downgrade users notice immediately.

What does it cost to add?#

Less than most AI features, because there is no generation step. You are not paying a model to write anything.

The spread in the build figure is almost entirely about your content, not the feature. Clean structured content indexes quickly; a corpus of scanned PDFs and inconsistent exports needs a parsing phase that can equal the rest of the work.

Compared to a full chat assistant, this is roughly half the cost and a third of the ongoing spend, because generation is where both the money and the risk concentrate.

How long does it take to build?#

One and a half to three weeks for a product with reasonably clean content, and the phases are predictable.

A few days on ingestion and chunking, which is where quality is decided. A few days on the hybrid query, ranking and permission filtering. A few days on the interface, which matters more than people expect — semantic results feel wrong if the UI does not show why something matched.

The variable is content preparation. I now parse twenty representative documents during scoping precisely because that is where estimates go wrong, and it is an afternoon that has repeatedly changed a quote in both directions.

What is genuinely quick is the search itself. Embedding a corpus, writing the hybrid query and returning ranked results is a couple of days for anyone who has done it before. What takes the remaining time is everything around it: getting the chunks right, filtering by permission correctly, building an interface that explains its matches, and producing the labelled set that tells you whether any of it worked.

That ratio is worth knowing when comparing quotes. A quote that prices only the search mechanism will be low and will land you with the other 70% of the work unbudgeted, which then arrives as change requests during the build.

How do you know if it is working?#

Not by trying it. Semantic search always returns something, ranked confidently, and the results look plausible whether or not they are right.

A labelled query set#

Fifty real searches, each labelled with which document should be returned. Then measure how often the correct document appears in the top five. That number is the whole quality story, and it takes an afternoon to produce.

Zero-result and abandoned searches#

Track searches where nothing scored above your relevance floor, and searches where the user clicked nothing. Both are lists of things people expected to find and did not, and they usually point at missing content rather than bad retrieval.

The support inbox#

The original signal. If "I could not find" tickets do not fall after launch, the search is not the thing that was broken.

What does the interface need to get right?#

More than a results list, because semantic matches are not self-explanatory the way keyword matches are.

When keyword search returns a document, the user can see their word in it. When semantic search returns a document with none of their words, it looks like a mistake unless you show why it matched — a snippet around the relevant passage, with the conceptually matching sentence highlighted.

Show the match location, not just the document. Deep-link into the position where the relevant passage sits, which is why storing character offsets at index time matters. Opening a forty-page document at page one and expecting the user to search again defeats the purpose.

And keep a visible fallback. When results are weak, saying so and offering browse or contact options is better than presenting four irrelevant documents with confidence.

Ranking presentation deserves a moment too. Semantic scores are not intuitive — a 0.82 and a 0.79 may be equally good matches, or one may be irrelevant, depending on the query. Exposing raw scores invites users to interpret a number that does not mean what they think. Show ordering and a relevance floor, not the arithmetic behind it.

Filters remain valuable, and people frequently drop them when adding semantic search on the assumption that meaning-matching replaces them. It does not. Date ranges, document types and categories are how users narrow a result set they can already see, and combining a semantic query with structured filters is one of the clearest advantages of keeping everything in one database.

How does it handle permissions?#

This is the requirement that quietly decides your architecture, and it is worth settling before anything is built.

Real search is never "find similar content". It is "find similar content this user may see". If embeddings live in your main database, that is a WHERE clause the query planner evaluates alongside the vector comparison — ordinary, correct, and fast.

If they live in a separate service, it becomes an application-side problem with no good solution. Over-fetch and filter afterwards and you sometimes return fewer results than requested; push permissions into the vector store and you now maintain an authorisation model in two systems that will drift.

That single consideration is why I keep embeddings in Postgres by default on the stack I build from, and why "we will add permissions later" is the most expensive sentence in a search project.

What about typos and misspellings?#

Semantic search handles them better than keyword search and not as well as people expect.

Minor misspellings usually still embed near the correct meaning, so "subscripton" finds subscription content. Badly mangled words do not, and neither approach recovers a query that has lost its meaning.

Where it matters, Postgres trigram matching handles fuzzy string similarity and slots into the same hybrid merge as a third candidate source. This is worth adding when your content is full of proper nouns — product names, people, places — where users routinely get the spelling nearly right.

What do you actually index?#

Not whole documents. The unit you index is the unit you can return, and returning a forty-page manual because page thirty-one is relevant helps nobody.

Chunk on the structure the content already has#

Headings, sections, list groups. A section under a heading is a unit of meaning somebody already decided was coherent, which is exactly what you would otherwise try to reconstruct statistically. Where structure is unreliable, fixed chunks with modest overlap are the fallback.

Chunks that are too large dilute meaning — a passage covering five topics is not strongly similar to a question about any one of them. Chunks that are too small arrive without context, so the reader sees an answer fragment with no idea what it refers to.

Index the title and heading path with the chunk#

Prefixing each chunk with its document title and heading trail measurably improves matching, because it supplies context the chunk text alone lacks. A paragraph about limits means something different under "Rate limits" than under "Refund limits", and the raw paragraph may not say which.

Decide what not to index#

Navigation text, boilerplate footers, cookie notices and legal preambles will match queries and waste result slots. Excluding them at index time is cheaper than filtering them at query time and produces visibly better results immediately.

How does it fit into an existing product?#

More easily than most AI features, because it replaces something rather than adding a new surface. Users already understand a search box.

The usual sequence is to run it alongside the existing search first. Index the content, build the hybrid query, and compare results against the current implementation on real queries before changing anything users see. That comparison is also your evaluation set, so the work does double duty.

Roll it out behind a flag to a proportion of traffic and watch zero-result rate and click-through. This is the same feature-flag discipline any other change gets — semantic search is not special enough to skip it, and the metrics are unusually clear.

One integration detail worth planning: search is often the first feature to expose a permissions model nobody had formalised. Content that was reachable only by people who knew the URL becomes findable, and that occasionally surfaces documents somebody assumed were private. Audit what becomes visible before the rollout rather than after.

When is semantic search the wrong answer?#

Three situations where I say so before quoting.

Your keyword search is badly configured#

A surprising number of "we need semantic search" conversations turn out to be a default search with no stemming, no field weighting and no synonym list. Fixing that is days rather than weeks and frequently resolves most of the complaints.

The content is the problem#

If the article people cannot find does not exist, or exists in a form nobody would recognise, search is not the constraint. This is diagnosable from the zero-result log, and it is a content project wearing a search costume.

Your corpus is small and browsable#

Below a few hundred documents, good navigation and filters often beat search entirely. People find things by browsing a well-organised structure, and building search for a corpus that fits on two screens is effort spent in the wrong place.

Half the products that ask for semantic search need better content and a properly configured keyword index. The half that genuinely need it usually needed it a year earlier.

What should you ask a developer about it?#

Four questions that separate an implementation that will work from one that will disappoint.

1. Is it hybrid, or semantic only? Semantic-only fails on exact identifiers, and that failure is the one users report most sharply.
2. Where does permission filtering happen? In the database query is correct. In the application after retrieval is a bug waiting to surface.
3. How is quality measured? If the answer is not "a labelled query set", quality is an impression and regressions will be invisible.
4. What happens when content changes? Re-embedding on write, with staleness handled explicitly, or the index silently rots.

The last one is the most commonly missed. A search index that drifts out of sync with the content produces confidently wrong results with no error anywhere, and it degrades over months rather than failing visibly.

A fifth question, if the answer to the first four is convincing: what happens when you want to change the embedding model? The correct answer involves a stored model identifier and a planned reindex, not a shrug. Models improve, and a system built without a migration path locks you into whichever one was current when it was built.

Conclusion#

Semantic search fixes a real and specific problem: users searching for meaning against a system matching characters. It is among the cheapest AI features to add, it needs no generation, and on most stacks it runs inside the database you already have.

Run it hybrid so exact identifiers keep working. Filter by permission inside the query rather than around it. Measure with a labelled set rather than by trying it. Show why each result matched, and deep-link to the passage rather than the document.

And check first that the problem is search rather than content or configuration. The cheapest version of this project is the one where a properly tuned keyword index and three missing help articles resolve the complaints entirely — and that is worth an afternoon of investigation before anyone quotes for weeks of work.

There is also a natural next step worth knowing about, because it changes how you build this one. Semantic search is the retrieval half of a question-answering assistant. If answering questions directly is plausibly on the roadmap, index with that in mind — store character offsets, keep chunks coherent, build the labelled query set — and the later project becomes an addition rather than a rebuild. Skip those and the assistant starts by redoing the indexing work you already paid for, which is the most common avoidable expense in this area.