When people talk about modern search, the focus often goes straight to the retrieval technology: embeddings, vector databases, BM25, semantic search.

But after working through the search architecture for a product catalog, I found the more important question comes earlier:

What is the user actually trying to find?

A natural-language query and a product identifier may enter through the same search box, but that does not mean they should take the same path through the architecture. That became the central design principle for us:

Search architecture should start with intent, then choose the retrieval strategy that fits it.

These two searches may come through exactly the same search box:

“waterproof black jacket for winter” vs. “JCK-4521-BLK”

But they are not the same search problem.

1. The first is asking about meaning.
2. The second is asking about identity.

And treating both the same way can make search worse, not better.

Dense search: when meaning matters

For natural-language queries, semantic search is extremely useful.

We use an embedding model to convert the query into a dense vector: essentially a numerical representation of its meaning. Products have already been embedded in the same way. The vector database can then retrieve products whose vectors are closest to the query, even when they don’t contain exactly the same words.

That means someone can describe what they want rather than having to know exactly how the catalog describes it.

This is one of the major advantages of vector search. But it is only one kind of retrieval.

Sparse search: when the words themselves matter

Now consider a SKU: JCK-4521-BLK

There isn’t much semantic interpretation required.

The user probably isn’t asking: “What products mean something similar to JCK-4521-BLK?”

They are saying: “Find this product.”

This is where the sparse side of the search becomes important. We tokenize the query and use BM25-style retrieval. BM25 rewards matching terms based on several factors, but one particularly important part is IDF — inverse document frequency.

IDF asks, roughly: How rare is this term across the catalog?

A common word carries relatively little information. A SKU that occurs on only one product carries a lot. So an exact identifier naturally becomes a very strong relevance signal without turning the entire search experience into an exact-match-only lookup. Exactness is heavily rewarded, but the retrieval mechanism still has room to rank other matching tokens when appropriate.

Hybrid search means keeping both strengths

For normal natural-language searches, we don’t choose between semantic and keyword retrieval. We use both.

The query produces two separate representations:

• a dense vector for semantic similarity
• a sparse vector for lexical relevance

These are two independent retrieval channels.

• The dense vector is compared against the semantic representation of the catalog.
• The sparse vector is evaluated using BM25.

They are not run through each other. And we don’t mathematically merge the two vectors into a new vector. Instead, each retrieval channel produces its own ranked list. Then we combine the rankings.

Why we fuse rankings instead of scores?

A semantic search might produce a cosine similarity score. BM25 produces a completely different kind of score. A cosine score of 0.83 and a BM25 score of 14.2 cannot meaningfully be averaged simply because both happen to be numbers.

They represent different things. So we use Reciprocal Rank Fusion, or RRF.

RRF cares about where a result appears in each ranked list rather than trying to normalize two unrelated scoring systems.

If a product ranks highly in both semantic and keyword retrieval, it becomes a strong candidate. If it appears strongly in only one channel, it can still survive.

The important point is: We fuse the rankings, not the vectors.

That allows both retrieval systems to keep doing what they are good at.

But hybrid does not mean every query needs both

This was one of the more important design decisions.

For queries that look like product codes or identifiers, we deliberately skip the dense retrieval path. They go through the sparse channel only. Why?

We saw this very clearly with product-code searches. BM25 was putting the exact product where we expected it. Adding the dense result could introduce an unrelated product high enough in the ranking to compete with it.

Adding semantic similarity can introduce results that happen to appear close in the embedding space but are irrelevant to what the user was actually asking for. So the architecture becomes query-aware:

That is a small distinction in the implementation. But I think it represents a much larger architectural principle:

Adding another retrieval mechanism does not automatically improve retrieval.

Sometimes good architecture means knowing when not to invoke one.

Filters solve a different problem

There is another distinction that is easy to blur:

retrieval vs. filtering.

Imagine a catalog with one million items and someone searches for:

“modern black dining table”

but only wants products available in Toronto. You could put “Toronto” into the text and hope the ranking algorithm gives Toronto products more weight.

But that turns a hard business constraint into a relevance preference. Those are different things.

If the requirement is:

Only products available in Toronto are eligible

then Toronto should be a filter. The vector database receives the filter along with the retrieval query and searches within the eligible subset.

Conceptually:

• Filter → define what is eligible
• Retrieval → determine what is relevant
• Ranking → determine what comes first

In our implementation, filters are attached to the retrieval branches themselves. They aren’t applied afterward to clean up the top results.

That means if there are one million products and 50,000 satisfy the filter, the search retrieves the best candidates from that eligible space rather than retrieving globally and throwing incompatible products away afterward.

I like thinking about metadata this way:

Metadata should reflect the retrieval contract, not become a copy of the entire domain model.

Store the structured information the retrieval system actually needs to constrain, rank, facet, or return. Not everything simply because it exists.

The same product doesn’t need the same representation everywhere

Another lesson came from indexing.

It is tempting to create one searchable string for every product and use it for everything. We ended up doing something different. The semantic representation contains information useful for understanding the product:

• product name
• useful description
• brand
• category
• format

The sparse representation contains information useful for identifying the product:

• SKU
• UPC
• product name
• description
• brand
• attributes
• format

The difference is intentional. Putting raw identifiers into an embedding doesn’t necessarily help the model understand the product.

But removing those identifiers from sparse retrieval would weaken one of its biggest advantages. So the same product gets represented differently depending on the retrieval objective.

One representation answers:

What is this product about?

The other helps answer:

Which exact product or terms are being requested?

Production search also has operational constraints

There were implementation lessons beyond retrieval quality.

For example, some real product descriptions were more than 12,000 characters long. Sending multiple descriptions of that size through a CPU-bound embedding service could overwhelm the service and affect other indexing work.

So the semantic representation was capped. The sparse representation wasn’t.

That tradeoff made sense because semantic retrieval does not necessarily need every word of a long description to capture its meaning.

Keyword retrieval, on the other hand, benefits from retaining exact token coverage.

This is the part of search architecture that doesn’t usually appear in the simple diagrams.

The architecture has to account not only for relevance, but also for latency, compute, indexing cost, pagination, filtering, faceting and failure modes.

The bigger lesson

Vector search, BM25, structured filtering, and fusion solve different parts of the retrieval problem. None replaces the others.

They answer different questions.

• Dense retrieval is good at: What does this mean?
• Sparse retrieval is good at: Which important terms actually match?
• Filters answer: Which records are even eligible?
• And fusion answers: How should evidence from different retrieval systems influence the final ranking?

That is why the more useful architecture is:

intent → constraints → retrieval strategy → ranking

And this goes well beyond product search.

The same issue appears in enterprise search and RAG.

• Someone asking: “What is our international travel policy?” is probably asking a semantic question.
• Someone asking: “Find contract MSA-2026-1847.” probably isn’t.

Both may arrive through the same AI interface. That doesn’t mean they should take the same path underneath it.

Good search isn’t about putting everything into a vector database.

It is about understanding what kind of question the user is asking — and selecting the retrieval mechanism that fits it.