Advancing Full-Text Search: Testing and Refining with Multi-User Platforms | July 06 2025, 04:35

I have developed expertise in full-text search testing. Essentially, it’s a turnkey multi-user platform that, given roughly 1000 queries and several search engine configurations, can produce reports with graphs, metrics, and conclusions by morning, showing why configuration A performs better than B, and here’s why. It calculates all those NDCG@k, MAP, precision, recall, and about a dozen other metrics. It uses LLM, but only at the final stage, after all the math is done.

So, here’s my question. I’m looking for someone who has faced the same issue in their project, to understand the demand and the ask.

The problem the system solves is defined as follows: there is a functional search for goods, documents – Solr, Coveo, Elasticsearch, Algolia – it doesn’t matter, and there are hypotheses on how to improve it, but there is also the fear that improving one aspect might break another. Well, my thing helps to see this in numbers and graphs, providing a conclusion with justification, including statistical significance and other metrics.

It also acts as a virtual search assessor. For each search result, it can give a rating, assessing how well each document matches the query. This is a very non-trivial task (especially for large documents), involving chunking, embeddings, LLM evaluation of relevant chunks, etc. Non-trivial, but it works.

It also can analyze search queries and break them into groups based on similarity. For instance, such segmentation might show that users sometimes separate the words forming a brand name with a space, and sometimes not. These different variants would be grouped together.

I would like to discuss this with someone who knows more about this topic than I do, someone who has/had such problems and has somehow solved them.

I currently feel like my product is unique in the market. Actually, it’s not even on the market yet. But I really don’t see anything similar out there. Maybe nobody needs it?

I won’t publically post screenshots yet. The picture is merely for attracting attention.

Please share if there might be relevant people in your network.