--- title: "RAG Development Services" section: "Services" canonical_url: "https://leverge.ai/services/rag-development" topic: "RAG development services" published: "2026-01-20" updated: "2026-07-28" publisher: "Ailoitte Technologies Private Limited" --- # RAG Development Services RAG development is the work of building a system that retrieves the right passages from your own documents and gives them to a language model, so answers are grounded in your data rather than in the model's training. Done properly it produces answers with citations a user can open and verify. Almost all RAG failures are retrieval failures, not generation failures — the model was simply handed the wrong context, so the fix is in chunking, hybrid search, reranking and evaluation rather than in prompting. ## Key takeaways - When a RAG system gives a wrong answer, the retrieval step is at fault roughly nine times out of ten — the model faithfully summarised the wrong passages. - Retrieval quality must be measured separately from answer quality, otherwise you cannot tell which half of the pipeline broke. - Pure vector search underperforms on exact identifiers, product codes and names. Hybrid keyword-plus-vector search with reranking is the reliable default. - Citations are not a nice-to-have. They are what makes a RAG answer auditable, and in regulated environments they are what gets the tool approved at all. - Document preparation and chunking strategy drive more of the final accuracy than the choice of model or vector database does. ## Almost every RAG failure is a retrieval failure When a RAG system answers badly, the instinct is to blame the model and start rewriting prompts. In our experience that is the wrong place to look roughly nine times out of ten. The model usually did its job faithfully — it summarised the passages it was given. The passages were simply the wrong ones. This matters because it changes where the engineering effort goes. Prompt iteration produces small, unstable gains. Fixing retrieval — better chunking, adding keyword search alongside vectors, reranking a wider candidate pool, filtering on metadata — produces large and durable ones. It also explains why so many teams plateau. They have built a single-stage vector search, they are measuring only end-to-end answer quality, and so they cannot see that their recall at k is 60%. No amount of prompt work recovers information that was never retrieved. ## Measure the two stages separately The most valuable instrumentation you can add is a retrieval metric that is independent of the answer. For a labelled set of questions, each with the passages that genuinely contain the answer, you can compute: - **Recall at k** — how often the correct passage appears in the top k results. If this is low, nothing downstream can save the answer. - **Precision at k** — how much of what you retrieved is actually relevant. Low precision means you are paying for tokens that dilute the context. - **Grounding rate** — what share of sentences in the final answer are supported by retrieved text. - **Refusal accuracy** — how often the system correctly declines when the corpus does not cover the question. With those four numbers, a quality change becomes diagnosable. Without them, every discussion about accuracy is someone's impression from six sample queries. ## Why hybrid search is the default, not an optimisation Dense vector search is good at meaning and bad at exact strings. That is not a tuning problem; it is what embeddings are. Ask for "contract AC-8871" and a vector index will happily return semantically similar contracts while missing the exact one, because the identifier carries almost no semantic signal. Real users do both kinds of query constantly — conceptual questions and precise lookups — often in the same session. So the reliable architecture is: 1. Retrieve a wide candidate set using **both** BM25 keyword search and vector search, fused with tuned weighting. 2. **Rerank** that candidate set with a cross-encoder, which reads query and passage together and orders them far more accurately than either retriever. 3. Pass only the **top few** passages to the expensive generation model. Step three is where the cost savings come from. Retrieving broadly and reranking cheaply, then generating narrowly, gives better accuracy than dumping twenty passages into a frontier model's context — and costs a fraction as much. ## Chunking decides more than the vector database does Teams spend weeks comparing vector databases and an afternoon on chunking. The weighting should be reversed. Any competent vector store will serve a mid-size corpus adequately; a bad chunking strategy caps your accuracy regardless of what is underneath. The failure mode is splitting on a fixed token count. It cuts tables in half, separates a clause from the heading that scopes it, and produces passages that are meaningless out of context. What works better: - Split on document structure — sections, headings, list boundaries, table rows. - Keep the parent context with the child chunk, so a retrieved paragraph arrives with the heading it sat under. - Extract tables separately and preserve their structure rather than flattening them into prose. - Attach metadata — source, date, version, owning team, jurisdiction — so retrieval can filter before it ranks. ## Refusal is a feature, and it has to be measured A system that always produces an answer is a system that invents one when the corpus is silent. For any use case where the answer has consequences, "this is not covered in the documents I have access to" is the correct output, and it needs to be treated as a scored behaviour in the evaluation set rather than as an embarrassing edge case. This is usually the point where a compliance or clinical reviewer decides whether to approve the tool. A system that cites its sources and admits its gaps is auditable. One that answers everything fluently is not, no matter how high its average accuracy looks. ## Frequently asked questions ### What is RAG in simple terms? Retrieval-augmented generation means looking things up before answering. Instead of relying on what a language model memorised during training, the system searches your own documents for the passages relevant to the question, hands those passages to the model, and asks it to answer using only that material. The practical benefits are that answers reflect your current data, that you can show the user exactly which source was used, and that adding new information means indexing a document rather than retraining anything. ### Is RAG better than fine-tuning a model? They solve different problems and are frequently confused. RAG supplies knowledge — facts, documents, policies, records that change over time. Fine-tuning shapes behaviour — tone, output format, following a domain-specific convention. If your problem is that the model does not know your data, fine-tuning is the wrong tool and will produce a model that confidently invents plausible details. If your problem is that the model knows the answer but formats it wrong, RAG will not help. Many production systems use both, for those two separate reasons. ### How accurate can a RAG system realistically be? On a well-prepared corpus with a properly built retrieval layer, 85-95% answer accuracy on questions the documents genuinely cover is a realistic target. The number is meaningless without two qualifiers: what counts as correct, and what the system does when the answer is not in the corpus. A system that answers 95% correctly but confidently guesses on the remaining 5% is often worse in practice than one that answers 88% correctly and says "not found" the rest of the time. ### How do you stop a RAG system from hallucinating? Four controls, in order of impact. First, retrieval quality — most hallucination is the model reasoning over irrelevant passages it was handed. Second, an explicit instruction and evaluation for refusal, so "the documents do not cover this" is treated as a correct answer rather than a failure. Third, citation enforcement, where a claim without a supporting retrieved passage is rejected. Fourth, grounding checks in the evaluation suite that score whether each sentence is supported by the retrieved context. ### What does a RAG system cost to run? Embedding and storage costs are usually minor. The recurring cost is inference on the answering step, and it is driven by how much context you retrieve per query. Retrieving twenty passages when four would do multiplies the bill by five for no accuracy gain, which is why we tune retrieval depth against the evaluation set rather than defaulting to "more context is safer". Reranking a wider candidate set with a small cheap model, then passing only the top few to the expensive one, is usually the best cost-accuracy trade. ### Can RAG work over data we cannot send to a third-party model? Yes. The retrieval layer runs entirely in your infrastructure, and the only question is where the generation step runs. Options in descending order of convenience: a commercial provider with zero-retention terms inside your cloud region, a managed model in your own cloud tenancy such as Bedrock or Vertex, or an open-weight model self-hosted on your hardware. We size the accuracy trade-off for each before you commit. --- Source: https://leverge.ai/services/rag-development — Leverge