Skip to content
IP FediuninFull-Stack & AI Engineer
← All case studies

Case 01 · AI Engineering / Backend / Vector Search

Intelligent Knowledge Base

A RAG system for searching corporate knowledge

Overview

The system answers natural-language questions from an organisation's internal documents: it finds relevant fragments with vector search and hands them to a language model as context.

Project work

The charts are computed by this case's model on synthetic data — the same model that runs the interactive demo. They show what the approach changes, not figures from a client's system.

What I did

  1. Designed the RAG pipeline
  2. Built the API with FastAPI
  3. Vector search on pgvector and Milvus
  4. LLM integration with retrieved context
  5. Redis cache for repeated work

Before

  • Knowledge is scattered across policies and instructions
  • Keyword search misses questions phrased in other words
  • The file it finds still has to be read through

After

  • A question in plain language gets a direct answer
  • Every answer links to the documents behind it
  • New knowledge needs indexing, not retraining the model

Answer time

89×faster when a question is asked again

  • First time: search and the LLM1.1 s
  • Asked again: from the Redis cache12 ms
Average over the demo questions, with the latencies the demo reports.

Answers backed by sources

100%of answers cite the documents they came from

  • LLM alone, no retrieval0%
  • RAG: answer from the fragments found100%
The demo questions against the demo handbook; an answer counts when it cites a fragment it found.

How it works

A model of the system you can run: watch requests and data move through the components. It is a simplified reconstruction on synthetic data, not the production code.

What is happening

Ask a question: it goes through the cache, vector search and the LLM. You can start by indexing the documents.

The model runs in the browser on synthetic data. Times on the cards are illustrative, and motion is slowed down so the path can be followed.

Ask the knowledge base

Example questions

Demo base: a made-up company handbook — 6 documents, 8 fragments.

Fragments found

Fragments appear after a question.

Engineering detailsThe full write-up: problem, architecture, implementation decisions and outcome.

Problem

An organisation's knowledge is spread across documents: policies, instructions, process descriptions. Keyword search handles this poorly: the question and the document use different words, and the file it finds still has to be read through. The goal was a system that takes a question in natural language and answers it from the organisation's own sources rather than from the language model's general knowledge.

Solution

A retrieval-augmented generation (RAG) service. Documents are indexed as vector fragments that keep a reference to their source; a question is matched against them by meaning, and the language model answers only from the closest fragments, so every answer can be traced back to its documents.

My role

  • Designed the RAG pipeline: preparing and indexing documents, retrieving context and producing the answer.
  • Built the backend API with FastAPI for questions and for working with the index.
  • Implemented splitting documents into meaningful fragments and storing the fragments with their metadata.
  • Implemented search over vector representations of documents with pgvector and Milvus.
  • Integrated the language model and organised how the retrieved context is passed to it.
  • Added Redis for caching, so repeated operations are not performed again.

Technical architecture

  • Indexing. Documents are cleaned and split into fragments, embeddings are computed for the fragments, and the vectors are stored together with their metadata.
  • Answering a question. The API checks the cache, turns the question into a vector, finds the closest fragments and hands them to the language model as context.
  • Vector stores. pgvector keeps vectors next to relational data in PostgreSQL, so search combines with ordinary SQL filters. Milvus is a dedicated vector database with indexes for approximate search over large collections.

Implementation

  • Fragments. Fragment size is a trade-off: a large fragment blurs the meaning of its vector, a small one loses context. Overlapping neighbours keep a thought that a boundary would cut in two.
  • Relevance. Only the closest fragments go into the context. Low similarity is a sign that the answer is not in the base, and the model should not invent one.
  • Request context. The number of fragments is limited, so the answer stays precise and the request fits the model's context.
  • Metadata. Each fragment keeps a reference to its source document, so an answer can be checked against its source.

Functional outcome

The system lets people ask questions about internal documents in natural language and get answers grounded in the fragments found, with links to the sources. Adding new knowledge only takes indexing the documents; the model does not need to be retrained.

Need a system like this?

Tell me about the problem — I'll come back with questions and a proposed approach.

Discuss your projectHiring? See my engineering profile