RAG Over Your Company Data: What It Costs and How I Build It With pgvector

A RAG chatbot over company data costs **$2,500-$15,000** to build with a senior engineer in 2026, and typically **$20-$500 per month** to run for a team of up to a few hundred users, with embeddings costing almost nothing and the LLM answer step making up nearly all of the bill. The architecture that keeps it accurate is not exotic: PostgreSQL with pgvector, hybrid search, permission-aware retrieval, citations, and a small eval set you run every time you change anything. I'm Md. Emamul Mursalin, a freelance AI product developer in Rajshahi, Bangladesh. I have built retrieval systems over support tickets, clinical notes, contracts, and product documentation for clients in the US, UK, and EU. This is the build I use, the reasons behind each choice, and the numbers. ## What a RAG chatbot over company data actually does Retrieval-augmented generation is a two-step trick. When a user asks a question, the system first retrieves the handful of passages from your documents most likely to contain the answer, then hands those passages to a large language model with the instruction to answer only from them and cite where the answer came from. The model never memorizes your data. That is the point. It means: - Updates are instant: re-index a document and the next answer reflects it. - Every answer can carry citations back to the source. - Permissions can be enforced at retrieval time, so a user only gets answers from documents they are allowed to read. - You never send your whole knowledge base to a provider, only the few passages relevant to each question. Fine-tuning does none of those things. It shapes tone and format; it is a poor way to teach a model facts that change. For nearly every "chatbot trained on our documents" request I receive, RAG is the right tool. ## The architecture, end to end ``` [Sources: Notion, Google Drive, Confluence, tickets, PDFs, database rows] | (connectors + change detection) v [Ingestion worker (NestJS + BullMQ)] -> chunk -> embed -> upsert | v [PostgreSQL + pgvector: chunks(tenant_id, doc_id, acl, content, embedding, tsv)] ^ | hybrid query: vector similarity + full-text, filtered by tenant + ACL [Query service] -> rerank -> build prompt -> LLM (streaming) -> answer + citations | v [Eval runner: golden questions, expected sources, expected answers] ``` ### Ingestion Documents arrive through connectors (Google Drive, Notion, a database table, or plain uploads). A background worker splits each document into chunks of roughly 300-800 tokens with overlap, preserving headings as metadata, embeds each chunk, and upserts it with the document's tenant and access-control list. Chunking is where most RAG systems quietly fail. Chunks cut mid-table or mid-clause produce confident wrong answers. I chunk by structure first (headings, list items, table rows) and by token count second. ### Storage: why pgvector and not a dedicated vector database For almost every company-data use case, pgvector inside the PostgreSQL you already run is the right answer. As of 2026, pgvector 0.8.x ships HNSW and IVFFlat indexes, half-precision vectors that halve storage with minimal recall loss, and iterative index scans that fix the old problem of filtered queries returning too few results. What that means in practice: - One database, one backup, one set of credentials, one transaction boundary. Chunks and their permissions live in the same rows. - Filtering by `tenant_id` and ACL happens in SQL, with the same row-level security you use everywhere else. - Hybrid search is a single query: cosine similarity on the embedding plus a `tsvector` full-text match, combined with reciprocal rank fusion. - Comfortable to several million chunks on a mid-size managed Postgres instance. Pinecone, Weaviate, or Qdrant earn their place when you are past tens of millions of vectors, need multi-region replication of the index itself, or run on infrastructure with no Postgres. For a company knowledge base, that is rare. ### The query, in SQL This is the core hybrid retrieval query, simplified from a production build. It runs in a NestJS service with Prisma's raw query support: ```sql WITH vec AS ( SELECT id, 1 - (embedding <=> $1::vector) AS score, ROW_NUMBER() OVER (ORDER BY embedding <=> $1::vector) AS rnk FROM chunks WHERE tenant_id = $2 AND acl && $3::text[] ORDER BY embedding <=> $1::vector LIMIT 40 ), fts AS ( SELECT id, ts_rank_cd(tsv, plainto_tsquery('english', $4)) AS score, ROW_NUMBER() OVER (ORDER BY ts_rank_cd(tsv, plainto_tsquery('english', $4)) DESC) AS rnk FROM chunks WHERE tenant_id = $2 AND acl && $3::text[] AND tsv @@ plainto_tsquery('english', $4) LIMIT 40 ) SELECT c.id, c.doc_id, c.content, c.heading, COALESCE(1.0 / (60 + vec.rnk), 0) + COALESCE(1.0 / (60 + fts.rnk), 0) AS fused FROM chunks c LEFT JOIN vec ON vec.id = c.id LEFT JOIN fts ON fts.id = c.id WHERE vec.id IS NOT NULL OR fts.id IS NOT NULL ORDER BY fused DESC LIMIT 8; ``` The `acl && $3::text[]` clause is the permission check: the user's group memberships are passed as an array and a chunk matches only if it shares at least one. An HNSW index on `embedding` and a GIN index on `tsv` keep this under 50 ms for a few million rows. ### Generation and citations The top chunks go into a prompt that says, in effect: answer from these passages only; if the answer is not in them, say so; cite the passage numbers you used. The response streams to the UI, and the citation numbers are mapped back to document titles and links. ### Evals Before launch I build a set of 30-100 golden questions with the expected source document and a reference answer. Every change to chunking, the embedding model, the retrieval query, or the prompt runs against that set and reports retrieval recall and answer accuracy. This is the difference between a system that improves and one that regresses silently. ## What it costs to build Priced for the architecture above, built by me directly with your team. US agencies quote $15,000-$50,000 for comparable RAG projects. | Scope | My fixed price | Timeline | What is included | |---|---|---|---| | Knowledge-base chatbot over one source, single tenant, citations, basic evals | $2,499 | 2-4 weeks | Ingestion, pgvector, hybrid search, streaming chat UI, 30 golden questions | | Multi-source, permission-aware, multi-tenant, admin dashboard, full evals | $6,999 | 4-8 weeks | Everything above plus connectors, ACL retrieval, reranking, usage analytics, observability | | RAG as a product feature inside your SaaS with billing and agents | $12,999+ | 8-16 weeks | Everything above plus tool-calling agent, Stripe usage billing, SLAs, 30 days of support | The first two tiers correspond to my [AI Integration Professional](/services/ai-integration-professional) and [Enterprise](/services/ai-integration-enterprise) offerings; the third is the [AI Agent SaaS Enterprise](/services/ai-agent-saas-enterprise) tier. All scope details are on the [pricing page](/pricing#ai-integration). If you are integrating into an existing product rather than building new, read [how to add AI to an existing SaaS product without rewriting it](/blog/add-ai-to-existing-saas-product) first; the sidecar pattern there is how this RAG service plugs in. ## What it costs to run The running cost surprises people in a good way. As of September 2026: - **Embedding** with OpenAI's text-embedding-3-small costs **$0.02 per million tokens**. Embedding a 10,000-page knowledge base (roughly 5 million tokens) costs about **$0.10**. Re-embedding changed documents each month is pennies. - **Storage** for one million chunks with 1,536-dimension vectors is a few gigabytes, or half that with half-precision vectors. On a managed Postgres plan this is $25-$100 per month including the rest of your database. - **Answering** is the real cost. A typical question sends 5,000-8,000 tokens of retrieved context plus the question and receives a 300-token answer. On Claude Haiku 4.5 ($1 / $5 per million tokens) that is about **$0.008 per question**; on Claude Sonnet 4.6 ($3 / $15) about $0.025; on OpenAI's budget tier roughly $0.002. At 5,000 questions a month, that is **$10-$125 per month** in LLM fees depending on the model. Route easy questions to the cheap model and escalate only when retrieval confidence is low, and you stay at the bottom of that range. Prompt caching on the system prompt and the static parts of the context shaves another 30-50% off input costs. Agency estimates for SaaS AI API fees of $50-$3,000 per month are consistent with this once you scale to thousands of users. ## Where RAG chatbots go wrong Every failed RAG project I have been asked to rescue had one of these: 1. **No permission filtering.** Retrieval across all documents, then hoping the model does not reveal something the user should not see. The model will. 2. **Naive chunking.** Fixed 500-character windows that split sentences and tables. 3. **Vector-only search.** Semantic similarity misses exact identifiers: part numbers, invoice IDs, error codes. Hybrid search fixes this. 4. **No "I don't know" path.** A model forced to answer will invent one. The prompt and the UI must allow abstention. 5. **No evals.** Prompt tweaks made by feel, regressions discovered by customers. 6. **Stale index.** No change detection, so the bot confidently quotes last quarter's policy. All of these are cheap to fix at design time and expensive after launch. ## Checklist before you commission a RAG build - Where the documents live, and whether there is an API or export for each source - Approximate volume: number of documents and total pages - Who is allowed to see what, and where those permissions are defined today - Twenty real questions your team asks, with the document that holds the answer - Whether the data includes PII, health, or financial information - Which LLM providers are acceptable, and any data-residency constraints - Whether the chatbot is internal, customer-facing, or sold as a product feature - How fresh answers need to be: hourly, daily, or on-demand re-indexing If you can answer these in an afternoon, a two-to-four-week build is realistic. ## Working with me on this I build these systems from Rajshahi, Bangladesh (UTC+6) for teams in the US, UK, and EU, with a full afternoon of overlap for the UK and EU and three to four hours across the US morning and evening. Contracts include IP assignment, payment runs by Payoneer or bank transfer in milestones, and the code lives in your GitHub organization from the first commit. [MedScribe](/projects/medscribe) is an example of retrieval over sensitive clinical documentation with strict access boundaries, and [Contently AI](/projects/contently-ai) uses retrieval over a client's own content library as part of an agentic workflow. ## FAQ ### Can I build an AI chatbot trained on my company documents? Yes, and the right way is retrieval-augmented generation, not training. Your documents are chunked, embedded, and stored in pgvector; each question retrieves the relevant passages and the model answers from them with citations. The model never memorizes your data, updates are instant, and permissions are enforced at retrieval time. ### Which vector database should I use, pgvector or Pinecone? Use pgvector if you already run PostgreSQL and have fewer than roughly ten million vectors, which covers nearly every company knowledge base. You get one database, SQL filtering by tenant and permission, and hybrid search in a single query. Choose Pinecone or a similar dedicated store only at very large scale or when you have no Postgres. ### How do I stop an AI chatbot from hallucinating? Retrieve well and constrain the model. Use hybrid search so exact terms are found, filter by permissions, pass only the top passages, instruct the model to answer only from them and to say when the answer is absent, show citations, and run a golden-question eval set on every change. Hallucination is mostly a retrieval problem, not a model problem. ### How much does it cost to build a RAG chatbot over company data? In 2026, $2,499 for a single-source knowledge-base chatbot with citations and evals, $6,999 for a multi-source, permission-aware, multi-tenant build, and $12,999 and up when it becomes a product feature with agents and billing, from a senior offshore engineer. US agencies quote $15,000-$50,000 for comparable scope. Monthly running cost is typically $20-$500. ### How much does RAG cost per month in API fees? Embedding is negligible: about ten cents to index 10,000 pages. Answering costs $0.002-$0.025 per question depending on the model, so 5,000 questions a month is $10-$125. Managed Postgres with pgvector adds $25-$100. With model routing and prompt caching, most teams under a few hundred users spend under $200 a month. ## Work with me If you want a RAG chatbot over company data that your team will actually trust, send me a description of your sources and twenty real questions, and I will reply within two business days with an architecture sketch and a fixed price. [Book a free discovery call](/appointments), see the [AI agent and RAG developer from Bangladesh](/hire/ai-agent-developer-bangladesh) page, or [start the Professional AI integration brief](/contact?service=ai-integration&tier=professional).