AI chatbot, RAG, or simple prompt: what you actually need at your scale

A twelve-person food industry SME received a EUR 45,000 quote for a full RAG project: a dedicated vector database, an embedding pipeline, technical orchestration. Its entire documentation fit in twelve PDF files.

RAG (Retrieval Augmented Generation) has become the word you hear the moment generative AI on internal data comes up. It's a real architecture that solves a real problem. But it has a break-even point, and that point rarely sits where the quotes place it for an SME under 50 employees.

Three levels of complexity, not three competing products

A chatbot is an interface. A simple prompt and RAG are two different ways to feed it information. They aren't mutually exclusive choices, they're tiers, and the right instinct is to start with the cheapest one.

The simple prompt: load the documents directly

Paste the content of a handful of documents directly into a request, with no database or search infrastructure. A modern language model can read several dozen pages in a single pass.

Good enough for: drafting a standard quote, summarizing meeting notes, answering a recurring email, rephrasing an existing procedure, pulling a specific piece of information out of a document.

Lightweight RAG: a search step, without the full rig

Once there are more documents than can fit in a single request, a search step is needed before generation. That doesn't mean building the full architecture described in most technical guides. For a few hundred documents, something like pgvector (a PostgreSQL extension, not an extra dedicated service) is usually plenty.

Good enough for: a modestly sized internal knowledge base, an FAQ that has outgrown thirty questions, product documentation that updates regularly.

Full RAG: when volume and stakes justify it

Managed vector database, ingestion pipeline, reranking, dedicated orchestration: this version makes sense when the document volume is large, answer accuracy isn't negotiable, and several people depend on it daily. Think a law firm with hundreds of pages of regulation, or SaaS technical support whose documentation changes with every release.

📌 Rule of thumb: prompt first, lightweight RAG if that hits a wall, full architecture only once volume and stakes justify it. Not the other way round.

The test before signing a quote

Three questions, in order:

1. Does the model just need good instructions? If so, a well-written prompt is enough. Stop there.
2. Does it need to pull from up-to-date documents and cite its sources? That's where RAG becomes relevant, provided the volume exceeds what a prompt can hold.
3. Does it need to reproduce a very specific format or tone, thousands of times a day? Only then does fine-tuning come into the conversation, and even so, only after testing RAG first.

🔍 If you get stuck on the first question, that's good news for your budget.

What it actually costs, at each tier

⚠️ The most common trap: jumping straight to the full version because it looks more serious on paper. An SME that commissions an architecture sized for tens of thousands of documents, when it has a few hundred, pays for capacity it will never use.

Two honest caveats

Even lightweight RAG doesn't fully eliminate generation errors: a model can still extrapolate beyond what a given document actually says. And a simple prompt has a clear limit: past a certain volume of text, answer quality degrades, which is exactly the signal that it's time to move to the next tier.

💡 Pro tip: always start by measuring the actual volume of documents and requests before picking a tier. That figure, not a preference for one technology or another, is what should decide it.

The Nightborn method: scope before selling an architecture

A free 30-minute discovery call to honestly assess your real document and request volume before proposing anything. If a simple prompt is enough, that's what gets recommended. RAG, lightweight or full, only comes after that assessment, never by default.

What to remember

• Simple prompt, lightweight RAG, and full RAG are three tiers, not three competing products.
• A simple prompt covers most needs for an SME under 50 employees.
• Full RAG is only justified with a large document volume and a real accuracy stake.
• The right instinct: measure the volume before choosing the architecture, never the reverse.

A quote that jumps straight to a full architecture, without having measured your actual volume, deserves a second question before you sign.