Skip to content
Rabat, Morocco
Tutorials

Building Production-Ready RAG Applications with Next.js, Vector Databases, and OpenAI

November 17, 2025
Building Production-Ready RAG Applications
So you want to build a RAG application? That's awesome! RAG (Retrieval-Augmented Generation) is honestly changing how we build AI applications. Here's the cool part: instead of relying on what the AI learned during training, RAG systems can actually use your own documents, knowledge bases, and data sources to answer questions. It's like giving the AI access to your own library. In this tutorial, we're going to build a production-ready RAG application from scratch. We'll use:
  • Next.js 14 (App Router) for the frontend and API routes - it's perfect for this
  • OpenAI for embeddings and chat completion - their models are really good
  • Pinecone (or Chroma) for vector storage - you could use Chroma for local dev if you want
  • TypeScript for type safety - because catching bugs early is always good
By the end of this, you'll have a fully functional RAG system that can ingest documents, answer questions, and even show you where it got the information from. Pretty cool, right? Before we dive in, let's make sure you have everything:
  • Node.js 18+ installed (check with
    node --version
    if you're not sure)
  • An OpenAI API key (get one here - it's free to sign up)
  • A Pinecone account (sign up here) - they have a free tier that's perfect for getting started. Or you can use Chroma for local development if you prefer
  • Basic knowledge of React, TypeScript, and Next.js (don't worry if you're not an expert, we'll walk through everything)
Got everything? Great! Let's build something cool. Alright, let's get started! First, we need to create a new Next.js project. I'll walk you through it:
npx create-next-app@latest rag-app --typescript --tailwind --app
cd rag-app
Install the required dependencies:
npm install @pinecone-database/pinecone openai langchain @langchain/openai @langchain/pinecone
npm install -D @types/node
Create a
.env.local
file:
OPENAI_API_KEY=your-openai-api-key
PINECONE_API_KEY=your-pinecone-api-key
PINECONE_ENVIRONMENT=your-pinecone-environment
PINECONE_INDEX_NAME=rag-index
NEXT_PUBLIC_APP_URL=http://localhost:3000
Before we start coding, let me explain how this all fits together. Our RAG application has three main parts:
  1. Document Ingestion: This is where we take documents, break them into chunks, and store them in the vector database. Think of it as organizing a library.
  2. Query Interface: This is the frontend where users can actually ask questions. Simple but effective.
  3. RAG Pipeline: This is the magic part - it retrieves relevant chunks from the database and uses them to generate answers. It's like having a librarian who can instantly find the right books and summarize them for you.
Makes sense? Let's build it! Okay, first things first. We need to set up our vector database. This is where we'll store all the document embeddings. Let's create a utility file to handle Pinecone:
// lib/pinecone.ts
import { Pinecone } from "@pinecone-database/pinecone";

let pineconeClient: Pinecone | null = null;

export async function getPineconeClient() {
  if (pineconeClient) {
    return pineconeClient;
  }

  pineconeClient = new Pinecone({
    apiKey: process.env.PINECONE_API_KEY!,
  });

  return pineconeClient;
}

export async function getPineconeIndex() {
  const client = await getPineconeClient();
  const indexName = process.env.PINECONE_INDEX_NAME!;

  // Create index if it doesn't exist
  const indexes = await client.listIndexes();
  const indexExists = indexes.indexes?.some(
    (index) => index.name === indexName
  );

  if (!indexExists) {
    await client.createIndex({
      name: indexName,
      dimension: 1536, // OpenAI ada-002 embedding dimension
      metric: "cosine",
      spec: {
        serverless: {
          cloud: "aws",
          region: "us-east-1",
        },
      },
    });

    // Wait for index to be ready
    await new Promise((resolve) => setTimeout(resolve, 10000));
  }

  return client.index(indexName);
}
Now for the fun part - ingesting documents! This is where we take text, break it into chunks, create embeddings, and store them in Pinecone. Let's create an API route for this:
// app/api/ingest/route.ts
import { NextRequest, NextResponse } from "next/server";
import { getPineconeIndex } from "@/lib/pinecone";
import { OpenAIEmbeddings } from "@langchain/openai";
import { PineconeStore } from "@langchain/pinecone";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";

export async function POST(request: NextRequest) {
  try {
    const { text, metadata } = await request.json();

    if (!text) {
      return NextResponse.json({ error: "Text is required" }, { status: 400 });
    }

    // Initialize embeddings
    const embeddings = new OpenAIEmbeddings({
      openAIApiKey: process.env.OPENAI_API_KEY!,
      modelName: "text-embedding-ada-002",
    });

    // Split text into chunks
    const textSplitter = new RecursiveCharacterTextSplitter({
      chunkSize: 1000,
      chunkOverlap: 200,
    });

    const chunks = await textSplitter.createDocuments([text]);

    // Get Pinecone index
    const index = await getPineconeIndex();

    // Store in Pinecone
    await PineconeStore.fromDocuments(chunks, embeddings, {
      pineconeIndex: index,
      namespace: "default",
    });

    return NextResponse.json({
      success: true,
      chunks: chunks.length,
      message: "Document ingested successfully",
    });
  } catch (error) {
    console.error("Ingestion error:", error);
    return NextResponse.json(
      { error: "Failed to ingest document" },
      { status: 500 }
    );
  }
}
Alright, this is where the magic happens. We're going to create the endpoint that actually answers questions using RAG. Here's how it works: we take a question, find relevant chunks from our vector database, and use them to generate an answer:
// app/api/query/route.ts
import { NextRequest, NextResponse } from "next/server";
import { getPineconeIndex } from "@/lib/pinecone";
import { OpenAIEmbeddings } from "@langchain/openai";
import { PineconeStore } from "@langchain/pinecone";
import { ChatOpenAI } from "@langchain/openai";
import { RetrievalQAChain } from "langchain/chains";
import { PromptTemplate } from "@langchain/core/prompts";

export async function POST(request: NextRequest) {
  try {
    const { question } = await request.json();

    if (!question) {
      return NextResponse.json(
        { error: "Question is required" },
        { status: 400 }
      );
    }

    // Initialize embeddings
    const embeddings = new OpenAIEmbeddings({
      openAIApiKey: process.env.OPENAI_API_KEY!,
      modelName: "text-embedding-ada-002",
    });

    // Get Pinecone index and create vector store
    const index = await getPineconeIndex();
    const vectorStore = await PineconeStore.fromExistingIndex(embeddings, {
      pineconeIndex: index,
      namespace: "default",
    });

    // Initialize LLM
    const llm = new ChatOpenAI({
      openAIApiKey: process.env.OPENAI_API_KEY!,
      modelName: "gpt-4-turbo-preview",
      temperature: 0,
    });

    // Create custom prompt
    const prompt = PromptTemplate.fromTemplate(`
You are a helpful AI assistant. Use the following pieces of context to answer the question at the end.
If you don't know the answer, just say that you don't know, don't try to make up an answer.

Context:
{context}

Question: {question}

Answer:
`);

    // Create retrieval chain
    const chain = RetrievalQAChain.fromLLM(llm, vectorStore.asRetriever(), {
      prompt,
      returnSourceDocuments: true,
    });

    // Query the chain
    const response = await chain.invoke({
      query: question,
    });

    // Extract source documents
    const sources =
      response.sourceDocuments?.map((doc: any) => ({
        content: doc.pageContent,
        metadata: doc.metadata,
      })) || [];

    return NextResponse.json({
      answer: response.text,
      sources,
    });
  } catch (error) {
    console.error("Query error:", error);
    return NextResponse.json(
      { error: "Failed to process query" },
      { status: 500 }
    );
  }
}
Now let's build the frontend. I'm going to keep it simple but functional - you can make it prettier later. The important thing is that it works:
// app/page.tsx
'use client';

import { useState } from 'react';

interface Source {
  content: string;
  metadata: Record<string, any>;
}

interface QueryResponse {
  answer: string;
  sources: Source[];
}

export default function Home() {
  const [question, setQuestion] = useState('');
  const [answer, setAnswer] = useState('');
  const [sources, setSources] = useState<Source[]>([]);
  const [loading, setLoading] = useState(false);
  const [ingestText, setIngestText] = useState('');
  const [ingesting, setIngesting] = useState(false);

  const handleQuery = async (e: React.FormEvent) => {
    e.preventDefault();
    if (!question.trim()) return;

    setLoading(true);
    setAnswer('');
    setSources([]);

    try {
      const response = await fetch('/api/query', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ question }),
      });

      const data: QueryResponse = await response.json();
      setAnswer(data.answer);
      setSources(data.sources);
    } catch (error) {
      console.error('Query error:', error);
      setAnswer('Sorry, there was an error processing your question.');
    } finally {
      setLoading(false);
    }
  };

  const handleIngest = async (e: React.FormEvent) => {
    e.preventDefault();
    if (!ingestText.trim()) return;

    setIngesting(true);

    try {
      const response = await fetch('/api/ingest', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ text: ingestText }),
      });

      const data = await response.json();
      if (data.success) {
        alert(`Document ingested successfully! ${data.chunks} chunks created.`);
        setIngestText('');
      }
    } catch (error) {
      console.error('Ingestion error:', error);
      alert('Failed to ingest document.');
    } finally {
      setIngesting(false);
    }
  };

  return (
    <div className="min-h-screen bg-gray-50 py-12 px-4 sm:px-6 lg:px-8">
      <div className="max-w-4xl mx-auto">
        <h1 className="text-4xl font-bold text-gray-900 mb-8 text-center">
          RAG Application
        </h1>

        {/* Document Ingestion Section */}
        <div className="bg-white rounded-lg shadow-md p-6 mb-8">
          <h2 className="text-2xl font-semibold mb-4">Add Documents</h2>
          <form onSubmit={handleIngest} className="space-y-4">
            <textarea
              value={ingestText}
              onChange={(e) => setIngestText(e.target.value)}
              placeholder="Paste your document text here..."
              className="w-full px-4 py-2 border border-gray-300 rounded-md focus:ring-2 focus:ring-blue-500 focus:border-transparent"
              rows={6}
            />
            <button
              type="submit"
              disabled={ingesting}
              className="w-full bg-blue-600 text-white py-2 px-4 rounded-md hover:bg-blue-700 disabled:opacity-50 disabled:cursor-not-allowed"
            >
              {ingesting ? 'Ingesting...' : 'Ingest Document'}
            </button>
          </form>
        </div>

        {/* Query Section */}
        <div className="bg-white rounded-lg shadow-md p-6">
          <h2 className="text-2xl font-semibold mb-4">Ask a Question</h2>
          <form onSubmit={handleQuery} className="space-y-4">
            <input
              type="text"
              value={question}
              onChange={(e) => setQuestion(e.target.value)}
              placeholder="Enter your question..."
              className="w-full px-4 py-2 border border-gray-300 rounded-md focus:ring-2 focus:ring-blue-500 focus:border-transparent"
            />
            <button
              type="submit"
              disabled={loading}
              className="w-full bg-green-600 text-white py-2 px-4 rounded-md hover:bg-green-700 disabled:opacity-50 disabled:cursor-not-allowed"
            >
              {loading ? 'Processing...' : 'Ask Question'}
            </button>
          </form>

          {/* Answer Display */}
          {answer && (
            <div className="mt-6 p-4 bg-gray-50 rounded-md">
              <h3 className="font-semibold mb-2">Answer:</h3>
              <p className="text-gray-800 whitespace-pre-wrap">{answer}</p>
            </div>
          )}

          {/* Sources Display */}
          {sources.length > 0 && (
            <div className="mt-6">
              <h3 className="font-semibold mb-2">Sources:</h3>
              <div className="space-y-2">
                {sources.map((source, index) => (
                  <div
                    key={index}
                    className="p-3 bg-gray-50 rounded-md border border-gray-200"
                  >
                    <p className="text-sm text-gray-700 mb-1">
                      {source.content.substring(0, 200)}...
                    </p>
                    {source.metadata && (
                      <p className="text-xs text-gray-500">
                        Source: {JSON.stringify(source.metadata)}
                      </p>
                    )}
                  </div>
                ))}
              </div>
            </div>
          )}
        </div>
      </div>
    </div>
  );
}
Okay, we've got the basics working. Now let's add some cool features that will make this actually useful: Right now we're just pasting text, but wouldn't it be cool to upload PDFs? Let's add that:
// app/api/ingest/file/route.ts
import { NextRequest, NextResponse } from "next/server";
import { getPineconeIndex } from "@/lib/pinecone";
import { OpenAIEmbeddings } from "@langchain/openai";
import { PineconeStore } from "@langchain/pinecone";
import { PDFLoader } from "langchain/document_loaders/fs/pdf";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";

export async function POST(request: NextRequest) {
  try {
    const formData = await request.formData();
    const file = formData.get("file") as File;

    if (!file) {
      return NextResponse.json({ error: "File is required" }, { status: 400 });
    }

    // Convert file to buffer
    const bytes = await file.arrayBuffer();
    const buffer = Buffer.from(bytes);

    // Load PDF (you can add support for other formats)
    const loader = new PDFLoader(new Blob([buffer]));
    const docs = await loader.load();

    // Split documents
    const textSplitter = new RecursiveCharacterTextSplitter({
      chunkSize: 1000,
      chunkOverlap: 200,
    });

    const chunks = await textSplitter.splitDocuments(docs);

    // Initialize embeddings
    const embeddings = new OpenAIEmbeddings({
      openAIApiKey: process.env.OPENAI_API_KEY!,
      modelName: "text-embedding-ada-002",
    });

    // Store in Pinecone
    const index = await getPineconeIndex();
    await PineconeStore.fromDocuments(chunks, embeddings, {
      pineconeIndex: index,
      namespace: "default",
    });

    return NextResponse.json({
      success: true,
      chunks: chunks.length,
      message: "File ingested successfully",
    });
  } catch (error) {
    console.error("File ingestion error:", error);
    return NextResponse.json(
      { error: "Failed to ingest file" },
      { status: 500 }
    );
  }
}
Here's a pro tip: sometimes the initial retrieval isn't perfect. Adding a re-ranking step can really improve the quality of your results. Let me show you how:
// lib/rerank.ts
import { CohereClient } from "cohere-ai";

const cohere = new CohereClient({
  token: process.env.COHERE_API_KEY!,
});

export async function rerankDocuments(
  query: string,
  documents: string[],
  topK: number = 5
) {
  const response = await cohere.rerank({
    model: "rerank-english-v3.0",
    query,
    documents,
    topN: topK,
  });

  return response.results.map((result) => ({
    index: result.index,
    relevanceScore: result.relevanceScore,
  }));
}
Alright, let's talk about making this production-ready. There are a few things you'll want to add before you deploy: Things will go wrong. Let's handle errors gracefully:
// lib/errors.ts
export class RAGError extends Error {
  constructor(
    message: string,
    public code: string,
    public statusCode: number = 500
  ) {
    super(message);
    this.name = "RAGError";
  }
}

export function handleRAGError(error: unknown) {
  if (error instanceof RAGError) {
    return {
      error: error.message,
      code: error.code,
      statusCode: error.statusCode,
    };
  }

  console.error("Unexpected error:", error);
  return {
    error: "An unexpected error occurred",
    code: "UNKNOWN_ERROR",
    statusCode: 500,
  };
}
You don't want someone to accidentally (or intentionally) spam your API and rack up costs. Let's add rate limiting:
// lib/rateLimit.ts
import { LRUCache } from "lru-cache";

const rateLimit = new LRUCache<string, number>({
  max: 500,
  ttl: 60000, // 1 minute
});

export function checkRateLimit(identifier: string, limit: number = 10) {
  const count = rateLimit.get(identifier) || 0;

  if (count >= limit) {
    throw new Error("Rate limit exceeded");
  }

  rateLimit.set(identifier, count + 1);
  return true;
}
Here's a money-saving tip: cache frequently asked questions. If someone asks the same thing twice, why pay for it twice?
// lib/cache.ts
import { LRUCache } from "lru-cache";

const queryCache = new LRUCache<string, any>({
  max: 100,
  ttl: 3600000, // 1 hour
});

export function getCachedQuery(query: string) {
  return queryCache.get(query);
}

export function setCachedQuery(query: string, result: any) {
  queryCache.set(query, result);
}
Ready to deploy? Let's make sure everything is set up correctly: First, make sure all your environment variables are set in production. Don't forget this step - I've made that mistake before:
OPENAI_API_KEY=your-production-key
PINECONE_API_KEY=your-production-key
PINECONE_ENVIRONMENT=production
PINECONE_INDEX_NAME=rag-index-prod
NEXT_PUBLIC_APP_URL=https://your-domain.com
  1. Enable Edge Runtime for API routes when possible
  2. Implement proper logging with structured logs
  3. Add monitoring with tools like Sentry or DataDog
  4. Set up alerts for errors and performance issues
  5. Use CDN for static assets
Before you deploy, let's test everything. Here's how to test the ingestion and query endpoints:
// Test ingestion
const testDoc = `
Artificial intelligence (AI) is intelligence demonstrated by machines, 
in contrast to the natural intelligence displayed by humans and animals. 
Leading AI textbooks define the field as the study of "intelligent agents": 
any device that perceives its environment and takes actions that maximize 
its chance of successfully achieving its goals.
`;

const response = await fetch("/api/ingest", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ text: testDoc }),
});
// Test query
const response = await fetch("/api/query", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ question: "What is artificial intelligence?" }),
});

const data = await response.json();
console.log("Answer:", data.answer);
console.log("Sources:", data.sources);
I've learned a few things building RAG systems. Here's what works:
  1. Chunking Strategy: Don't just use default values. Experiment with different chunk sizes and overlap values - it makes a huge difference
  2. Embedding Models: Consider domain-specific embedding models. They're often better than general-purpose ones
  3. Hybrid Search: Combine semantic and keyword search. Sometimes old-school keyword matching works better
  4. Evaluation: Regularly check how your RAG system is performing. Are the answers actually good?
  5. Monitoring: Track everything - query patterns, latency, error rates. You'll thank yourself later
Congratulations! You've built a complete, production-ready RAG application. That's pretty impressive! Here's what we covered:
  • Setting up vector databases with Pinecone (the foundation)
  • Document ingestion and processing (getting your data in)
  • Building the RAG query pipeline (the magic part)
  • Creating a user-friendly frontend (making it usable)
  • Advanced features like file uploads and re-ranking (making it better)
  • Production considerations and best practices (making it reliable)
You've got everything you need to build something awesome. Now go make it your own!
  1. Enhance the UI: Add streaming responses, better source display, and conversation history
  2. Improve Retrieval: Implement hybrid search, query expansion, and better chunking
  3. Add Features: Support for multiple document types, user authentication, and analytics
  4. Optimize Performance: Implement caching, connection pooling, and async processing
  5. Scale: Consider using serverless functions, edge computing, and distributed systems
Happy building! 🚀

Enjoyed this tutorial?

Seriously, thank you for spending your time here. If you made it to the end, you’re the kind of person who cares about craft, not just quick fixes.If you’d like to support my work, pick one (no pressure). It genuinely helps me keep writing and improving these tutorials.
Subscribe to get a heads-up when I publish the next one. I keep it low-volume, and I only send what I’d personally want to read.

Subscribe to Josias's Newsletter

I occasionally write about software engineering, technology, and share thoughts on innovation and engineering.
One last thing: if you know someone who’d benefit from this, sharing it with them is one of the best compliments you can give. I appreciate you.