Building Production-Ready RAG Applications with Next.js, Vector Databases, and OpenAI
November 17, 2025
Introduction
So you want to build a RAG application? That's awesome! RAG (Retrieval-Augmented Generation) is honestly changing how we build AI applications. Here's the cool part: instead of relying on what the AI learned during training, RAG systems can actually use your own documents, knowledge bases, and data sources to answer questions. It's like giving the AI access to your own library.In this tutorial, we're going to build a production-ready RAG application from scratch. We'll use:
Next.js 14 (App Router) for the frontend and API routes - it's perfect for this
OpenAI for embeddings and chat completion - their models are really good
Pinecone (or Chroma) for vector storage - you could use Chroma for local dev if you want
TypeScript for type safety - because catching bugs early is always good
By the end of this, you'll have a fully functional RAG system that can ingest documents, answer questions, and even show you where it got the information from. Pretty cool, right?
Prerequisites
Before we dive in, let's make sure you have everything:
Node.js 18+ installed (check with
node --version
if you're not sure)
An OpenAI API key (get one here - it's free to sign up)
A Pinecone account (sign up here) - they have a free tier that's perfect for getting started. Or you can use Chroma for local development if you prefer
Basic knowledge of React, TypeScript, and Next.js (don't worry if you're not an expert, we'll walk through everything)
Before we start coding, let me explain how this all fits together. Our RAG application has three main parts:
Document Ingestion: This is where we take documents, break them into chunks, and store them in the vector database. Think of it as organizing a library.
Query Interface: This is the frontend where users can actually ask questions. Simple but effective.
RAG Pipeline: This is the magic part - it retrieves relevant chunks from the database and uses them to generate answers. It's like having a librarian who can instantly find the right books and summarize them for you.
Makes sense? Let's build it!
Step 1: Setting Up the Vector Database
Okay, first things first. We need to set up our vector database. This is where we'll store all the document embeddings. Let's create a utility file to handle Pinecone:
// lib/pinecone.ts
import { Pinecone } from "@pinecone-database/pinecone";
let pineconeClient: Pinecone | null = null;
export async function getPineconeClient() {
if (pineconeClient) {
return pineconeClient;
}
pineconeClient = new Pinecone({
apiKey: process.env.PINECONE_API_KEY!,
});
return pineconeClient;
}
export async function getPineconeIndex() {
const client = await getPineconeClient();
const indexName = process.env.PINECONE_INDEX_NAME!;
// Create index if it doesn't exist
const indexes = await client.listIndexes();
const indexExists = indexes.indexes?.some(
(index) => index.name === indexName
);
if (!indexExists) {
await client.createIndex({
name: indexName,
dimension: 1536, // OpenAI ada-002 embedding dimension
metric: "cosine",
spec: {
serverless: {
cloud: "aws",
region: "us-east-1",
},
},
});
// Wait for index to be ready
await new Promise((resolve) => setTimeout(resolve, 10000));
}
return client.index(indexName);
}
Step 2: Document Ingestion API
Now for the fun part - ingesting documents! This is where we take text, break it into chunks, create embeddings, and store them in Pinecone. Let's create an API route for this:
// app/api/ingest/route.ts
import { NextRequest, NextResponse } from "next/server";
import { getPineconeIndex } from "@/lib/pinecone";
import { OpenAIEmbeddings } from "@langchain/openai";
import { PineconeStore } from "@langchain/pinecone";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
export async function POST(request: NextRequest) {
try {
const { text, metadata } = await request.json();
if (!text) {
return NextResponse.json({ error: "Text is required" }, { status: 400 });
}
// Initialize embeddings
const embeddings = new OpenAIEmbeddings({
openAIApiKey: process.env.OPENAI_API_KEY!,
modelName: "text-embedding-ada-002",
});
// Split text into chunks
const textSplitter = new RecursiveCharacterTextSplitter({
chunkSize: 1000,
chunkOverlap: 200,
});
const chunks = await textSplitter.createDocuments([text]);
// Get Pinecone index
const index = await getPineconeIndex();
// Store in Pinecone
await PineconeStore.fromDocuments(chunks, embeddings, {
pineconeIndex: index,
namespace: "default",
});
return NextResponse.json({
success: true,
chunks: chunks.length,
message: "Document ingested successfully",
});
} catch (error) {
console.error("Ingestion error:", error);
return NextResponse.json(
{ error: "Failed to ingest document" },
{ status: 500 }
);
}
}
Step 3: RAG Query API
Alright, this is where the magic happens. We're going to create the endpoint that actually answers questions using RAG. Here's how it works: we take a question, find relevant chunks from our vector database, and use them to generate an answer:
// app/api/query/route.ts
import { NextRequest, NextResponse } from "next/server";
import { getPineconeIndex } from "@/lib/pinecone";
import { OpenAIEmbeddings } from "@langchain/openai";
import { PineconeStore } from "@langchain/pinecone";
import { ChatOpenAI } from "@langchain/openai";
import { RetrievalQAChain } from "langchain/chains";
import { PromptTemplate } from "@langchain/core/prompts";
export async function POST(request: NextRequest) {
try {
const { question } = await request.json();
if (!question) {
return NextResponse.json(
{ error: "Question is required" },
{ status: 400 }
);
}
// Initialize embeddings
const embeddings = new OpenAIEmbeddings({
openAIApiKey: process.env.OPENAI_API_KEY!,
modelName: "text-embedding-ada-002",
});
// Get Pinecone index and create vector store
const index = await getPineconeIndex();
const vectorStore = await PineconeStore.fromExistingIndex(embeddings, {
pineconeIndex: index,
namespace: "default",
});
// Initialize LLM
const llm = new ChatOpenAI({
openAIApiKey: process.env.OPENAI_API_KEY!,
modelName: "gpt-4-turbo-preview",
temperature: 0,
});
// Create custom prompt
const prompt = PromptTemplate.fromTemplate(`
You are a helpful AI assistant. Use the following pieces of context to answer the question at the end.
If you don't know the answer, just say that you don't know, don't try to make up an answer.
Context:
{context}
Question: {question}
Answer:
`);
// Create retrieval chain
const chain = RetrievalQAChain.fromLLM(llm, vectorStore.asRetriever(), {
prompt,
returnSourceDocuments: true,
});
// Query the chain
const response = await chain.invoke({
query: question,
});
// Extract source documents
const sources =
response.sourceDocuments?.map((doc: any) => ({
content: doc.pageContent,
metadata: doc.metadata,
})) || [];
return NextResponse.json({
answer: response.text,
sources,
});
} catch (error) {
console.error("Query error:", error);
return NextResponse.json(
{ error: "Failed to process query" },
{ status: 500 }
);
}
}
Step 4: Building the Frontend
Now let's build the frontend. I'm going to keep it simple but functional - you can make it prettier later. The important thing is that it works:
Okay, we've got the basics working. Now let's add some cool features that will make this actually useful:
Adding File Upload Support
Right now we're just pasting text, but wouldn't it be cool to upload PDFs? Let's add that:
// app/api/ingest/file/route.ts
import { NextRequest, NextResponse } from "next/server";
import { getPineconeIndex } from "@/lib/pinecone";
import { OpenAIEmbeddings } from "@langchain/openai";
import { PineconeStore } from "@langchain/pinecone";
import { PDFLoader } from "langchain/document_loaders/fs/pdf";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
export async function POST(request: NextRequest) {
try {
const formData = await request.formData();
const file = formData.get("file") as File;
if (!file) {
return NextResponse.json({ error: "File is required" }, { status: 400 });
}
// Convert file to buffer
const bytes = await file.arrayBuffer();
const buffer = Buffer.from(bytes);
// Load PDF (you can add support for other formats)
const loader = new PDFLoader(new Blob([buffer]));
const docs = await loader.load();
// Split documents
const textSplitter = new RecursiveCharacterTextSplitter({
chunkSize: 1000,
chunkOverlap: 200,
});
const chunks = await textSplitter.splitDocuments(docs);
// Initialize embeddings
const embeddings = new OpenAIEmbeddings({
openAIApiKey: process.env.OPENAI_API_KEY!,
modelName: "text-embedding-ada-002",
});
// Store in Pinecone
const index = await getPineconeIndex();
await PineconeStore.fromDocuments(chunks, embeddings, {
pineconeIndex: index,
namespace: "default",
});
return NextResponse.json({
success: true,
chunks: chunks.length,
message: "File ingested successfully",
});
} catch (error) {
console.error("File ingestion error:", error);
return NextResponse.json(
{ error: "Failed to ingest file" },
{ status: 500 }
);
}
}
Improving Retrieval with Re-ranking
Here's a pro tip: sometimes the initial retrieval isn't perfect. Adding a re-ranking step can really improve the quality of your results. Let me show you how:
Before you deploy, let's test everything. Here's how to test the ingestion and query endpoints:
// Test ingestion
const testDoc = `
Artificial intelligence (AI) is intelligence demonstrated by machines,
in contrast to the natural intelligence displayed by humans and animals.
Leading AI textbooks define the field as the study of "intelligent agents":
any device that perceives its environment and takes actions that maximize
its chance of successfully achieving its goals.
`;
const response = await fetch("/api/ingest", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ text: testDoc }),
});
Seriously, thank you for spending your time here. If you made it to the end, you’re the kind of person who cares about craft, not just quick fixes.If you’d like to support my work, pick one (no pressure). It genuinely helps me keep writing and improving these tutorials.