Have you ever tried to build an AI feature, only to find your code is permanently tied to a single company’s service? I was there last month. I built a tool using OpenAI, only to watch my costs climb as usage spiked. I needed to switch providers, but the thought of rewriting dozens of API calls was overwhelming. That pain led me here, to building a system that treats AI models like pluggable components. If you’ve ever felt locked in or worried about costs, this is for you. Let’s build something better.
The core idea is simple. Instead of writing fetch calls directly to OpenAI, we use a framework that sits in the middle. This framework, LangChain.js, speaks a common language to our app. We tell it to run a conversation. It then decides whether to use OpenAI, Anthropic, or Google in the background. Our application code doesn’t need to care. This is the freedom we want.
To start, we create a new Next.js project. Make sure you use the App Router and TypeScript. The commands are straightforward. After that, we install our key tools. We need LangChain and its packages for each provider we plan to use. We also need the Vercel AI SDK. It’s not another AI model; think of it as a translator. It takes the streaming data from LangChain and formats it perfectly for a React component to display in real-time.
Now, let’s talk about those API keys. We store them securely in environment variables. I made a mistake early on by hardcoding them in a config file. Don’t do that. Use a validation library like Zod to ensure they are present when your app runs. This prevents confusing errors later. Your .env.local file should hold the keys for all the providers you intend to use.
The real magic begins with the provider factory. This is a fancy term for a function that gives us the right AI model. We define our primary provider in our environment settings. The factory function checks this setting and returns a configured ChatOpenAI, ChatAnthropic, or ChatGoogleGenerativeAI instance. All of these are LangChain objects that share a common interface. This means the rest of our code can interact with them in the same way.
But what happens when your primary provider fails? Maybe you hit a rate limit, or the service has an outage. A robust system needs a backup plan. We wrap our model calls in a retry logic. If a request to the primary provider fails, the code can automatically retry with the next provider on our list. This fallback strategy is what turns a demo into a reliable feature.
How do we keep the conversation flowing? We need memory. LangChain provides memory classes that help. A ConversationBufferWindowMemory will keep the last few messages of a chat in its context. This gives the AI model the immediate history it needs to make coherent replies. We attach this memory to our chain, and it manages the state for us, session by session.
Let’s write our core pipeline. We’ll build this in a Next.js API Route designed to run on the Edge runtime. This is crucial for speed. The Edge runtime lets us start responding to the user with the first token almost instantly. We create a PromptTemplate to shape our instructions. Then, we combine the prompt, the memory, and our chosen model into a ConversationChain.
The Vercel AI SDK enters the picture here. Its streamText function is the bridge. It takes our LangChain chain and connects it to a ReadableStream. This stream is what our frontend will consume. We pass the user’s message from the request into this chain, and it begins generating a response, token by token.
On the frontend, we use the useChat hook from the same SDK. This hook manages the state of our chat interface—the messages list, the input field, and the loading state. It connects to our API route. When we send a message, it handles the streaming response, appending each new piece of text to the latest message. The user sees the words appear as they are generated.
What about structured data? Sometimes, you don’t want a paragraph of text; you want the AI to fill out a form. For example, you might ask it to analyze a support ticket and output a severity level and category. We can use Zod, a validation library, to define the exact shape of the output we expect. LangChain can bind a model to this schema. The model will then output valid JSON that matches our structure. We can parse it with confidence, knowing the types are correct.
Managing cost is a real concern. Each model has a different price per token. We can add simple checks before processing a request. Calculate the rough token count of the conversation history. If it exceeds a budget you set, you can trim the memory or return a friendly error. This prevents a surprise bill at the end of the month.
Let me show you a piece of the provider factory. This is where the flexibility is born.
// src/lib/ai/provider-factory.ts
import { ChatOpenAI } from "@langchain/openai";
import { ChatAnthropic } from "@langchain/anthropic";
import { ChatGoogleGenerativeAI } from "@langchain/google-genai";
import { env } from "@/env";
export function getChatModel() {
const provider = env.PRIMARY_PROVIDER;
switch (provider) {
case "openai":
return new ChatOpenAI({
apiKey: env.OPENAI_API_KEY,
modelName: "gpt-4",
temperature: 0.7,
streaming: true,
});
case "anthropic":
return new ChatAnthropic({
apiKey: env.ANTHROPIC_API_KEY,
modelName: "claude-3-sonnet",
temperature: 0.7,
streaming: true,
});
case "google":
return new ChatGoogleGenerativeAI({
apiKey: env.GOOGLE_AI_API_KEY,
modelName: "gemini-pro",
temperature: 0.7,
streaming: true,
});
default:
throw new Error(`Unsupported provider: ${provider}`);
}
}See how the function returns a different object, but the rest of our application just calls getChatModel()? That’s the abstraction in action.
Testing is vital. You need to simulate failure. Turn off your Wi-Fi, or use an invalid API key for your primary provider. Does your system gracefully try the next one? Check the streaming response in your browser. Does the text appear smoothly, or does it chunk awkwardly? These are the details that separate a prototype from production-ready code.
I encourage you to start simple. Get a basic chain working with one provider and streaming. Then, add memory. After that, implement the provider fallback. Each step is a building block. This approach might seem like more work upfront, but it saves immense time and stress later. When a new, better model is released, you can plug it in with a few lines of code.
What challenges do you think you’ll face when moving from a single provider to this multi-provider setup? The initial configuration can be the biggest hurdle, but once it’s done, the path is clear.
In the end, you’ll have a system that gives you control. Control over cost, reliability, and performance. You are no longer at the mercy of a single API’s limitations. Your AI features become robust and adaptable. This is the modern way to build.
I hope this guide helps you break free from vendor lock-in. The code examples here are starting points. Adapt them, extend them, and make them your own. If you found this useful, please share it with a colleague who might be stuck in a single-provider trap. Have you built something similar? I’d love to hear about your experience in the comments. Let’s build more resilient applications together.
As a best-selling author, I invite you to explore my books on Amazon. Don’t forget to follow me on Medium and show your support. Thank you! Your support means the world!
101 Books
101 Books is an AI-driven publishing company co-founded by author Aarav Joshi. By leveraging advanced AI technology, we keep our publishing costs incredibly low—some books are priced as low as $4—making quality knowledge accessible to everyone.
Check out our book Golang Clean Code available on Amazon.
Stay tuned for updates and exciting news. When shopping for books, search for Aarav Joshi to find more of our titles. Use the provided link to enjoy special discounts!
📘 Checkout my latest ebook for free on my channel!
Be sure to like, share, comment, and subscribe to the channel!
Our Creations
Be sure to check out our creations:
Investor Central | Investor Central Spanish | Investor Central German | Smart Living | Epochs & Echoes | Puzzling Mysteries | Hindutva | Elite Dev | JS Schools
We are on Medium
Tech Koala Insights | Epochs & Echoes World | Investor Central Medium | Puzzling Mysteries Medium | Science & Epochs Medium | Modern Hindutva
