I’ve been thinking about microservices a lot lately. Not the theory, but the messy reality. You know that moment when you’re building something that works perfectly on your laptop, but the thought of putting it in front of real users makes you nervous? That’s where I was. I had services that talked to each other directly, databases that became bottlenecks, and no clear way to track what happened when a request failed somewhere in the chain. It felt fragile.
This led me down a path. I wanted to build something that could handle real traffic, survive failures, and let me sleep at night. The answer wasn’t a single tool, but a combination: NestJS for structure, MikroORM for clean data handling, and Kafka to make everything talk reliably. Let me show you what I built.
First, we need to set the stage. Think of a simple online store. A customer places an order. That single action needs to check stock, charge a card, and prepare for shipping. If the card fails, we need to put the stock back. This is a distributed transaction, and it’s the heart of the challenge.
How do you coordinate actions across separate, independent services without them becoming tightly bound together?
We use events. Instead of Service A calling Service B directly and waiting, Service A announces, “Hey, an order was created!” It then moves on. Any other service interested in that event can listen and act. This is where Apache Kafka shines. It’s a robust message broker that ensures events are delivered and stored.
Let’s look at the initial setup. We’ll use Docker to run our infrastructure so it’s consistent.
# docker-compose.yml
version: '3.8'
services:
kafka:
image: confluentinc/cp-kafka:latest
depends_on: [ zookeeper ]
ports: [ "9092:9092" ]
environment:
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
postgres:
image: postgres:15-alpine
environment:
POSTGRES_DB: order_db
ports: [ "5432:5432" ]Run docker-compose up -d. Now we have Kafka and PostgreSQL running. Next, we create our first service.
nest new order-service
cd order-service
npm install @mikro-orm/nestjs @mikro-orm/postgresql kafkajsWe start with the Order Service because it’s the orchestrator. Its job is to begin the process. We configure MikroORM to handle our database interactions. MikroORM is great because it manages entity state for us, which reduces bugs.
// src/mikro-orm.config.ts
import { Options } from '@mikro-orm/core';
import { PostgreSqlDriver } from '@mikro-orm/postgresql';
const config: Options = {
driver: PostgreSqlDriver,
host: 'localhost',
port: 5432,
dbName: 'order_db',
entities: ['dist/**/*.entity.js'],
entitiesTs: ['src/**/*.entity.ts'],
};
export default config;Now, let’s define what an Order looks like in our system.
// src/order/order.entity.ts
import { Entity, PrimaryKey, Property, Enum } from '@mikro-orm/core';
export enum OrderStatus {
PENDING = 'PENDING',
CONFIRMED = 'CONFIRMED',
CANCELLED = 'CANCELLED',
FAILED = 'FAILED',
}
@Entity()
export class Order {
@PrimaryKey({ type: 'uuid' })
id: string = v4();
@Property()
userId: string;
@Property({ type: 'jsonb' })
items: Array<{ productId: string; quantity: number }>;
@Enum(() => OrderStatus)
status: OrderStatus = OrderStatus.PENDING;
@Property()
createdAt: Date = new Date();
}The entity is simple. It has an ID, a user, some items, and a status. The status is crucial. It will change as our process moves forward. But creating the order in our database is just step one. We need to tell the world about it.
This is where Kafka comes in. We set up a producer to send messages.
// src/kafka/kafka.producer.service.ts
import { Injectable, OnModuleDestroy } from '@nestjs/common';
import { Kafka, Producer } from 'kafkajs';
@Injectable()
export class KafkaProducerService implements OnModuleDestroy {
private producer: Producer;
constructor() {
const kafka = new Kafka({ brokers: ['localhost:9092'] });
this.producer = kafka.producer();
}
async onModuleInit() {
await this.producer.connect();
}
async sendEvent(topic: string, event: object) {
await this.producer.send({
topic,
messages: [
{ value: JSON.stringify(event), key: event['orderId'] || 'default' },
],
});
}
async onModuleDestroy() {
await this.producer.disconnect();
}
}In our Order service, after we save the new order, we send an event.
// src/order/order.service.ts
async createOrder(dto: CreateOrderDto) {
const order = this.orderRepository.create(dto);
await this.orderRepository.persistAndFlush(order);
// Send the event
await this.kafkaProducer.sendEvent('order.created', {
eventType: 'ORDER_CREATED',
orderId: order.id,
userId: order.userId,
items: order.items,
timestamp: new Date().toISOString(),
});
return order;
}And just like that, the order is saved and an event is fired into the Kafka topic order.created. The Order Service’s job is done for now. It doesn’t know or care who listens. This is powerful.
But what good is an event if no one hears it? Let’s build the Inventory Service. Its job is to listen for order.created and reserve the stock.
We create a new NestJS app for it. The setup is similar, but this time we focus on consuming messages.
// src/kafka/kafka.consumer.service.ts (in Inventory Service)
import { Injectable, OnModuleDestroy } from '@nestjs/common';
import { Kafka, Consumer } from 'kafkajs';
@Injectable()
export class KafkaConsumerService implements OnModuleDestroy {
private consumer: Consumer;
constructor() {
const kafka = new Kafka({ brokers: ['localhost:9092'] });
this.consumer = kafka.consumer({ groupId: 'inventory-service-group' });
}
async onModuleInit() {
await this.consumer.connect();
await this.consumer.subscribe({ topic: 'order.created', fromBeginning: false });
await this.consumer.run({
eachMessage: async ({ message }) => {
const event = JSON.parse(message.value.toString());
await this.handleOrderCreated(event);
},
});
}
async handleOrderCreated(event: any) {
console.log('Reserving stock for order:', event.orderId);
// Logic to check and reserve inventory in DB
// If successful, send a new event: 'inventory.reserved'
// If failed, send 'inventory.failed'
}
async onModuleDestroy() {
await this.consumer.disconnect();
}
}See the pattern? The Inventory Service is completely independent. It runs its own database, its own logic. It hears about the order and tries to reserve stock. But here’s a critical question: what happens if the inventory check passes, but the payment fails later? The stock is stuck in a “reserved” state.
This is the classic problem of distributed transactions. We can’t use a simple database transaction across two separate services. The solution is a pattern called a Saga. A Saga is a sequence of local transactions. Each step publishes an event that triggers the next step. If a step fails, the Saga executes compensating actions to undo the previous steps.
Our order process is a Saga. ORDER_CREATED -> INVENTORY_RESERVED -> PAYMENT_PROCESSED. If payment fails, we need to publish an INVENTORY_RELEASED event.
We need a way to track the Saga’s progress. We can add a simple Saga log in our Order Service.
// src/saga/saga-log.entity.ts
@Entity()
export class SagaLog {
@PrimaryKey()
id: number;
@Property()
sagaId: string; // Usually the orderId
@Property()
step: string; // e.g., 'INVENTORY_RESERVED'
@Property()
status: 'PENDING' | 'COMPLETED' | 'COMPENSATING' | 'FAILED';
@Property({ type: 'jsonb', nullable: true })
payload: any;
}When the Order Service gets an inventory.reserved event, it updates the log and then tells the Payment Service to start. It’s orchestrating the flow.
But what about errors? What if the Inventory Service crashes while processing? Kafka consumers have a built-in mechanism for this. They track their “offset” (position) in the topic. If a consumer fails, it can restart from the last message it successfully processed. This gives us at-least-once delivery semantics. We must make our event handlers idempotent—able to be run multiple times without causing problems.
For example, our handleOrderCreated method should check: “Have I already reserved stock for this order ID?” If yes, it should do nothing.
async handleOrderCreated(event: any) {
const existing = await this.reservationRepo.findOne({ orderId: event.orderId });
if (existing) {
return; // Already processed, idempotency guard
}
// ... process reservation
}For truly poisonous messages that always fail, we use a Dead Letter Queue (DLQ). After several retries, we push the message to a special order.created.DLQ topic for manual inspection.
Performance is another consideration. Kafka topics are split into partitions. We can run multiple instances of our Inventory Service, each taking a partition, to process events in parallel. The key we set when producing the message (like the orderId) ensures all messages for the same order go to the same partition, keeping events for one order in sequence.
Finally, we need to see what’s happening. We can emit structured logs for each event and use a tracing ID that flows through all services. This lets us follow a single order’s journey across our entire system, which is invaluable for debugging.
Putting this all together feels solid. Each service is focused, scalable, and resilient. Failures in one part don’t cascade and bring down everything. Data is consistent in the end, even if the path to get there has detours.
Building software this way requires a shift in thinking. You move from “making function calls” to “managing state through events.” It’s more work upfront, but the payoff in robustness and scalability is immense. You stop worrying about servers talking to each other and start thinking about data flowing through the system.
What problems are you trying to solve with your architecture? Have you hit the limits of a simple monolithic approach? I’d love to hear about your experiences and challenges. If this guide helped you connect the dots, please share it with your team or anyone else wrestling with microservice communication. Let me know in the comments what your biggest hurdle has been—maybe we can tackle it next.
As a best-selling author, I invite you to explore my books on Amazon. Don’t forget to follow me on Medium and show your support. Thank you! Your support means the world!
101 Books
101 Books is an AI-driven publishing company co-founded by author Aarav Joshi. By leveraging advanced AI technology, we keep our publishing costs incredibly low—some books are priced as low as $4—making quality knowledge accessible to everyone.
Check out our book Golang Clean Code available on Amazon.
Stay tuned for updates and exciting news. When shopping for books, search for Aarav Joshi to find more of our titles. Use the provided link to enjoy special discounts!
📘 Checkout my latest ebook for free on my channel!
Be sure to like, share, comment, and subscribe to the channel!
Our Creations
Be sure to check out our creations:
Investor Central | Investor Central Spanish | Investor Central German | Smart Living | Epochs & Echoes | Puzzling Mysteries | Hindutva | Elite Dev | JS Schools
We are on Medium
Tech Koala Insights | Epochs & Echoes World | Investor Central Medium | Puzzling Mysteries Medium | Science & Epochs Medium | Modern Hindutva
