Pub/Sub Architecture: A Complete Guide to Publish-Subscribe Pattern

Pub/Sub (publish-subscribe) is one of the most powerful messaging patterns in modern software architecture. This guide explains what it is, how it works, when to use it, and how to implement it in your projects.

What is Pub/Sub Architecture?

Pub/Sub is an asynchronous messaging pattern where:

Publishers send messages without knowing who receives them
Subscribers listen for messages without knowing where they come from
A message broker sits in the middle, decoupling sender from receiver

This is fundamentally different from direct communication (HTTP requests, RPC calls) because publishers and subscribers never talk directly.

Real-World Analogy

Think of a newspaper:

Publisher: The newspaper company
Message: The daily newspaper
Broker: The distribution center
Subscribers: People who signed up for the newspaper

The newspaper company doesn't care who subscribes. Subscribers don't care who publishes. They're completely decoupled.

Key Concepts

1. Topics (Channels)

Messages are organized by topics. Subscribers choose which topics to listen to.

Topic: "user.registered"
  └─ Subscriber A (send welcome email)
  └─ Subscriber B (add to analytics)
  └─ Subscriber C (create user profile)

2. Publishers

Emit messages to a topic without waiting for responses.

// Pseudocode: Publisher doesn't care who receives this
messageBroker.publish('user.registered', {
  userId: '123',
  email: 'user@example.com'
});

3. Subscribers

Listen on a topic and react when messages arrive.

// Pseudocode: Subscriber only cares about 'user.registered' messages
messageBroker.subscribe('user.registered', (message) => {
  sendWelcomeEmail(message.email);
});

4. Message Broker

The intermediary that receives, stores, and delivers messages.

Examples: RabbitMQ, Apache Kafka, Google Cloud Pub/Sub, AWS SNS/SQS, Redis Pub/Sub

Pub/Sub vs Similar Patterns

Pub/Sub vs Observer Pattern

Aspect	Pub/Sub	Observer
Coupling	Decoupled (broker between them)	Tightly coupled (direct reference)
Scale	Works at system/network scale	Works within a single process
Message persistence	Often persisted by broker	No persistence
Use case	Distributed systems	Local event handling

When to use Observer: Single process, local event propagation (like DOM events, Redux actions)

When to use Pub/Sub: Distributed systems, cross-service communication, asynchronous workflows

Pub/Sub vs Event Driven Architecture

Pub/Sub is a type of event-driven architecture. Event-driven is the broader umbrella term for any system where components communicate via events.

Event-driven: Umbrella term (includes Pub/Sub, Event Sourcing, CQRS, etc.)
Pub/Sub: Specific pattern within event-driven architecture

How Pub/Sub Works: Step by Step

Here's what happens when a message is published:

1. Publisher sends message to broker:
   messageBroker.publish('order.created', orderData)

2. Broker receives and routes message:
   [Message Broker checks subscriptions for 'order.created']

3. Broker delivers to all subscribers:
   └─ Email Service receives message → sends confirmation email
   └─ Inventory Service receives message → updates stock
   └─ Analytics Service receives message → logs event
   └─ Notification Service receives message → sends push notification

4. Each subscriber processes independently and asynchronously

Key point: Publisher doesn't wait for subscribers to finish. It's completely asynchronous.

Types of Pub/Sub Implementations

1. Fire-and-Forget (At-Most-Once)

Messages may be lost if subscriber isn't listening.

// Redis Pub/Sub - no persistence
redis.publish('channel', message);

Use when: Notifications, real-time updates where loss is acceptable

Risk: Messages can disappear if no one's listening

2. Durable Queues (At-Least-Once)

Messages are stored until delivered. Guarantees delivery.

// RabbitMQ with durable queues
channel.assertQueue('queue-name', { durable: true });

Use when: Critical business logic (order processing, payments)

Trade-off: Slower, requires storage

3. Message Streaming (Kafka)

Messages stored in log with replay capability. Subscribers can catch up.

// Kafka - messages persisted in order
producer.send({
  topic: 'order.created',
  messages: [{ value: JSON.stringify(order) }]
});

Use when: High-volume data pipelines, event sourcing, real-time analytics

Benefit: Can replay entire event history

When Should You Use Pub/Sub?

Use Pub/Sub when:

✅ Multiple services need to react to the same event
✅ You want loose coupling between services
✅ Processing can be asynchronous
✅ You have unpredictable subscriber count (may add more later)
✅ Messages need to be durable/persisted

Don't use Pub/Sub when:

❌ You need synchronous response (use request-reply instead)
❌ Single process, single subscriber (use Observer pattern)
❌ Real-time, critical path operations that can't wait (use direct calls)
❌ Simple one-off tasks (use message queues instead)

Real-World Examples

Example 1: E-commerce Order Processing

Customer places order
     ↓
[Pub/Sub Broker: "order.created"]
     ↓
   ┌─────────────────────────────┬──────────────────┬────────────────┐
   ↓                             ↓                  ↓                ↓
Payment Service         Inventory Service    Notification Service  Analytics
Charges card           Updates stock        Sends confirmation     Logs event
Publishes:            Publishes:           Publishes:            Publishes:
"payment.processed"   "stock.updated"      "email.sent"          "order.logged"

Each service is independent and can fail without breaking the entire flow.

Example 2: Real-Time Chat Application

User A sends message
     ↓
[Pub/Sub Broker: "chat.message"]
     ↓
   ┌──────────────┬──────────────┬──────────────┐
   ↓              ↓              ↓              ↓
User B's client  User C's client  Archive DB  Moderation Queue
Receives instantly (near real-time)

Using Pub/Sub instead of WebSocket directly allows:

Horizontal scaling (message broker handles distribution)
Message persistence (can replay if client goes offline)
Multiple services consuming same event

Popular Pub/Sub Tools and Brokers

1. Redis Pub/Sub (Simplest)

✅ Extremely simple to use
❌ No message persistence
Use for: Real-time updates, notifications, live features

2. RabbitMQ (Most Common)

✅ Durable queues, reliable delivery
✅ Excellent documentation
✅ Multiple messaging patterns (Pub/Sub, Work Queues, RPC)
Use for: Backend service communication, reliable messaging

3. Apache Kafka (High-Volume)

✅ Horizontal scaling to millions of messages/second
✅ Built-in event sourcing
✅ Consumer groups for parallel processing
Use for: Data pipelines, event streaming, analytics

4. Google Cloud Pub/Sub (Managed)

✅ Fully managed, no infrastructure
✅ Automatic scaling
❌ Vendor lock-in
Use for: Serverless, cloud-native applications

5. AWS SNS/SQS (AWS Ecosystem)

✅ Native AWS integration
✅ Simple to set up
Use for: AWS-based systems, fan-out patterns

How to Implement Pub/Sub (Node.js Example)

Using Redis (Simple Real-Time Updates)

const redis = require('redis');
const publisher = redis.createClient();
const subscriber = redis.createClient();

// Subscribe
subscriber.subscribe('notifications', (message) => {
  console.log('Received:', message);
});

// Publish
publisher.publish('notifications', 'Hello subscribers!');

Using RabbitMQ (Reliable Messaging)

const amqp = require('amqplib');

async function publishMessage() {
  const connection = await amqp.connect('amqp://localhost');
  const channel = await connection.createChannel();

  // Declare exchange (Pub/Sub broker)
  await channel.assertExchange('events', 'fanout', { durable: true });

  // Publish message
  channel.publish('events', '', Buffer.from('Order created'));
}

async function subscribeToMessages() {
  const connection = await amqp.connect('amqp://localhost');
  const channel = await connection.createChannel();

  // Subscribe to exchange
  const queue = await channel.assertQueue('', { exclusive: true });
  await channel.bindQueue(queue.queue, 'events', '');

  // Consume messages
  channel.consume(queue.queue, (message) => {
    console.log('Message:', message.content.toString());
    channel.ack(message);
  });
}

Using Google Cloud Pub/Sub

const {PubSub} = require('@google-cloud/pubsub');
const pubsub = new PubSub();

// Publisher
async function publishMessage() {
  const topic = pubsub.topic('orders');
  const message = { orderId: '123', amount: 99.99 };
  await topic.publish(Buffer.from(JSON.stringify(message)));
}

// Subscriber
async function subscribeToMessages() {
  const subscription = pubsub.subscription('orders-subscription');
  const messageHandler = (message) => {
    console.log('Received:', message.data.toString());
    message.ack();
  };
  subscription.on('message', messageHandler);
}

Pub/Sub Trade-offs

Advantages

✅ Loose coupling - Services don't know about each other ✅ Scalability - Easy to add new subscribers ✅ Reliability - Message brokers ensure delivery (with durable options) ✅ Flexibility - Same event can trigger multiple workflows

Disadvantages

❌ Complexity - Harder to debug and trace message flow ❌ Latency - Asynchronous = slightly slower than direct calls ❌ Message ordering - Hard to guarantee message ordering across subscribers ❌ Infrastructure - Requires running a message broker ❌ Testing - More complex to test asynchronous workflows

Common Pitfalls and How to Avoid Them

1. Lost Messages

Problem: Using Redis Pub/Sub without persistence Solution: Use durable queues (RabbitMQ) or message logs (Kafka)

2. Subscriber Failures

Problem: Subscriber crashes and messages are lost Solution: Implement acknowledgments and dead-letter queues

3. Message Ordering

Problem: Subscribers process messages out of order Solution: Use single-partition topics (Kafka) or message ordering guarantees

4. Duplicate Processing

Problem: Same message processed twice Solution: Implement idempotent handlers or deduplication logic

Best Practices

Use Topics Wisely - Organize by domain events (user., order., etc.)
Version Your Messages - Add version field to handle schema changes
Use Dead-Letter Queues - For messages that fail processing
Monitor Your Broker - Track queue lengths, lag, consumer health
Implement Timeouts - Don't let subscribers wait indefinitely
Document Your Events - What data does each event contain?
Test Failure Scenarios - What if a subscriber is down?

Conclusion

Pub/Sub is essential for building scalable, loosely-coupled distributed systems. Whether you choose Redis for simplicity, RabbitMQ for reliability, or Kafka for high-volume streaming, the pattern remains the same:

Decouple your services, publish events asynchronously, and let subscribers react independently.

For deeper learning, check out our complete guide on message-driven architecture and explore production implementations in your tech stack.