Event-Driven Architecture: Principles and Patterns
A comprehensive guide to the foundational principles and design patterns behind event-driven architecture, with practical TypeScript examples for building resilient, loosely coupled systems.
Event-driven architecture (EDA) represents a fundamental shift in how we think about building software systems. Rather than relying on synchronous request-response cycles, EDA models computation as a series of events flowing through a system, each representing something meaningful that has occurred. For teams building real-time data processing services, understanding these principles is not just academic --- it is the foundation upon which scalable, maintainable, and resilient systems are built.
In this article, we explore the core principles of event-driven architecture, examine the most important design patterns, and demonstrate how to implement them in TypeScript using Starburst as a reference for real-world event processing.
What Makes Architecture "Event-Driven"
At its core, event-driven architecture is built on a simple idea: components in a system communicate by producing and consuming events rather than directly calling each other. An event is an immutable record of something that has happened --- a user signed up, an order was placed, a sensor reading was captured. This seemingly small distinction has profound implications for how systems are designed.
The three pillars of EDA are event producers, event channels, and event consumers. Producers emit events without knowledge of who will consume them. Channels (often implemented as message brokers or event streams) transport and buffer events. Consumers subscribe to events they care about and react accordingly.
// Defining a strongly-typed event in TypeScript
interface DomainEvent<T extends string = string, P = unknown> {
readonly eventId: string;
readonly eventType: T;
readonly timestamp: Date;
readonly aggregateId: string;
readonly version: number;
readonly payload: P;
readonly metadata: EventMetadata;
}
interface EventMetadata {
readonly correlationId: string;
readonly causationId: string;
readonly userId?: string;
readonly source: string;
}
// Concrete event types
interface OrderPlacedEvent extends DomainEvent<"OrderPlaced", {
customerId: string;
items: Array<{ productId: string; quantity: number; price: number }>;
totalAmount: number;
}> {}
interface PaymentReceivedEvent extends DomainEvent<"PaymentReceived", {
orderId: string;
amount: number;
paymentMethod: string;
transactionId: string;
}> {}This type-safe approach ensures that events flowing through your system are well-defined and that any consumer can rely on a specific structure. The correlationId and causationId fields in the metadata are particularly important --- they allow you to trace the chain of events that led to any particular state change, which becomes invaluable during debugging.
The Producer-Consumer Pattern
The most fundamental pattern in EDA is the producer-consumer pattern. A producer emits events into a channel, and one or more consumers process those events independently. This decoupling is what gives event-driven systems their flexibility and scalability.
// Event bus abstraction
interface EventBus {
publish<E extends DomainEvent>(event: E): Promise<void>;
subscribe<E extends DomainEvent>(
eventType: string,
handler: EventHandler<E>
): Subscription;
}
type EventHandler<E extends DomainEvent> = (event: E) => Promise<void>;
interface Subscription {
unsubscribe(): void;
}
// In-process event bus implementation
class InMemoryEventBus implements EventBus {
private handlers = new Map<string, Set<EventHandler<any>>>();
async publish<E extends DomainEvent>(event: E): Promise<void> {
const eventHandlers = this.handlers.get(event.eventType);
if (!eventHandlers) return;
const promises = Array.from(eventHandlers).map(handler =>
handler(event).catch(error => {
console.error(
`Handler failed for event ${event.eventId}:`,
error
);
})
);
await Promise.allSettled(promises);
}
subscribe<E extends DomainEvent>(
eventType: string,
handler: EventHandler<E>
): Subscription {
if (!this.handlers.has(eventType)) {
this.handlers.set(eventType, new Set());
}
this.handlers.get(eventType)!.add(handler);
return {
unsubscribe: () => {
this.handlers.get(eventType)?.delete(handler);
},
};
}
}In production systems, you would replace this in-memory implementation with one backed by a message broker such as Apache Kafka, RabbitMQ, or a managed streaming service. The key insight is that the interface remains the same --- producers and consumers are decoupled from the transport mechanism.
One critical consideration is whether your events should be delivered with "at-least-once" or "exactly-once" semantics. At-least-once delivery is far simpler to implement and is sufficient for most use cases, provided your consumers are idempotent. This means that processing the same event twice should produce the same result as processing it once.
// Idempotent event handler with deduplication
class IdempotentHandler<E extends DomainEvent> {
private processedEvents = new Set<string>();
constructor(
private readonly handler: EventHandler<E>,
private readonly deduplicationStore: DeduplicationStore
) {}
async handle(event: E): Promise<void> {
const alreadyProcessed = await this.deduplicationStore.exists(
event.eventId
);
if (alreadyProcessed) {
console.log(`Event ${event.eventId} already processed, skipping.`);
return;
}
await this.handler(event);
await this.deduplicationStore.mark(event.eventId);
}
}
interface DeduplicationStore {
exists(eventId: string): Promise<boolean>;
mark(eventId: string): Promise<void>;
}Event Choreography vs. Orchestration
Two competing approaches exist for coordinating work across multiple services in an event-driven system: choreography and orchestration.
In choreography, each service listens for events and decides independently what to do. There is no central coordinator. When an order is placed, the inventory service reserves stock because it heard the OrderPlaced event. The payment service initiates a charge for the same reason. The notification service sends a confirmation email. Each service acts autonomously based on the events it observes.
// Choreography: Each service reacts independently
class InventoryService {
constructor(private eventBus: EventBus) {
this.eventBus.subscribe<OrderPlacedEvent>(
"OrderPlaced",
this.handleOrderPlaced.bind(this)
);
}
private async handleOrderPlaced(event: OrderPlacedEvent): Promise<void> {
for (const item of event.payload.items) {
await this.reserveStock(item.productId, item.quantity);
}
await this.eventBus.publish({
eventId: generateId(),
eventType: "StockReserved",
timestamp: new Date(),
aggregateId: event.aggregateId,
version: 1,
payload: { orderId: event.aggregateId },
metadata: {
correlationId: event.metadata.correlationId,
causationId: event.eventId,
source: "inventory-service",
},
});
}
private async reserveStock(
productId: string,
quantity: number
): Promise<void> {
// Stock reservation logic
}
}In orchestration, a central process (often called a saga or a process manager) coordinates the workflow. It listens for events and issues commands to the appropriate services to carry out the next step.
// Orchestration: A saga coordinates the workflow
class OrderFulfillmentSaga {
private state: SagaState = "INITIATED";
constructor(
private eventBus: EventBus,
private commandBus: CommandBus
) {
this.eventBus.subscribe("OrderPlaced", this.onOrderPlaced.bind(this));
this.eventBus.subscribe("StockReserved", this.onStockReserved.bind(this));
this.eventBus.subscribe("PaymentProcessed", this.onPaymentProcessed.bind(this));
this.eventBus.subscribe("PaymentFailed", this.onPaymentFailed.bind(this));
}
private async onOrderPlaced(event: OrderPlacedEvent): Promise<void> {
this.state = "RESERVING_STOCK";
await this.commandBus.send({
type: "ReserveStock",
payload: { orderId: event.aggregateId, items: event.payload.items },
});
}
private async onStockReserved(event: DomainEvent): Promise<void> {
this.state = "PROCESSING_PAYMENT";
await this.commandBus.send({
type: "ProcessPayment",
payload: { orderId: event.aggregateId },
});
}
private async onPaymentProcessed(event: DomainEvent): Promise<void> {
this.state = "COMPLETED";
await this.commandBus.send({
type: "ShipOrder",
payload: { orderId: event.aggregateId },
});
}
private async onPaymentFailed(event: DomainEvent): Promise<void> {
this.state = "COMPENSATING";
await this.commandBus.send({
type: "ReleaseStock",
payload: { orderId: event.aggregateId },
});
}
}
type SagaState =
| "INITIATED"
| "RESERVING_STOCK"
| "PROCESSING_PAYMENT"
| "COMPLETED"
| "COMPENSATING";Choreography works well for simple flows with few participants. Orchestration is better when the workflow is complex, has many steps, or requires compensating actions (rollbacks). In practice, most systems use a combination of both approaches.
Handling Failure and Back-Pressure
One of the often-overlooked aspects of event-driven architecture is failure handling. When a consumer fails to process an event, what happens? The answer depends on your delivery guarantees and your error-handling strategy.
A common pattern is the dead letter queue (DLQ). When an event cannot be processed after a configurable number of retries, it is moved to a separate queue for manual inspection and reprocessing.
class ResilientEventProcessor<E extends DomainEvent> {
private readonly maxRetries: number;
private readonly retryDelayMs: number;
constructor(
private handler: EventHandler<E>,
private deadLetterQueue: DeadLetterQueue,
options: { maxRetries?: number; retryDelayMs?: number } = {}
) {
this.maxRetries = options.maxRetries ?? 3;
this.retryDelayMs = options.retryDelayMs ?? 1000;
}
async process(event: E): Promise<void> {
let lastError: Error | undefined;
for (let attempt = 1; attempt <= this.maxRetries; attempt++) {
try {
await this.handler(event);
return;
} catch (error) {
lastError = error instanceof Error ? error : new Error(String(error));
console.warn(
`Attempt ${attempt}/${this.maxRetries} failed for ` +
`event ${event.eventId}: ${lastError.message}`
);
if (attempt < this.maxRetries) {
await this.delay(this.retryDelayMs * Math.pow(2, attempt - 1));
}
}
}
await this.deadLetterQueue.enqueue(event, lastError!);
console.error(
`Event ${event.eventId} moved to dead letter queue ` +
`after ${this.maxRetries} failed attempts.`
);
}
private delay(ms: number): Promise<void> {
return new Promise(resolve => setTimeout(resolve, ms));
}
}
interface DeadLetterQueue {
enqueue(event: DomainEvent, error: Error): Promise<void>;
peek(count: number): Promise<Array<{ event: DomainEvent; error: string }>>;
reprocess(eventId: string): Promise<void>;
}Back-pressure is equally important. When events arrive faster than a consumer can process them, you need a strategy to prevent the system from being overwhelmed. Common approaches include rate limiting the consumer, buffering events in a bounded queue, or signaling the producer to slow down.
class BackPressureBuffer<E extends DomainEvent> {
private buffer: E[] = [];
private processing = false;
constructor(
private handler: EventHandler<E>,
private readonly maxBufferSize: number = 10000,
private readonly batchSize: number = 100
) {}
async enqueue(event: E): Promise<void> {
if (this.buffer.length >= this.maxBufferSize) {
throw new Error(
"Back-pressure limit reached. Cannot accept more events."
);
}
this.buffer.push(event);
if (!this.processing) {
this.processBuffer();
}
}
private async processBuffer(): Promise<void> {
this.processing = true;
while (this.buffer.length > 0) {
const batch = this.buffer.splice(0, this.batchSize);
await Promise.allSettled(
batch.map(event => this.handler(event))
);
}
this.processing = false;
}
}Practical Tips for Getting Started
Adopting event-driven architecture does not require rewriting your entire system overnight. Here are practical steps to start:
Start with a single bounded context. Pick one area of your application where events are a natural fit --- perhaps order processing, user activity tracking, or notification delivery. Build the event infrastructure for that context first and expand from there.
Invest in event schema design early. Events are contracts between producers and consumers. Getting the schema right from the beginning saves enormous pain later. Use a schema registry and version your events from day one.
Make consumers idempotent. This is non-negotiable. Network failures, retries, and duplicate deliveries will happen. If your consumers cannot safely process the same event twice, your system will produce incorrect results.
Log and trace everything. Correlation IDs and causation IDs are essential. Without them, debugging a distributed event-driven system is like finding a needle in a haystack --- blindfolded.
Test with realistic event volumes. Event-driven systems behave differently under load. A system that works perfectly with ten events per second may fall apart at ten thousand. Load test early and often.
Conclusion
Event-driven architecture provides a powerful foundation for building systems that are resilient, scalable, and adaptable to change. The key principles --- loose coupling through events, asynchronous communication, and independent consumers --- enable teams to build and evolve complex systems without creating tangled dependencies.
The patterns we have explored --- producer-consumer, choreography, orchestration, dead letter queues, and back-pressure management --- form the toolkit for implementing EDA effectively. With TypeScript's strong type system, you can add compile-time safety to these patterns, catching errors before they reach production.
Whether you are building a real-time data pipeline, a microservices architecture, or a complex event-processing platform like Starburst, these principles and patterns will serve as your guide. Start small, iterate often, and let the events tell the story of your system.
Related Articles
Monitoring Event-Driven Systems at Scale
A practical guide to building comprehensive monitoring and observability for event-driven systems, covering metrics, distributed tracing, alerting strategies, and operational dashboards for maintaining healthy event processing pipelines.
Migrating to Event-Driven Architecture
A practical guide for planning and executing a migration from traditional request-response systems to event-driven architecture, covering assessment frameworks, migration strategies, risk management, and organizational change.
Real-Time Data Processing: Business Impact and ROI
An exploration of the business value of real-time data processing, covering measurable ROI, competitive advantages, and practical frameworks for justifying investment in event-driven infrastructure.