Hermes Deep Dive: The Developer’s Guide

Published: August 21, 2026

What is Hermes?

In the rapidly evolving landscape of developer tools, few releases have generated as much quiet buzz as Hermes. Originally conceived in 2024 as an internal tool at a Y Combinator-backed infrastructure startup, Hermes was open-sourced in March 2025 and has since become the de facto standard for event-driven workflow orchestration in microservices architectures.

But let’s be precise about what Hermes actually is: it’s a lightweight, language-agnostic message routing and workflow state machine that sits between your application services and your message broker. Think of it as a declarative middleware layer that handles the messy parts of distributed systems—retries, dead-lettering, idempotency, and state persistence—without requiring you to adopt a heavy orchestration platform like Temporal or Airflow.

Core Value Proposition

Hermes’s core value proposition is “state without servers.” Traditional workflow engines require you to run dedicated worker pools, maintain databases for workflow state, and manage complex SDKs. Hermes instead embeds directly into your existing services as a lightweight library (available in Go, Rust, TypeScript, and Python), using your existing message broker (Kafka, RabbitMQ, or NATS) as the source of truth for workflow state.

The key differentiator is Hermes’s deterministic replay engine. When a workflow step fails, Hermes doesn’t just retry the message—it reconstructs the exact execution context from the message broker’s log, replays the failed step with full state, and continues. This means you get exactly-once semantics without the overhead of a separate transaction log.

What Makes It Different

AspectHermesTemporalPlain Message Queues
State managementEmbedded, broker-backedExternal databaseManual
Language support4 languages (Go, Rust, TS, Python)10+ languagesAny
Infrastructure footprintZero (uses existing broker)3 services (frontend, backend, DB)Zero
Learning curve30 minutes2-3 daysMinimal
Exactly-once processing✅ Deterministic replay✅❌

🚀 Getting Started

Installation

Hermes is distributed as a set of language-specific packages. Here’s how to install each:

# Go (most mature implementation)
go get github.com/hermes-workflow/hermes-go@v1.4.2

# Rust
cargo add hermes-rs --version 0.9.1

# TypeScript/Node.js
npm install @hermes-workflow/core

# Python
pip install hermes-workflow

For this tutorial, we’ll focus on the Go implementation, which is the reference implementation and has the most complete feature set.

Configuration

Hermes uses a YAML configuration file that defines your workflows declaratively. Here’s a minimal configuration:

# hermes.yaml
version: "1.0"
broker:
  type: "kafka"
  brokers: ["localhost:9092"]
  consumer_group: "hermes-orders"
  
workflows:
  - name: "order_processing"
    steps:
      - name: "validate_order"
        timeout: "30s"
        retries: 3
        on_failure: "dead_letter"
      - name: "charge_payment"
        timeout: "1m"
        retries: 5
        on_failure: "compensate"
      - name: "fulfill_order"
        timeout: "5m"
        retries: 2
        on_failure: "notify_support"
        
dead_letter:
  topic: "orders-dlq"
  retention: "7d"

Initialize Hermes in your application:

package main

import (
    "context"
    "github.com/hermes-workflow/hermes-go"
)

func main() {
    h, err := hermes.New("hermes.yaml")
    if err != nil {
        panic(err)
    }
    
    // Register step handlers
    h.Handle("validate_order", validateOrderHandler)
    h.Handle("charge_payment", chargePaymentHandler)
    h.Handle("fulfill_order", fulfillOrderHandler)
    
    ctx := context.Background()
    if err := h.Start(ctx); err != nil {
        panic(err)
    }
    
    // Wait for shutdown signal
    <-ctx.Done()
}

💡 Core Features

Feature 1: Deterministic Replay Engine

The deterministic replay engine is Hermes’s signature feature. When a workflow step fails, Hermes captures the exact state—including all input parameters, intermediate computations, and side-effect outcomes—and stores it as a special “checkpoint” message in the broker.

How it works:

func chargePaymentHandler(ctx context.Context, input *hermes.Message) (*hermes.Message, error) {
    // This handler might fail due to external API issues
    paymentService := getPaymentService()
    
    // Hermes automatically snapshots the input state
    result, err := paymentService.Charge(input.Data)
    if err != nil {
        // Return error - Hermes will capture state and retry
        return nil, fmt.Errorf("payment failed: %w", err)
    }
    
    return hermes.NewMessage(result), nil
}

When chargePaymentHandler fails, Hermes doesn’t just retry—it records the complete execution context (including the state of any local variables marked with hermes.State("varName")). On retry, it restores that exact state before re-invoking the handler.

Real-world application: In a fintech scenario, this prevents double-charging. If the payment API times out but actually processed the charge, Hermes’s replay engine can detect the side-effect signature and skip the actual charge on retry, returning the stored result instead.

Feature 2: Compensation Sagas

Hermes implements the saga pattern natively, but with a twist: automatic compensation graph construction. Instead of manually writing compensating transactions, you define compensation handlers for each step, and Hermes builds the reverse execution graph automatically.

func fulfillOrderHandler(ctx context.Context, input *hermes.Message) (*hermes.Message, error) {
    // This step might fail after partially completing
    err := inventoryService.ReserveItems(input.Data)
    if err != nil {
        return nil, err
    }
    
    // If this fails, we need to unreserve the items
    err = shippingService.CreateShipment(input.Data)
    if err != nil {
        // Hermes will automatically call the compensation for this step
        return nil, err
    }
    
    return hermes.NewMessage(map[string]interface{}{
        "shipment_id": shipment.ID,
    }), nil
}

// Compensation handler
func compensateFulfillOrder(ctx context.Context, original *hermes.Message) error {
    return inventoryService.UnreserveItems(original.Data)
}

Real-world application: In e-commerce, if order fulfillment fails after inventory reservation, Hermes automatically invokes compensateFulfillOrder to release the inventory, then walks backward through the saga to compensate charge_payment (issuing a refund) and validate_order (releasing any holds).

Feature 3: Adaptive Retry with Circuit Breaking

Hermes’s retry mechanism is adaptive, not just fixed-interval. It uses a token bucket algorithm per step, combined with circuit breaking based on error rates.

// In hermes.yaml
workflows:
  - name: "order_processing"
    steps:
      - name: "charge_payment"
        retry_policy:
          max_retries: 5
          initial_backoff: "100ms"
          max_backoff: "10s"
          multiplier: 2.0
          circuit_breaker:
            failure_threshold: 3
            reset_timeout: "30s"
            half_open_retries: 2

The circuit breaker works at the step level: if a step fails 3 times within a window, Hermes opens the circuit and immediately fails subsequent invocations without attempting them. After 30 seconds, it enters half-open state, allowing 2 test requests. If those succeed, the circuit closes; otherwise, it reopens.

Real-world application: When integrating with a flaky third-party API, this prevents cascading failures. If the payment gateway is down, Hermes stops attempting charges after 3 failures, fails fast, and routes to the dead-letter queue. When the gateway recovers, Hermes automatically resumes processing from the half-open state.

🛠️ Advanced Workflows

Workflow 1: Multi-Service Fan-Out with Aggregation

This workflow demonstrates Hermes’s ability to orchestrate parallel operations across services and aggregate results.

# hermes.yaml
workflows:
  - name: "fraud_check"
    steps:
      - name: "split_checks"
        fan_out:
          parallel_steps:
            - name: "credit_check"
              service: "credit-service"
            - name: "identity_check"
              service: "identity-service"
            - name: "velocity_check"
              service: "velocity-service"
        fan_in:
          strategy: "wait_all"
          timeout: "45s"
      - name: "aggregate_results"
        timeout: "10s"

func aggregateResultsHandler(ctx context.Context, input *hermes.AggregateMessage) (*hermes.Message, error) {
    // input.Results contains results from all parallel steps
    creditResult := input.Results["credit_check"]
    identityResult := input.Results["identity_check"]
    velocityResult := input.Results["velocity_check"]
    
    // Combine results with weighted scoring
    score := creditResult.Score*0.4 + identityResult.Score*0.3 + velocityResult.Score*0.3
    
    if score < 0.7 {
        return nil, fmt.Errorf("fraud score too high: %f", score)
    }
    
    return hermes.NewMessage(map[string]interface{}{
        "fraud_score": score,
        "approved": true,
    }), nil
}

Real-world application: A payment processor checks credit history, identity verification, and transaction velocity in parallel. If any check fails or the aggregate score is too low, the transaction is flagged. The fan-out timeout ensures the workflow completes within the payment gateway’s 45-second window.

Workflow 2: Event Sourcing with External System Integration

This workflow shows how to use Hermes with event sourcing and handle external system calls with idempotency.

# hermes.yaml
workflows:
  - name: "inventory_sync"
    steps:
      - name: "consume_inventory_event"
        event_source: "inventory-events"
        event_filter: "type == 'stock_updated'"
      - name: "transform_event"
        transform: "jsonata"
        transform_expression: |
          {
            "product_id": $string(data.product_id),
            "new_stock": data.quantity,
            "timestamp": $fromMillis(data.timestamp)
          }
      - name: "sync_to_erp"
        idempotency_key: "product_id + timestamp"
        service: "erp-sync-service"

func syncToErpHandler(ctx context.Context, input *hermes.Message) (*hermes.Message, error) {
    // The idempotency key ensures we don't double-sync
    idempotencyKey := input.ID
    
    // Check if we've already processed this event
    if erpService.HasProcessed(idempotencyKey) {
        return hermes.NewMessage(map[string]interface{}{
            "status": "already_synced",
            "skipped": true,
        }), nil
    }
    
    // Perform the sync
    err := erpService.Sync(input.Data)
    if err != nil {
        return nil, err
    }
    
    return hermes.NewMessage(map[string]interface{}{
        "status": "synced",
        "idempotency_key": idempotencyKey,
    }), nil
}

Real-world application: An e-commerce platform syncs inventory changes to an external ERP system. The idempotency key (product ID + timestamp) ensures that even if the same event is processed multiple times (due to broker redelivery or replay), the ERP system isn’t updated twice.

📊 Comparison with Alternatives

FeatureHermesTemporalAWS Step Functions
State persistence✅ Broker-based✅ Database-backed✅ AWS-managed
Exactly-once semantics✅ Deterministic replay✅❌ At-least-once
Compensation sagas✅ Automatic✅ Manual✅ Manual
Circuit breaking✅ Built-in❌ Requires custom❌
Language support4 languages10+ languagesSDKs for major languages
Self-hosted✅ Zero infrastructure✅ Requires 3 services❌ AWS-only
Local development✅ docker-compose with Kafka✅ docker-compose with Temporal❌ AWS account required
CostFree (open source)Free (open source)Pay per state transition
Learning curveLow (30 min)Medium (2-3 days)Medium
Debugging✅ Replay from broker log✅ Web UI✅ AWS Console
Failure recovery✅ Automated✅ Automated✅ Automated
Throughput10k+ events/sec (benchmarked)5k events/sec1k events/sec

Key takeaway: Hermes wins for teams already using Kafka or RabbitMQ who want workflow orchestration without adding infrastructure. Temporal is better for complex workflows spanning many services with diverse language requirements. Step Functions is best for AWS-locked teams that don’t want to manage infrastructure.

🎯 Pro Tips

Tip 1: Use Message Versioning from Day One

Hermes doesn’t enforce schema evolution, but you’ll thank yourself later. Add a schema_version field to all messages:

func validateOrderHandler(ctx context.Context, input *hermes.Message) (*hermes.Message, error) {
    version, ok := input.Data["schema_version"]
    if !ok {
        // Migrate old messages
        migrated := migrateV1ToV2(input.Data)
        return hermes.NewMessage(migrated), nil
    }
    // ... rest of handler
}

Tip 2: Leverage Hermes’s Built-in Metrics

Hermes exposes Prometheus metrics by default. Track these key metrics:

# Key metrics to monitor
hermes_workflow_duration_seconds{workflow="order_processing"}
hermes_step_retries_total{step="charge_payment"}
hermes_circuit_breaker_state{step="charge_payment"}
hermes_dead_letter_messages_total{workflow="order_processing"}

Set up alerts for:

Tip 3: Use the Hermes CLI for Local Debugging

The Hermes CLI (hermes-cli) is invaluable for local development:

# Replay a specific workflow from the broker log
hermes-cli replay --workflow order_processing --message-id abc123

# Inspect workflow state
hermes-cli inspect --workflow order_processing --id order-456

# Test a step handler locally
hermes-cli test-step --step charge_payment --input payment_test.json

# Generate a workflow diagram
hermes-cli visualize --workflow order_processing --format mermaid

Tip 4: Design for Idempotency Even with Hermes’s Guarantees

While Hermes provides exactly-once semantics within a single workflow execution, external systems might have their own side effects. Always include idempotency keys when calling external services:

func chargePaymentHandler(ctx context.Context, input *hermes.Message) (*hermes.Message, error) {
    // Generate idempotency key from workflow ID + step name
    idempotencyKey := fmt.Sprintf("%s:%s", input.WorkflowID, "charge_payment")
    
    // Pass to external service
    result, err := paymentService.Charge(input.Data, idempotencyKey)
    // ...
}

Tip 5: Use Hermes’s Built-in Dead Letter Queue Management

Don’t just let dead letters sit. Hermes provides a dead letter replay mechanism:

# Replay dead letter messages after fixing the issue
hermes-cli replay-dlq --workflow order_processing --topic orders-dlq

# Or set up automatic replay with backoff
hermes-cli schedule-replay --workflow order_processing --interval 1h --max-retries 3

🔗 Resources

Official Documentation

Community

Learning Resources

Conclusion

Hermes represents a pragmatic middle ground in the workflow orchestration space. It doesn’t try to replace your entire infrastructure—instead, it augments what you already have with battle-tested patterns for state management, retries, and compensation. For teams already invested in Kafka or RabbitMQ, Hermes offers a path to robust workflow orchestration without the operational overhead of dedicated workflow engines.

The deterministic replay engine alone is worth the adoption cost if you’re dealing with fintech, e-commerce, or any domain where exactly-once processing matters. And with the active community and growing ecosystem, Hermes is positioned to become a standard tool in the modern developer’s toolkit.

Have you tried Hermes? Share your experience in the comments below, or join the Discord community to connect with other users.

Have questions? Join our Discord community or follow us on X.