Skip to content
XVI parts · C patterns
A ten-by-ten grid of one hundred cells filled in reading order across sixteen parts

Book I. · Proposition VIII.

100 Patterns for Production Go and Python

To prove: That a hundred design patterns, concurrency idioms, resilience strategies and architectural habits are what separate production Go and Python systems from prototypes, catalogued with an example for each.

Most codebases don’t fail for lack of talent. They fail because the team never settled on the structural patterns that keep complexity from compounding. A service starts out with clean function signatures and a clear purpose. Six months later its error handling is inconsistent, its concurrency is ad hoc, its configuration is scattered across environment variables and hardcoded defaults, and every new feature means touching five files that shouldn’t know about each other.

Patterns, I would argue, are not academic abstractions. They’re the vocabulary that makes a system predictable. When every service in a codebase wraps errors the same way, shuts down the same way and injects its dependencies the same way, a new team member can find their way around any service after reading just one. When the patterns are missing, every service is a snowflake that needs its own orientation.

What follows is a catalog of one hundred patterns I reach for again and again in production Go and Python. They cover interface design, concurrency, resilience, data access, observability, testing and architecture. Each comes with a short example, in Go where the pattern is native to the language, in Python where the idiom differs in an interesting way, and in both where the contrast teaches something.

It isn’t a tutorial so much as a reference (and a long one, I’ll grant). Read it from end to end to build a mental model, or jump to whichever section addresses the problem in front of you today.


Part I: Interface Design and Composition Link to heading

1. Small, Consumer-Owned Interfaces Link to heading

Define interfaces where they’re consumed, not where they’re implemented. A service that needs to read users should declare a two-method interface of its own rather than import a twenty-method one from the repository package. Small interfaces are easier to mock and easier to satisfy, and they don’t ripple when something unrelated changes.

// In the consumer package, not the provider package.
// The service declares exactly what it needs.
package userservice

type UserReader interface {
    GetByID(ctx context.Context, id string) (User, error)
    GetByEmail(ctx context.Context, email string) (User, error)
}

type Service struct {
    users UserReader // not *postgres.UserRepository
}
# Python equivalent using Protocol — defined where consumed.
from typing import Protocol

class UserReader(Protocol):
    def get_by_id(self, user_id: str) -> User | None: ...
    def get_by_email(self, email: str) -> User | None: ...

class UserService:
    def __init__(self, users: UserReader) -> None:
        self._users = users

2. Context Propagation Everywhere Link to heading

Every function that does I/O, crosses a service boundary or might be cancelled should take a context.Context as its first parameter. The context carries deadlines, cancellation and request-scoped values such as trace IDs, and leaving it out makes graceful shutdown impossible and distributed tracing incomplete.

// Context flows from HTTP handler through service to repository.
func (s *Service) ProcessOrder(ctx context.Context, orderID string) error {
    order, err := s.orders.GetByID(ctx, orderID)
    if err != nil {
        return fmt.Errorf("fetch order %s: %w", orderID, err)
    }

    // Context carries the deadline — if the caller cancels,
    // the payment call respects it.
    return s.payments.Charge(ctx, order.CustomerID, order.Total)
}
# Python does not have context.Context natively, but the pattern
# applies through asyncio task cancellation and contextvars.
import contextvars

request_id: contextvars.ContextVar[str] = contextvars.ContextVar("request_id")

async def process_order(order_id: str) -> None:
    # contextvars propagate automatically through async calls.
    rid = request_id.get()
    logger.info("processing order", order_id=order_id, request_id=rid)
    order = await orders.get_by_id(order_id)
    await payments.charge(order.customer_id, order.total)

3. Functional Options Pattern Link to heading

Constructors with many parameters quickly become unreadable. Functional options give you a clean API for optional configuration, and adding a new option later doesn’t break any existing caller. Each option is simply a function that modifies a config struct.

type Server struct {
    addr         string
    readTimeout  time.Duration
    writeTimeout time.Duration
    logger       Logger
}

type Option func(*Server)

func WithReadTimeout(d time.Duration) Option {
    return func(s *Server) { s.readTimeout = d }
}

func WithLogger(l Logger) Option {
    return func(s *Server) { s.logger = l }
}

// NewServer applies defaults, then overrides with caller options.
func NewServer(addr string, opts ...Option) *Server {
    s := &Server{
        addr:         addr,
        readTimeout:  5 * time.Second,
        writeTimeout: 10 * time.Second,
        logger:       noopLogger{},
    }
    for _, opt := range opts {
        opt(s)
    }
    return s
}

// Usage: clean, self-documenting, extensible.
srv := NewServer(":8080",
    WithReadTimeout(10*time.Second),
    WithLogger(logger),
)
# Python achieves similar ergonomics with dataclass defaults
# and keyword-only arguments.
from dataclasses import dataclass, field

@dataclass(frozen=True)
class ServerConfig:
    addr: str
    read_timeout: float = 5.0
    write_timeout: float = 10.0
    logger: Logger = field(default_factory=NoopLogger)

class Server:
    def __init__(self, addr: str, **kwargs) -> None:
        self._config = ServerConfig(addr=addr, **kwargs)

4. Explicit Dependency Injection Without Frameworks Link to heading

Pass dependencies in through constructors. There are no global variables, no service locators and no annotation-driven magic, and the whole dependency graph is visible in the composition root, the one place where everything gets wired together.

func main() {
    // Composition root: every dependency is explicit.
    db := postgres.NewClient(cfg.DatabaseURL)
    cache := redis.NewClient(cfg.RedisURL)
    logger := slog.New(slog.NewJSONHandler(os.Stdout, nil))

    userRepo := postgres.NewUserRepository(db)
    userCache := redis.NewUserCache(cache)
    userService := userservice.New(userRepo, userCache, logger)
    userHandler := api.NewUserHandler(userService)

    mux := http.NewServeMux()
    mux.Handle("/users", userHandler)
    http.ListenAndServe(":8080", mux)
}
# Python equivalent — no frameworks, just constructors.
def create_app(settings: Settings) -> FastAPI:
    db = AsyncpgClient(settings.database_url)
    cache = RedisClient(settings.redis_url)
    logger = structlog.get_logger()

    user_repo = PostgresUserRepository(db)
    user_service = UserService(repo=user_repo, cache=cache, logger=logger)
    user_controller = UserController(service=user_service)

    app = FastAPI()
    app.include_router(user_controller.router)
    return app

5. Composition Over Inheritance Link to heading

Go has no inheritance at all, and Python has it but should mostly avoid it for structural purposes. Embed structs in Go and compose objects in Python. Inheritance produces rigid hierarchies, whereas composition produces assemblies you can rearrange.

// Composition via embedding — Logger gains both capabilities
// without inheriting from either.
type TimestampWriter struct{ w io.Writer }

func (tw *TimestampWriter) Write(p []byte) (int, error) {
    prefix := []byte(time.Now().Format(time.RFC3339) + " ")
    return tw.w.Write(append(prefix, p...))
}

type PrefixWriter struct {
    Prefix string
    w      io.Writer
}

func (pw *PrefixWriter) Write(p []byte) (int, error) {
    return pw.w.Write(append([]byte(pw.Prefix), p...))
}

// Compose behaviors by wrapping.
out := &TimestampWriter{w: &PrefixWriter{Prefix: "[APP] ", w: os.Stdout}}
# Python: compose via delegation, not inheritance.
class RetryingClient:
    """Adds retry behavior to any HTTP client."""

    def __init__(self, inner: HttpClient, max_retries: int = 3) -> None:
        self._inner = inner
        self._max_retries = max_retries

    async def get(self, url: str) -> Response:
        for attempt in range(self._max_retries):
            try:
                return await self._inner.get(url)
            except TransientError:
                if attempt == self._max_retries - 1:
                    raise
                await asyncio.sleep(2 ** attempt)

6. Compile-Time Interface Satisfaction Assertions Link to heading

Catch a missing interface method at compile time rather than at runtime. A single line guarantees that a type satisfies an interface, and the compiler checks it on every build.

// If UserRepository is missing any method from Repository[User, UserParams, string],
// this line fails compilation — not a test, not a runtime panic.
var _ Repository[User, UserParams, string] = (*UserRepository)(nil)

type UserRepository struct {
    db *sql.DB
}

func (r *UserRepository) GetByID(ctx context.Context, id string) (User, error) {
    // implementation
}

func (r *UserRepository) Create(ctx context.Context, u User) (User, error) {
    // implementation
}
# Python: use runtime_checkable Protocol + assert at module level.
from typing import Protocol, runtime_checkable

@runtime_checkable
class Repository(Protocol):
    def get_by_id(self, entity_id: str) -> object | None: ...

class UserRepository:
    def get_by_id(self, entity_id: str) -> User | None:
        ...

# Module-level assertion — fails on import if contract is broken.
assert isinstance(UserRepository(), Repository)

7. Interface Segregation Link to heading

Split fat interfaces into focused ones. A consumer that only writes shouldn’t depend on an interface that also reads. Segregating them reduces coupling and makes testing simpler, since there are fewer methods to fake.

// Bad: one fat interface forces every consumer to depend on everything.
type UserStore interface {
    Create(ctx context.Context, u User) error
    GetByID(ctx context.Context, id string) (User, error)
    Update(ctx context.Context, u User) error
    Delete(ctx context.Context, id string) error
    List(ctx context.Context, params ListParams) ([]User, error)
    Count(ctx context.Context) (int64, error)
}

// Good: segregated interfaces — consumers depend only on what they use.
type UserReader interface {
    GetByID(ctx context.Context, id string) (User, error)
    List(ctx context.Context, params ListParams) ([]User, error)
}

type UserWriter interface {
    Create(ctx context.Context, u User) error
    Update(ctx context.Context, u User) error
}

// A single implementation can satisfy both.
type userRepo struct{ db *sql.DB }
var _ UserReader = (*userRepo)(nil)
var _ UserWriter = (*userRepo)(nil)

8. Factory Constructor Pattern (NewX) Link to heading

Go constructors are plain functions named NewX that validate their inputs, set defaults and return a concrete type. Python’s __init__ deserves the same discipline: validate early, and fail loudly.

type Client struct {
    baseURL    string
    httpClient *http.Client
    logger     Logger
}

// New validates and applies defaults. Callers cannot create
// a half-initialized Client by using a struct literal.
func New(baseURL string, opts ...Option) (*Client, error) {
    if baseURL == "" {
        return nil, errors.New("base URL is required")
    }
    c := &Client{
        baseURL:    strings.TrimRight(baseURL, "/"),
        httpClient: &http.Client{Timeout: 30 * time.Second},
        logger:     noopLogger{},
    }
    for _, opt := range opts {
        opt(c)
    }
    return c, nil
}

9. Interface Nil-Safety Patterns Link to heading

A nil pointer behind an interface is not a nil interface (one of Go’s more notorious gotchas). It passes a nil check and then panics when you call a method on it. Guard against it with validation in the constructor and typed nil checks.

type Logger interface {
    Info(msg string, args ...any)
}

func NewService(logger Logger) (*Service, error) {
    // This catches the case where a caller passes (*ZapLogger)(nil),
    // which is a non-nil interface wrapping a nil pointer.
    if logger == nil {
        return nil, errors.New("logger is required")
    }
    return &Service{logger: logger}, nil
}

// For optional dependencies, use a no-op implementation — never nil.
func NewServiceWithDefaults(logger Logger) *Service {
    if logger == nil {
        logger = noopLogger{}
    }
    return &Service{logger: logger}
}

Part II: Configuration and Initialization Link to heading

10. Immutable Configuration Objects Link to heading

Configuration should be loaded once, validated, and then frozen. Mutable configuration invites race conditions and makes it impossible to say what values a running service is actually using.

// Config is a value type — no pointers, no mutation after construction.
type Config struct {
    DatabaseURL string
    RedisURL    string
    Port        int
    LogLevel    string
}

func LoadConfig() (Config, error) {
    cfg := Config{
        DatabaseURL: os.Getenv("DATABASE_URL"),
        RedisURL:    os.Getenv("REDIS_URL"),
        Port:        getEnvInt("PORT", 8080),
        LogLevel:    getEnvOrDefault("LOG_LEVEL", "info"),
    }
    if cfg.DatabaseURL == "" {
        return Config{}, errors.New("DATABASE_URL is required")
    }
    return cfg, nil // returned by value — caller gets a copy
}
# Python: frozen dataclass or Pydantic BaseSettings.
from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    database_url: str
    redis_url: str
    port: int = 8080
    log_level: str = "info"

    model_config = {"frozen": True}  # immutable after creation

11. Builder-Style Configuration APIs Link to heading

When you’re constructing a complex object with many optional fields, a builder gives you a fluent, readable API that guides the caller toward a valid configuration.

type Pipeline struct {
    stages   []Stage
    retries  int
    timeout  time.Duration
    logger   Logger
}

type PipelineBuilder struct {
    p Pipeline
}

func NewPipelineBuilder() *PipelineBuilder {
    return &PipelineBuilder{p: Pipeline{retries: 3, timeout: 30 * time.Second}}
}

func (b *PipelineBuilder) WithStage(s Stage) *PipelineBuilder {
    b.p.stages = append(b.p.stages, s)
    return b
}

func (b *PipelineBuilder) WithRetries(n int) *PipelineBuilder {
    b.p.retries = n
    return b
}

func (b *PipelineBuilder) Build() (*Pipeline, error) {
    if len(b.p.stages) == 0 {
        return nil, errors.New("pipeline requires at least one stage")
    }
    return &b.p, nil
}

// Usage reads like a specification.
pipeline, err := NewPipelineBuilder().
    WithStage(validateStage).
    WithStage(transformStage).
    WithStage(publishStage).
    WithRetries(5).
    Build()

12. Lazy Initialization with sync.Once Link to heading

Defer expensive initialization until it’s first needed, but make sure it runs exactly once, even under concurrent access. In Go, sync.Once is the idiomatic tool.

type TemplateCache struct {
    once      sync.Once
    templates *template.Template
    err       error
}

func (tc *TemplateCache) Get() (*template.Template, error) {
    tc.once.Do(func() {
        tc.templates, tc.err = template.ParseGlob("templates/*.html")
    })
    return tc.templates, tc.err
}
# Python: functools.cache or a manual flag.
import functools

@functools.cache
def get_db_engine() -> Engine:
    """Created on first call, reused on subsequent calls."""
    return create_engine(settings.database_url)

13. Declarative Configuration Loaders Link to heading

Load configuration from several sources (environment variables, files, defaults) with a single declarative definition, and let the loader handle precedence: the environment overrides the file, and the file overrides the default.

type Config struct {
    Port     int    `env:"PORT"     default:"8080"  validate:"min=1,max=65535"`
    LogLevel string `env:"LOG_LEVEL" default:"info" validate:"oneof=debug info warn error"`
    DBHost   string `env:"DB_HOST"  required:"true"`
}

func Load() (Config, error) {
    var cfg Config
    if err := envconfig.Process("APP", &cfg); err != nil {
        return Config{}, fmt.Errorf("load config: %w", err)
    }
    if err := validator.New().Struct(cfg); err != nil {
        return Config{}, fmt.Errorf("validate config: %w", err)
    }
    return cfg, nil
}

14. Validation-Before-Startup Link to heading

Validate all configuration and connectivity before the service starts accepting traffic. A service that starts successfully should be fully working, not half-initialized and destined to fail on its first request.

func main() {
    cfg, err := config.Load()
    if err != nil {
        log.Fatalf("config: %v", err)
    }

    // Validate connectivity before serving.
    db, err := sql.Open("postgres", cfg.DatabaseURL)
    if err != nil {
        log.Fatalf("db connect: %v", err)
    }
    if err := db.PingContext(context.Background()); err != nil {
        log.Fatalf("db ping: %v", err)
    }

    cache, err := redis.Dial(cfg.RedisURL)
    if err != nil {
        log.Fatalf("redis: %v", err)
    }

    log.Println("all dependencies healthy, starting server")
    // ... start HTTP server
}

15. Explicit Startup Dependency Ordering Link to heading

Services often have requirements about order: the database must connect before the repositories initialize, and the cache must be ready before the service layer starts. Spell the order out in the composition root rather than relying on timing that merely happens to work.

func bootstrap(ctx context.Context, cfg Config) (*App, error) {
    // Phase 1: Infrastructure (no dependencies)
    logger := logger.New(cfg.LogLevel)
    metrics := metrics.New(cfg.MetricsAddr)

    // Phase 2: Clients (depend on infrastructure)
    db, err := postgres.Connect(ctx, cfg.DatabaseURL)
    if err != nil {
        return nil, fmt.Errorf("database: %w", err)
    }

    // Phase 3: Repositories (depend on clients)
    userRepo := postgres.NewUserRepo(db)

    // Phase 4: Services (depend on repositories)
    userSvc := userservice.New(userRepo, logger)

    // Phase 5: Transport (depend on services)
    handler := api.NewHandler(userSvc, logger, metrics)

    return &App{handler: handler, db: db}, nil
}

16. Declarative Registration / Init Patterns Link to heading

Registering plugins, codecs or drivers at init time is fine, provided the registry stays explicit and testable. Avoid init() functions that quietly modify global state.

// Registry is explicit — no hidden init() side effects.
type CodecRegistry struct {
    codecs map[string]Codec
}

func NewCodecRegistry() *CodecRegistry {
    return &CodecRegistry{codecs: make(map[string]Codec)}
}

func (r *CodecRegistry) Register(name string, c Codec) {
    r.codecs[name] = c
}

func (r *CodecRegistry) Get(name string) (Codec, error) {
    c, ok := r.codecs[name]
    if !ok {
        return nil, fmt.Errorf("unknown codec: %s", name)
    }
    return c, nil
}

// Registration happens in the composition root, not in init().
registry := NewCodecRegistry()
registry.Register("json", jsonCodec{})
registry.Register("proto", protoCodec{})

Part III: Error Handling Link to heading

17. Error Wrapping with Contextual Errors Link to heading

Wrap errors with fmt.Errorf("%w") to add context at each call site while keeping the original error available for inspection. The resulting chain tells the whole story: what failed, where, and why.

func (s *Service) GetUser(ctx context.Context, id string) (User, error) {
    user, err := s.repo.GetByID(ctx, id)
    if err != nil {
        // Adds context ("get user") while preserving the original error.
        // Callers can still use errors.Is/As to inspect the root cause.
        return User{}, fmt.Errorf("get user %s: %w", id, err)
    }
    return user, nil
}

// Produces error chains like:
// "get user abc123: query users table: connection refused"
# Python: chain exceptions with `from`.
class UserService:
    async def get_user(self, user_id: str) -> User:
        try:
            return await self._repo.get_by_id(user_id)
        except DatabaseError as exc:
            raise UserNotFoundError(
                f"get user {user_id}"
            ) from exc  # preserves the original traceback

18. Sentinel Errors and Typed Errors Link to heading

Define sentinel errors for expected conditions and typed errors for conditions that carry data, then use errors.Is to check for the former and errors.As to extract the latter.

// Sentinel errors — expected conditions, not bugs.
var (
    ErrNotFound     = errors.New("not found")
    ErrConflict     = errors.New("conflict")
    ErrUnauthorized = errors.New("unauthorized")
)

// Typed error — carries structured context.
type ValidationError struct {
    Field   string
    Message string
}

func (e *ValidationError) Error() string {
    return fmt.Sprintf("validation: %s: %s", e.Field, e.Message)
}

// Checking errors at the call site.
user, err := svc.GetUser(ctx, id)
if errors.Is(err, ErrNotFound) {
    http.Error(w, "user not found", http.StatusNotFound)
    return
}

var valErr *ValidationError
if errors.As(err, &valErr) {
    http.Error(w, valErr.Message, http.StatusBadRequest)
    return
}

19. Structured Error Classification Link to heading

Classify errors into categories that decide retry behavior and HTTP status codes. One classification function replaces a great many scattered switch statements.

type ErrorKind int

const (
    KindTransient   ErrorKind = iota // retry
    KindNotFound                      // 404
    KindValidation                    // 400
    KindUnauthorized                  // 401
    KindInternal                      // 500
)

func Classify(err error) ErrorKind {
    if errors.Is(err, ErrNotFound) {
        return KindNotFound
    }
    var valErr *ValidationError
    if errors.As(err, &valErr) {
        return KindValidation
    }
    if isTransient(err) {
        return KindTransient
    }
    return KindInternal
}

func HTTPStatus(kind ErrorKind) int {
    switch kind {
    case KindNotFound:     return http.StatusNotFound
    case KindValidation:   return http.StatusBadRequest
    case KindUnauthorized: return http.StatusUnauthorized
    case KindTransient:    return http.StatusServiceUnavailable
    default:               return http.StatusInternalServerError
    }
}

20. Retryable vs Terminal Error Distinction Link to heading

Not every error deserves a retry. A network timeout does; a validation failure doesn’t. Make the distinction explicit, so that your retry logic doesn’t waste time on errors that will never succeed.

type RetryableError struct {
    Err error
}

func (e *RetryableError) Error() string { return e.Err.Error() }
func (e *RetryableError) Unwrap() error { return e.Err }

func IsRetryable(err error) bool {
    var retryable *RetryableError
    return errors.As(err, &retryable)
}

// In the repository layer — classify at the source.
func (r *repo) Query(ctx context.Context, q string) ([]Row, error) {
    rows, err := r.db.QueryContext(ctx, q)
    if err != nil {
        if isConnectionError(err) {
            return nil, &RetryableError{Err: fmt.Errorf("query: %w", err)}
        }
        return nil, fmt.Errorf("query: %w", err) // terminal
    }
    return scan(rows)
}

21. Error Aggregation / Multi-Error Handling Link to heading

When you’re validating several fields or running operations in parallel, collect all the errors instead of stopping at the first. Go 1.20 and later provide errors.Join, and Python has exception groups.

func Validate(order Order) error {
    var errs []error
    if order.CustomerID == "" {
        errs = append(errs, errors.New("customer_id is required"))
    }
    if order.Total <= 0 {
        errs = append(errs, errors.New("total must be positive"))
    }
    if len(order.Items) == 0 {
        errs = append(errs, errors.New("at least one item is required"))
    }
    return errors.Join(errs...) // nil if no errors
}
# Python 3.11+ ExceptionGroup for parallel error collection.
async def validate_all(items: list[Item]) -> None:
    errors = []
    for item in items:
        try:
            validate_item(item)
        except ValidationError as e:
            errors.append(e)
    if errors:
        raise ExceptionGroup("validation failed", errors)

22. Panic Recovery at Service Boundaries Link to heading

A panic in Go shouldn’t take down the entire server. Recover at the outermost boundary (the HTTP middleware), log the stack trace and return a 500. Never recover inside business logic, though; a panic there is a bug to be fixed, not hidden.

func RecoveryMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        defer func() {
            if rec := recover(); rec != nil {
                stack := debug.Stack()
                slog.Error("panic recovered",
                    "error", rec,
                    "stack", string(stack),
                    "path", r.URL.Path,
                )
                http.Error(w, "internal server error", http.StatusInternalServerError)
            }
        }()
        next.ServeHTTP(w, r)
    })
}

Part IV: Context and Resource Management Link to heading

23. Timeout-Wrapped External Calls Link to heading

Every call to an external system needs a timeout. Without one, a hung upstream holds a goroutine and a connection indefinitely. Wrap external calls in a derived context that enforces a deadline.

func (c *Client) FetchUser(ctx context.Context, id string) (User, error) {
    // 5-second timeout on this specific call, regardless of parent deadline.
    ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
    defer cancel()

    req, err := http.NewRequestWithContext(ctx, http.MethodGet,
        fmt.Sprintf("%s/users/%s", c.baseURL, id), nil)
    if err != nil {
        return User{}, fmt.Errorf("build request: %w", err)
    }

    resp, err := c.httpClient.Do(req)
    if err != nil {
        return User{}, fmt.Errorf("fetch user %s: %w", id, err)
    }
    defer resp.Body.Close()

    var user User
    if err := json.NewDecoder(resp.Body).Decode(&user); err != nil {
        return User{}, fmt.Errorf("decode user: %w", err)
    }
    return user, nil
}
# Python: asyncio.timeout (3.11+) or async_timeout.
async def fetch_user(self, user_id: str) -> User:
    async with asyncio.timeout(5.0):
        resp = await self._session.get(f"/users/{user_id}")
        resp.raise_for_status()
        return User(**await resp.json())

24. Resource Cleanup with defer Link to heading

defer in Go guarantees that cleanup runs when the function exits, whether normally or by panic. Use it to close files, release locks, stop timers and roll back transactions.

func (s *Service) ProcessFile(ctx context.Context, path string) error {
    f, err := os.Open(path)
    if err != nil {
        return fmt.Errorf("open %s: %w", path, err)
    }
    defer f.Close() // guaranteed cleanup

    tx, err := s.db.BeginTx(ctx, nil)
    if err != nil {
        return fmt.Errorf("begin tx: %w", err)
    }
    defer tx.Rollback() // no-op if already committed

    // ... process file, insert records ...

    return tx.Commit() // if this succeeds, deferred Rollback is a no-op
}
# Python: context managers serve the same role.
async def process_file(self, path: str) -> None:
    async with aiofiles.open(path) as f:
        async with self._db.transaction() as tx:
            data = await f.read()
            await tx.execute("INSERT INTO ...", data)
            # Transaction commits on clean exit, rolls back on exception.

25. Context-Aware Retries and Cancellation Link to heading

Retry loops have to check for cancellation between attempts. A retry that ignores its context will go on hammering a failing upstream long after the caller has given up.

func RetryWithContext(ctx context.Context, maxAttempts int, fn func() error) error {
    var lastErr error
    for attempt := range maxAttempts {
        if err := ctx.Err(); err != nil {
            return fmt.Errorf("retry cancelled after %d attempts: %w", attempt, err)
        }

        lastErr = fn()
        if lastErr == nil {
            return nil
        }

        if !IsRetryable(lastErr) {
            return lastErr // terminal error — do not retry
        }

        backoff := time.Duration(1<<attempt) * 100 * time.Millisecond
        select {
        case <-time.After(backoff):
        case <-ctx.Done():
            return ctx.Err()
        }
    }
    return fmt.Errorf("exhausted %d attempts: %w", maxAttempts, lastErr)
}

Part V: Concurrency Patterns Link to heading

26. Worker Pool Pattern Link to heading

Bound the number of concurrent operations. Creating goroutines without limit leads to exhausted memory and thundering herds, and a pool with a fixed number of goroutines pulling from a channel is the standard remedy.

func WorkerPool[T any](ctx context.Context, workers int, jobs <-chan T, process func(T) error) error {
    g, ctx := errgroup.WithContext(ctx)
    for range workers {
        g.Go(func() error {
            for {
                select {
                case job, ok := <-jobs:
                    if !ok {
                        return nil
                    }
                    if err := process(job); err != nil {
                        slog.Error("job failed", "error", err)
                    }
                case <-ctx.Done():
                    return ctx.Err()
                }
            }
        })
    }
    return g.Wait()
}
# Python: asyncio.TaskGroup with a semaphore for bounded concurrency.
async def worker_pool(
    items: list[T],
    process: Callable[[T], Awaitable[None]],
    max_workers: int = 10,
) -> None:
    sem = asyncio.Semaphore(max_workers)

    async def bounded(item: T) -> None:
        async with sem:
            await process(item)

    async with asyncio.TaskGroup() as tg:
        for item in items:
            tg.create_task(bounded(item))

27. Channel-Based Coordination Link to heading

Channels are Go’s primitive for communication between goroutines. Prefer them to shared memory and mutexes when goroutines need to exchange data or signal that they’re done.

// done channel signals all workers to stop.
func produce(ctx context.Context, out chan<- Event) {
    defer close(out) // closing signals consumers that no more data is coming
    for {
        event, err := pollSource(ctx)
        if err != nil {
            return
        }
        select {
        case out <- event:
        case <-ctx.Done():
            return
        }
    }
}

func consume(ctx context.Context, in <-chan Event) {
    for {
        select {
        case event, ok := <-in:
            if !ok {
                return // channel closed — producer is done
            }
            handleEvent(event)
        case <-ctx.Done():
            return
        }
    }
}

28. Fan-Out / Fan-In Concurrency Pipelines Link to heading

Fan-out distributes work across several goroutines, and fan-in merges the results back into a single channel. Together they let you process independent tasks in parallel with one place to collect the results.

func FanOutFanIn[In, Out any](
    ctx context.Context,
    inputs []In,
    workers int,
    process func(context.Context, In) (Out, error),
) ([]Out, error) {
    in := make(chan In)
    out := make(chan Out, len(inputs))

    // Fan-out: distribute inputs to workers.
    g, ctx := errgroup.WithContext(ctx)
    for range workers {
        g.Go(func() error {
            for input := range in {
                result, err := process(ctx, input)
                if err != nil {
                    return err
                }
                out <- result
            }
            return nil
        })
    }

    // Feed inputs.
    go func() {
        defer close(in)
        for _, input := range inputs {
            select {
            case in <- input:
            case <-ctx.Done():
                return
            }
        }
    }()

    // Fan-in: collect results after all workers finish.
    err := g.Wait()
    close(out)
    if err != nil {
        return nil, err
    }

    var results []Out
    for result := range out {
        results = append(results, result)
    }
    return results, nil
}

29. Rate-Limited Work Queues Link to heading

When you call external services, rate limiting keeps you from overwhelming the upstream. A token bucket or a ticker controls throughput at the source.

func RateLimitedProcessor(ctx context.Context, jobs <-chan Job, rps int) error {
    limiter := rate.NewLimiter(rate.Limit(rps), 1) // golang.org/x/time/rate

    for {
        select {
        case job, ok := <-jobs:
            if !ok {
                return nil
            }
            // Wait blocks until a token is available or ctx is cancelled.
            if err := limiter.Wait(ctx); err != nil {
                return err
            }
            if err := process(job); err != nil {
                slog.Error("job failed", "error", err)
            }
        case <-ctx.Done():
            return ctx.Err()
        }
    }
}

30. Bounded Concurrency with Semaphores Link to heading

A semaphore limits how many goroutines can be inside a critical section at once. Unlike a worker pool, which creates a fixed set of goroutines, a semaphore lets you create goroutines freely while capping concurrent access.

func ProcessAll(ctx context.Context, items []Item) error {
    sem := make(chan struct{}, 10) // at most 10 concurrent operations
    g, ctx := errgroup.WithContext(ctx)

    for _, item := range items {
        g.Go(func() error {
            sem <- struct{}{}        // acquire
            defer func() { <-sem }() // release

            return processItem(ctx, item)
        })
    }

    return g.Wait()
}

31. Ownership-Based Concurrency Design Link to heading

Each piece of mutable state should be owned by exactly one goroutine, and other goroutines interact with it by sending messages to the owner. That eliminates data races without a single mutex.

// The stats goroutine owns the counters map. No mutex needed.
type StatsCollector struct {
    updates chan stat
    queries chan statsQuery
}

type stat struct {
    key   string
    delta int64
}

type statsQuery struct {
    reply chan map[string]int64
}

func (sc *StatsCollector) Run(ctx context.Context) {
    counters := make(map[string]int64)
    for {
        select {
        case s := <-sc.updates:
            counters[s.key] += s.delta
        case q := <-sc.queries:
            snapshot := make(map[string]int64, len(counters))
            for k, v := range counters {
                snapshot[k] = v
            }
            q.reply <- snapshot
        case <-ctx.Done():
            return
        }
    }
}

func (sc *StatsCollector) Increment(key string) {
    sc.updates <- stat{key: key, delta: 1}
}

32. Single-Writer Concurrency Pattern Link to heading

Many readers are safe; many writers cause contention. Design your data flows so that exactly one goroutine writes to any given structure, and readers work from snapshots or immutable copies.

type ConfigWatcher struct {
    mu     sync.RWMutex
    config atomic.Pointer[Config] // single writer, multiple readers
}

// Update is called by exactly one goroutine (the config watcher).
func (cw *ConfigWatcher) Update(cfg Config) {
    cw.config.Store(&cfg)
}

// Get is called by any number of goroutines concurrently — lock-free.
func (cw *ConfigWatcher) Get() Config {
    return *cw.config.Load()
}

33. Actor-Like Goroutine Ownership Model Link to heading

Each “actor” goroutine manages its own state and communicates only through channels. This maps naturally onto Go’s concurrency model, and it does away with bugs that come from shared state.

type OrderProcessor struct {
    orders chan Order
    done   chan struct{}
}

func NewOrderProcessor() *OrderProcessor {
    op := &OrderProcessor{
        orders: make(chan Order, 100),
        done:   make(chan struct{}),
    }
    go op.run()
    return op
}

func (op *OrderProcessor) run() {
    defer close(op.done)
    pending := make(map[string]*Order)

    for order := range op.orders {
        // All state mutation happens here — single goroutine, no races.
        if existing, ok := pending[order.ID]; ok {
            existing.Merge(order)
        } else {
            pending[order.ID] = &order
        }

        if order.IsComplete() {
            delete(pending, order.ID)
            // publish completion event
        }
    }
}

func (op *OrderProcessor) Submit(o Order) { op.orders <- o }
func (op *OrderProcessor) Stop()          { close(op.orders); <-op.done }

34. Goroutine Lifecycle Ownership Rules Link to heading

Whoever starts a goroutine is responsible for making sure it stops. A leaked goroutine is a memory leak, so use errgroup, context cancellation or explicit shutdown channels to enforce ownership of each goroutine’s lifecycle.

type Server struct {
    cancel context.CancelFunc
    g      *errgroup.Group
}

func NewServer(ctx context.Context) *Server {
    ctx, cancel := context.WithCancel(ctx)
    g, ctx := errgroup.WithContext(ctx)
    s := &Server{cancel: cancel, g: g}

    // The server owns these goroutines — it starts them, it stops them.
    g.Go(func() error { return s.runHTTP(ctx) })
    g.Go(func() error { return s.runGRPC(ctx) })
    g.Go(func() error { return s.runMetrics(ctx) })

    return s
}

func (s *Server) Shutdown() error {
    s.cancel()     // signal all owned goroutines to stop
    return s.g.Wait() // wait for all to finish
}

Part VI: Lifecycle and Orchestration Link to heading

35. Reconciliation Loop Pattern Link to heading

Rather than reacting to individual events, periodically compare the desired state with the actual state and correct the difference. The pattern heals itself: if an event is missed or a handler crashes, the next cycle fixes the drift. (Kubernetes is, more or less, this one idea applied with great persistence.)

func (c *Controller) reconcileLoop(ctx context.Context) error {
    ticker := time.NewTicker(30 * time.Second)
    defer ticker.Stop()

    for {
        select {
        case <-ticker.C:
            if err := c.reconcile(ctx); err != nil {
                slog.Error("reconciliation failed", "error", err)
            }
        case <-ctx.Done():
            return ctx.Err()
        }
    }
}

func (c *Controller) reconcile(ctx context.Context) error {
    desired, err := c.store.ListDesired(ctx)
    if err != nil {
        return fmt.Errorf("list desired: %w", err)
    }
    actual, err := c.store.ListActual(ctx)
    if err != nil {
        return fmt.Errorf("list actual: %w", err)
    }

    for _, d := range desired {
        a, exists := actual[d.ID]
        if !exists {
            if err := c.create(ctx, d); err != nil {
                return err
            }
        } else if !d.Equal(a) {
            if err := c.update(ctx, d); err != nil {
                return err
            }
        }
    }
    return nil
}

36. Graceful Shutdown with Contexts and Signals Link to heading

Trap OS signals, cancel the root context, drain the work in flight, and close resources in the reverse order of their initialization. A service that exits cleanly on SIGTERM is one you can deploy without dropping requests.

func main() {
    ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
    defer stop()

    app, err := bootstrap(ctx, loadConfig())
    if err != nil {
        log.Fatalf("bootstrap: %v", err)
    }

    // Start server in a goroutine.
    srv := &http.Server{Addr: ":8080", Handler: app.handler}
    go func() {
        if err := srv.ListenAndServe(); err != http.ErrServerClosed {
            log.Fatalf("serve: %v", err)
        }
    }()

    <-ctx.Done() // block until signal received
    slog.Info("shutting down")

    // Give in-flight requests 15 seconds to complete.
    shutdownCtx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
    defer cancel()

    if err := srv.Shutdown(shutdownCtx); err != nil {
        slog.Error("shutdown error", "error", err)
    }
    app.db.Close()
    slog.Info("shutdown complete")
}

37. Manager/Controller Lifecycle Orchestration Link to heading

A manager coordinates the lifecycle of several subsystems, starting them in order, watching their health and shutting them down cleanly on exit.

type Manager struct {
    services []Service
}

type Service interface {
    Name() string
    Start(ctx context.Context) error
    Stop(ctx context.Context) error
    Healthy() bool
}

func (m *Manager) Start(ctx context.Context) error {
    for _, svc := range m.services {
        slog.Info("starting", "service", svc.Name())
        if err := svc.Start(ctx); err != nil {
            // Rollback: stop already-started services in reverse.
            m.stopReverse(ctx, m.services[:len(m.services)-1])
            return fmt.Errorf("start %s: %w", svc.Name(), err)
        }
    }
    return nil
}

func (m *Manager) Stop(ctx context.Context) {
    // Stop in reverse order of startup.
    for i := len(m.services) - 1; i >= 0; i-- {
        slog.Info("stopping", "service", m.services[i].Name())
        if err := m.services[i].Stop(ctx); err != nil {
            slog.Error("stop error", "service", m.services[i].Name(), "error", err)
        }
    }
}

38. Background Daemon / Service Manager Pattern Link to heading

Long-running background tasks (warming caches, flushing metrics, reloading config) need the same lifecycle management as the main server. Run them as managed goroutines that respect context cancellation.

type Daemon struct {
    name     string
    interval time.Duration
    fn       func(context.Context) error
}

func (d *Daemon) Run(ctx context.Context) error {
    slog.Info("daemon started", "name", d.name, "interval", d.interval)
    ticker := time.NewTicker(d.interval)
    defer ticker.Stop()

    for {
        select {
        case <-ticker.C:
            if err := d.fn(ctx); err != nil {
                slog.Error("daemon tick failed", "name", d.name, "error", err)
            }
        case <-ctx.Done():
            slog.Info("daemon stopped", "name", d.name)
            return nil
        }
    }
}

// Usage: managed alongside the HTTP server.
g, ctx := errgroup.WithContext(ctx)
g.Go(func() error { return httpServer.Serve(ctx) })
g.Go(func() error { return cacheWarmer.Run(ctx) })
g.Go(func() error { return metricsFlush.Run(ctx) })
return g.Wait()

39. Plugin Registry Pattern Link to heading

Let components register themselves with a central registry, which then provides discovery and lifecycle management without coupling the registry tightly to the plugins.

type HealthChecker interface {
    Name() string
    Check(ctx context.Context) error
}

type HealthRegistry struct {
    mu       sync.RWMutex
    checkers []HealthChecker
}

func (r *HealthRegistry) Register(c HealthChecker) {
    r.mu.Lock()
    defer r.mu.Unlock()
    r.checkers = append(r.checkers, c)
}

func (r *HealthRegistry) CheckAll(ctx context.Context) map[string]error {
    r.mu.RLock()
    defer r.mu.RUnlock()

    results := make(map[string]error, len(r.checkers))
    for _, c := range r.checkers {
        results[c.Name()] = c.Check(ctx)
    }
    return results
}

Part VII: Resilience and Reliability Link to heading

40. Exponential Backoff and Retry Link to heading

Retry transient failures with exponential backoff and jitter. Without jitter, retrying clients fall into step with each other and create thundering-herd spikes; without backoff, aggressive retries overwhelm an upstream that is already struggling.

func Retry(ctx context.Context, maxAttempts int, fn func() error) error {
    for attempt := range maxAttempts {
        err := fn()
        if err == nil {
            return nil
        }
        if !IsRetryable(err) {
            return err
        }
        if attempt == maxAttempts-1 {
            return fmt.Errorf("exhausted %d attempts: %w", maxAttempts, err)
        }

        // Exponential backoff with jitter.
        base := time.Duration(1<<attempt) * 100 * time.Millisecond
        jitter := time.Duration(rand.Int64N(int64(base / 2)))
        backoff := base + jitter

        select {
        case <-time.After(backoff):
        case <-ctx.Done():
            return ctx.Err()
        }
    }
    return nil
}
import random

async def retry(
    fn: Callable[[], Awaitable[T]],
    max_attempts: int = 3,
    base_delay: float = 0.1,
) -> T:
    for attempt in range(max_attempts):
        try:
            return await fn()
        except RetryableError:
            if attempt == max_attempts - 1:
                raise
            delay = base_delay * (2 ** attempt) + random.uniform(0, base_delay)
            await asyncio.sleep(delay)

41. Circuit Breaker Pattern Link to heading

A circuit breaker stops calling a failing upstream once consecutive failures cross a threshold. It moves between three states, closed (normal), open (failing fast) and half-open (testing whether the upstream has recovered), and in doing so keeps a degraded dependency from consuming your resources and spreading its failure.

type CircuitBreaker struct {
    mu          sync.Mutex
    failures    int
    threshold   int
    state       State
    lastFailure time.Time
    cooldown    time.Duration
}

func (cb *CircuitBreaker) Execute(fn func() error) error {
    cb.mu.Lock()
    if cb.state == Open {
        if time.Since(cb.lastFailure) < cb.cooldown {
            cb.mu.Unlock()
            return ErrCircuitOpen
        }
        cb.state = HalfOpen // allow one probe request
    }
    cb.mu.Unlock()

    err := fn()

    cb.mu.Lock()
    defer cb.mu.Unlock()
    if err != nil {
        cb.failures++
        cb.lastFailure = time.Now()
        if cb.failures >= cb.threshold {
            cb.state = Open
        }
        return err
    }

    cb.failures = 0
    cb.state = Closed
    return nil
}

42. Bulkhead Isolation Pattern Link to heading

Isolate different kinds of work so that a failure in one can’t exhaust resources the others depend on. A bulkhead per upstream service keeps a slow dependency from starving unrelated request paths.

type Bulkhead struct {
    sem chan struct{}
}

func NewBulkhead(maxConcurrent int) *Bulkhead {
    return &Bulkhead{sem: make(chan struct{}, maxConcurrent)}
}

func (b *Bulkhead) Execute(ctx context.Context, fn func() error) error {
    select {
    case b.sem <- struct{}{}:
        defer func() { <-b.sem }()
        return fn()
    case <-ctx.Done():
        return ctx.Err()
    default:
        return ErrBulkheadFull // fail fast if all slots are taken
    }
}

// Usage: separate bulkheads per upstream.
var (
    paymentBulkhead  = NewBulkhead(20)
    inventoryBulkhead = NewBulkhead(50)
)

43. Idempotent Handlers and Reconciliation Link to heading

Every operation that can be retried must be idempotent, meaning that executing it twice has the same effect as executing it once. Use idempotency keys to deduplicate requests, and store the result of the first execution.

func (h *Handler) CreateOrder(ctx context.Context, req CreateOrderRequest) (Order, error) {
    // Check if this idempotency key has already been processed.
    existing, err := h.idempotencyStore.Get(ctx, req.IdempotencyKey)
    if err == nil {
        return existing, nil // already processed — return cached result
    }

    order, err := h.service.CreateOrder(ctx, req)
    if err != nil {
        return Order{}, err
    }

    // Store result for deduplication.
    _ = h.idempotencyStore.Set(ctx, req.IdempotencyKey, order, 24*time.Hour)
    return order, nil
}

44. Retry Transport Wrappers Link to heading

Wrap http.RoundTripper to add retries transparently, and every HTTP client in the application benefits without any call site changing.

type RetryTransport struct {
    inner       http.RoundTripper
    maxAttempts int
}

func (t *RetryTransport) RoundTrip(req *http.Request) (*http.Response, error) {
    var lastErr error
    for attempt := range t.maxAttempts {
        resp, err := t.inner.RoundTrip(req)
        if err == nil && resp.StatusCode < 500 {
            return resp, nil
        }
        if err != nil {
            lastErr = err
        } else {
            lastErr = fmt.Errorf("HTTP %d", resp.StatusCode)
            resp.Body.Close()
        }

        backoff := time.Duration(1<<attempt) * 100 * time.Millisecond
        select {
        case <-time.After(backoff):
        case <-req.Context().Done():
            return nil, req.Context().Err()
        }
    }
    return nil, lastErr
}

// Applied once — all requests through this client get retries.
client := &http.Client{
    Transport: &RetryTransport{inner: http.DefaultTransport, maxAttempts: 3},
}

45. Shared Transport / Client Reuse Link to heading

Create HTTP clients and database connections once and reuse them. Each http.Client maintains its own connection pool, so creating a new client per request wastes connections and causes a great many needless TCP handshakes.

// Bad: new client per request — leaks connections.
func fetchBad(url string) (*http.Response, error) {
    client := &http.Client{} // new pool every call
    return client.Get(url)
}

// Good: shared client configured once.
var httpClient = &http.Client{
    Timeout: 30 * time.Second,
    Transport: &http.Transport{
        MaxIdleConns:        100,
        MaxIdleConnsPerHost: 10,
        IdleConnTimeout:     90 * time.Second,
    },
}

func fetchGood(ctx context.Context, url string) (*http.Response, error) {
    req, _ := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
    return httpClient.Do(req)
}

46. Outbox Pattern for Reliable Event Publishing Link to heading

Writing a database record and publishing an event are two separate operations that can fail independently. The outbox pattern writes the event to a database table in the same transaction as the business data, and a separate process reads and publishes it. That guarantees at-least-once delivery without any distributed transaction.

func (s *Service) CreateOrder(ctx context.Context, order Order) error {
    return s.db.WithTx(ctx, func(tx *sql.Tx) error {
        // Business operation and event written in the same transaction.
        if err := insertOrder(tx, order); err != nil {
            return fmt.Errorf("insert order: %w", err)
        }

        event := OutboxEvent{
            AggregateID: order.ID,
            EventType:   "order.created",
            Payload:     mustMarshal(order),
            CreatedAt:   time.Now(),
        }
        if err := insertOutboxEvent(tx, event); err != nil {
            return fmt.Errorf("insert outbox event: %w", err)
        }
        return nil
    })
}

// Separate poller reads outbox and publishes to Kafka.
func (p *OutboxPoller) poll(ctx context.Context) error {
    events, err := p.store.GetUnpublished(ctx, 100)
    if err != nil {
        return err
    }
    for _, event := range events {
        if err := p.publisher.Publish(ctx, event); err != nil {
            return err
        }
        _ = p.store.MarkPublished(ctx, event.ID)
    }
    return nil
}

47. Lease / Leader-Election Coordination Link to heading

In a deployment with several replicas, certain tasks (cron jobs, reconciliation loops, outbox polling) should run on exactly one of them. Leader election through a distributed lock (a database row lock, Redis, etcd) ensures that they do.

func (w *Worker) RunWithLease(ctx context.Context) error {
    ticker := time.NewTicker(10 * time.Second)
    defer ticker.Stop()

    for {
        select {
        case <-ticker.C:
            // Attempt to acquire lease. Only the leader succeeds.
            acquired, err := w.lock.TryAcquire(ctx, "reconciler", 30*time.Second)
            if err != nil {
                slog.Error("lease error", "error", err)
                continue
            }
            if !acquired {
                continue // another replica holds the lease
            }

            if err := w.reconcile(ctx); err != nil {
                slog.Error("reconcile failed", "error", err)
            }
            _ = w.lock.Release(ctx, "reconciler")

        case <-ctx.Done():
            return nil
        }
    }
}

Part VIII: Data Access and Transactions Link to heading

48. Repository Pattern (Lightweight) Link to heading

A repository gives storage an interface shaped by the domain. It accepts and returns domain entities, not database rows or ORM models, and its implementation translates between the domain’s representation and the storage’s.

type UserRepository interface {
    GetByID(ctx context.Context, id string) (User, error)
    Save(ctx context.Context, user User) error
    Delete(ctx context.Context, id string) error
}

type postgresUserRepo struct {
    db *sql.DB
}

func (r *postgresUserRepo) GetByID(ctx context.Context, id string) (User, error) {
    var row userRow
    err := r.db.QueryRowContext(ctx,
        "SELECT id, email, name, created_at FROM users WHERE id = $1", id,
    ).Scan(&row.ID, &row.Email, &row.Name, &row.CreatedAt)
    if errors.Is(err, sql.ErrNoRows) {
        return User{}, ErrNotFound
    }
    if err != nil {
        return User{}, fmt.Errorf("query user: %w", err)
    }
    return row.toDomain(), nil
}

49. API Boundary DTO Separation Link to heading

Never expose domain entities directly in API responses; use separate DTOs (data transfer objects) for serialization instead. That decouples your internal model from your external contract, so you can refactor the domain without breaking API clients.

// Domain entity — internal.
type User struct {
    ID           string
    Email        string
    PasswordHash string // never expose this
    CreatedAt    time.Time
}

// API DTO — external contract.
type UserResponse struct {
    ID        string `json:"id"`
    Email     string `json:"email"`
    CreatedAt string `json:"created_at"`
}

func toUserResponse(u User) UserResponse {
    return UserResponse{
        ID:        u.ID,
        Email:     u.Email,
        CreatedAt: u.CreatedAt.Format(time.RFC3339),
    }
}
# Python: Pydantic models as DTOs, separate from domain entities.
from pydantic import BaseModel

# Domain entity.
@dataclass
class User:
    id: str
    email: str
    password_hash: str
    created_at: datetime

# API DTO — controls what the client sees.
class UserResponse(BaseModel):
    id: str
    email: str
    created_at: datetime

    @classmethod
    def from_domain(cls, user: User) -> "UserResponse":
        return cls(id=user.id, email=user.email, created_at=user.created_at)

50. Explicit Serialization Models Link to heading

Define separate structs for JSON, YAML and Protobuf. A single struct with several sets of tags becomes unreadable and couples concerns that have nothing to do with each other, whereas separate models make each format’s contract explicit.

// Domain model — no serialization tags.
type Order struct {
    ID        string
    Items     []Item
    Total     Money
    Status    OrderStatus
    CreatedAt time.Time
}

// JSON API model.
type OrderJSON struct {
    ID        string     `json:"id"`
    Items     []ItemJSON `json:"items"`
    Total     string     `json:"total"` // Money serialized as string
    Status    string     `json:"status"`
    CreatedAt string     `json:"created_at"`
}

// Database model.
type OrderRow struct {
    ID        string    `db:"id"`
    Items     []byte    `db:"items"`     // JSONB
    TotalCents int64    `db:"total_cents"` // stored as cents
    Status    string    `db:"status"`
    CreatedAt time.Time `db:"created_at"`
}

51. Transaction Boundary Encapsulation Link to heading

Wrap transaction management in a helper that handles begin, commit and rollback. Business logic shouldn’t contain transaction plumbing; it should receive a transaction scoped to a function and get on with the domain operations.

func (db *DB) WithTx(ctx context.Context, fn func(tx *sql.Tx) error) error {
    tx, err := db.BeginTx(ctx, nil)
    if err != nil {
        return fmt.Errorf("begin tx: %w", err)
    }
    defer tx.Rollback() // no-op after commit

    if err := fn(tx); err != nil {
        return err
    }
    return tx.Commit()
}

// Usage: business logic stays clean.
err := db.WithTx(ctx, func(tx *sql.Tx) error {
    if err := createOrder(tx, order); err != nil {
        return err
    }
    return updateInventory(tx, order.Items)
})

52. Optimistic Concurrency / Version Checks Link to heading

Prevent lost updates by including a version or timestamp in update queries. If the version has changed since the read, the update fails, and the caller has to re-read and try again.

func (r *repo) Update(ctx context.Context, user User) error {
    result, err := r.db.ExecContext(ctx,
        `UPDATE users SET email = $1, name = $2, version = version + 1
         WHERE id = $3 AND version = $4`,
        user.Email, user.Name, user.ID, user.Version,
    )
    if err != nil {
        return fmt.Errorf("update user: %w", err)
    }

    rows, _ := result.RowsAffected()
    if rows == 0 {
        return ErrConcurrentModification // another writer got there first
    }
    return nil
}

53. Time-Ordered IDs (ULID / KSUID) Link to heading

Random UUIDs can’t be sorted meaningfully. Time-ordered IDs such as ULIDs sort lexicographically by creation time, which can spare you a secondary index on created_at and makes B-tree inserts cheaper.

import "github.com/oklog/ulid/v2"

func NewID() string {
    return ulid.Make().String()
    // Example: "01HXYZ1234567890ABCDEFGHIJ"
    // Sortable, unique, timestamp-embedded.
}

// IDs sort by creation time without an explicit timestamp column.
// SELECT * FROM events ORDER BY id  -- gives chronological order
import ulid

def new_id() -> str:
    return str(ulid.new())
    # Lexicographically sortable, globally unique, embeds timestamp.

54. Soft-Delete and Tombstone Patterns Link to heading

Delete data logically rather than physically. A deleted_at timestamp marks a row as deleted while keeping it for audit trails, undo and referential integrity, and queries filter on deleted_at IS NULL by default.

func (r *repo) SoftDelete(ctx context.Context, id string) error {
    _, err := r.db.ExecContext(ctx,
        "UPDATE users SET deleted_at = NOW() WHERE id = $1 AND deleted_at IS NULL",
        id,
    )
    return err
}

func (r *repo) GetByID(ctx context.Context, id string) (User, error) {
    // Default queries exclude soft-deleted rows.
    return r.queryOne(ctx,
        "SELECT * FROM users WHERE id = $1 AND deleted_at IS NULL",
        id,
    )
}

Part IX: API, Transport, and CLI Link to heading

55. Middleware Chaining Link to heading

Middleware adds cross-cutting behavior (authentication, logging, CORS, rate limiting) to HTTP handlers. Chain it by wrapping handlers, so that each layer adds exactly one concern.

type Middleware func(http.Handler) http.Handler

func Chain(handler http.Handler, middlewares ...Middleware) http.Handler {
    // Apply in reverse so the first middleware in the list is outermost.
    for i := len(middlewares) - 1; i >= 0; i-- {
        handler = middlewares[i](handler)
    }
    return handler
}

// Usage: reads top-to-bottom as the request flow.
handler := Chain(
    appHandler,
    RecoveryMiddleware,
    LoggingMiddleware(logger),
    AuthMiddleware(verifier),
    RateLimitMiddleware(limiter),
)
# FastAPI/Starlette: middleware applied in order.
app = FastAPI()
app.add_middleware(CORSMiddleware, allow_origins=["*"])
app.add_middleware(RequestLoggingMiddleware)
app.add_middleware(AuthMiddleware, verifier=token_verifier)

56. HTTP Transport Abstraction Layers Link to heading

Wrap external HTTP APIs behind an interface. The details of the transport (headers, authentication, building URLs) live in the client implementation and never in the business logic.

type PaymentGateway interface {
    Charge(ctx context.Context, customerID string, amount Money) (Receipt, error)
    Refund(ctx context.Context, receiptID string) error
}

type stripeGateway struct {
    client  *http.Client
    baseURL string
    apiKey  string
}

func (g *stripeGateway) Charge(ctx context.Context, customerID string, amount Money) (Receipt, error) {
    body := buildChargeRequest(customerID, amount)
    req, _ := http.NewRequestWithContext(ctx, http.MethodPost,
        g.baseURL+"/charges", body)
    req.Header.Set("Authorization", "Bearer "+g.apiKey)
    req.Header.Set("Content-Type", "application/json")

    resp, err := g.client.Do(req)
    if err != nil {
        return Receipt{}, fmt.Errorf("stripe charge: %w", err)
    }
    defer resp.Body.Close()

    return parseReceipt(resp)
}

57. Adapter Pattern Around External Services Link to heading

Wrap third-party SDKs in an adapter that exposes only the operations your application needs. That confines the vendor to a single file and makes the dependency easy to mock in tests.

// Application interface — defined by your needs, not the vendor.
type ObjectStore interface {
    Put(ctx context.Context, key string, data []byte) error
    Get(ctx context.Context, key string) ([]byte, error)
    Delete(ctx context.Context, key string) error
}

// Adapter wraps the AWS SDK.
type s3Store struct {
    client *s3.Client
    bucket string
}

func (s *s3Store) Put(ctx context.Context, key string, data []byte) error {
    _, err := s.client.PutObject(ctx, &s3.PutObjectInput{
        Bucket: &s.bucket,
        Key:    &key,
        Body:   bytes.NewReader(data),
    })
    return err
}

58. Strategy Pattern via Interfaces / Functions Link to heading

Select behavior at runtime by passing in different implementations of an interface, or different function values. Polymorphism of this sort replaces a good many switch statements.

// Strategy as interface.
type PricingStrategy interface {
    CalculatePrice(base Money, quantity int) Money
}

type flatPricing struct{}
func (f flatPricing) CalculatePrice(base Money, qty int) Money {
    return base.Multiply(qty)
}

type tieredPricing struct{ tiers []Tier }
func (t tieredPricing) CalculatePrice(base Money, qty int) Money {
    // Apply volume discounts based on tiers.
    for _, tier := range t.tiers {
        if qty >= tier.MinQty {
            return base.Multiply(qty).ApplyDiscount(tier.Discount)
        }
    }
    return base.Multiply(qty)
}

// Strategy as function — simpler when the interface is a single method.
type PriceFunc func(base Money, qty int) Money

func WithBulkDiscount(threshold int, discount float64) PriceFunc {
    return func(base Money, qty int) Money {
        if qty >= threshold {
            return base.Multiply(qty).ApplyDiscount(discount)
        }
        return base.Multiply(qty)
    }
}

59. Command Pattern for CLI Applications Link to heading

Structure CLI commands as self-contained objects that encapsulate the operation, its arguments and how it runs. This keeps argument parsing apart from business logic.

type Command struct {
    Name        string
    Description string
    Flags       *flag.FlagSet
    Run         func(ctx context.Context, args []string) error
}

func migrateCommand(db *sql.DB) *Command {
    fs := flag.NewFlagSet("migrate", flag.ExitOnError)
    direction := fs.String("direction", "up", "migration direction")

    return &Command{
        Name:        "migrate",
        Description: "Run database migrations",
        Flags:       fs,
        Run: func(ctx context.Context, args []string) error {
            fs.Parse(args)
            return runMigrations(ctx, db, *direction)
        },
    }
}

60. Cobra-Style CLI Composition Link to heading

For a complex CLI, compose the commands into a tree, with each subcommand self-contained: its own flags, its own help text and its own function to run.

func main() {
    root := &cobra.Command{Use: "app", Short: "Application management"}

    root.AddCommand(
        serveCmd(),
        migrateCmd(),
        seedCmd(),
        workerCmd(),
    )

    if err := root.Execute(); err != nil {
        os.Exit(1)
    }
}

func serveCmd() *cobra.Command {
    cmd := &cobra.Command{
        Use:   "serve",
        Short: "Start the HTTP server",
        RunE: func(cmd *cobra.Command, args []string) error {
            port, _ := cmd.Flags().GetInt("port")
            return startServer(cmd.Context(), port)
        },
    }
    cmd.Flags().Int("port", 8080, "server port")
    return cmd
}

61. Pipeline Stage Composition Link to heading

Model multi-step processing as a pipeline of composable stages, each transforming its input into its output and each testable on its own.

type Stage[In, Out any] func(ctx context.Context, in In) (Out, error)

func Chain2[A, B, C any](first Stage[A, B], second Stage[B, C]) Stage[A, C] {
    return func(ctx context.Context, in A) (C, error) {
        var zero C
        mid, err := first(ctx, in)
        if err != nil {
            return zero, err
        }
        return second(ctx, mid)
    }
}

// Usage: compose stages into a pipeline.
pipeline := Chain2(
    Chain2(validateStage, enrichStage),
    publishStage,
)
result, err := pipeline(ctx, rawInput)

Part X: Events, Streams, and State Machines Link to heading

62. Informer / Watch / Event-Subscription Pattern Link to heading

Instead of polling, subscribe to change events from the data source. An informer keeps the current state cached locally and receives incremental updates, which reduces load on the source and allows processing to be reactive.

type EventHandler func(eventType string, obj any)

type Informer struct {
    client   Client
    cache    sync.Map
    handlers []EventHandler
}

func (inf *Informer) Run(ctx context.Context) error {
    // Initial list — populate the cache.
    items, err := inf.client.List(ctx)
    if err != nil {
        return err
    }
    for _, item := range items {
        inf.cache.Store(item.ID, item)
    }

    // Watch for changes — receive incremental updates.
    watcher, err := inf.client.Watch(ctx)
    if err != nil {
        return err
    }
    for {
        select {
        case event := <-watcher.Events():
            inf.cache.Store(event.Object.ID, event.Object)
            for _, h := range inf.handlers {
                h(event.Type, event.Object)
            }
        case <-ctx.Done():
            return nil
        }
    }
}

63. CQRS-Lite Read/Write Separation Link to heading

Separate the read and write paths without building full CQRS infrastructure. Writes go through the domain model, with validation; reads use optimized queries that bypass the domain layer entirely.

// Write side: full domain validation.
type OrderCommandService struct {
    repo OrderRepository
}

func (s *OrderCommandService) PlaceOrder(ctx context.Context, cmd PlaceOrderCmd) error {
    order, err := NewOrder(cmd) // domain validation
    if err != nil {
        return err
    }
    return s.repo.Save(ctx, order)
}

// Read side: optimized query, no domain objects.
type OrderQueryService struct {
    db *sql.DB
}

func (s *OrderQueryService) ListRecent(ctx context.Context, limit int) ([]OrderSummary, error) {
    // Direct SQL — skips domain model overhead for read-only views.
    rows, err := s.db.QueryContext(ctx,
        `SELECT id, customer_name, total, status, created_at
         FROM orders ORDER BY created_at DESC LIMIT $1`, limit)
    if err != nil {
        return nil, err
    }
    return scanOrderSummaries(rows)
}

64. Queue + Deduplication Processing Link to heading

Process messages from a queue with at-least-once delivery and deduplicate them, which gives you exactly-once semantics at the level of the application.

func (w *Worker) processMessages(ctx context.Context) error {
    for {
        msg, err := w.queue.Receive(ctx)
        if err != nil {
            return err
        }

        // Deduplication: skip already-processed messages.
        processed, err := w.dedup.HasProcessed(ctx, msg.ID)
        if err != nil {
            slog.Error("dedup check failed", "error", err)
            msg.Nack()
            continue
        }
        if processed {
            msg.Ack() // already handled — skip silently
            continue
        }

        if err := w.handle(ctx, msg); err != nil {
            slog.Error("processing failed", "error", err, "msg_id", msg.ID)
            msg.Nack()
            continue
        }

        _ = w.dedup.MarkProcessed(ctx, msg.ID, 24*time.Hour)
        msg.Ack()
    }
}

65. Backpressure-Aware Processing Link to heading

When a producer is faster than its consumer, the system has to apply backpressure, slowing the producer or buffering the work, rather than dropping data or running out of memory. Bounded channels and semaphores enforce backpressure naturally.

func (p *Pipeline) Run(ctx context.Context) error {
    // Bounded buffer: producer blocks when buffer is full.
    buffer := make(chan Event, 1000) // backpressure at 1000 pending events

    g, ctx := errgroup.WithContext(ctx)

    // Producer: blocks on buffer <- when full.
    g.Go(func() error {
        defer close(buffer)
        return p.source.Stream(ctx, buffer)
    })

    // Consumer pool: drains buffer.
    for range p.workers {
        g.Go(func() error {
            for event := range buffer {
                if err := p.process(ctx, event); err != nil {
                    slog.Error("event failed", "error", err)
                }
            }
            return nil
        })
    }

    return g.Wait()
}

66. Streaming API Patterns Link to heading

For large result sets, stream the data to the client rather than buffering the whole response in memory. In Go that means writing to http.ResponseWriter incrementally; in Python, it means async generators.

func (h *Handler) StreamEvents(w http.ResponseWriter, r *http.Request) {
    flusher, ok := w.(http.Flusher)
    if !ok {
        http.Error(w, "streaming not supported", http.StatusInternalServerError)
        return
    }
    w.Header().Set("Content-Type", "text/event-stream")

    ctx := r.Context()
    events := h.store.Subscribe(ctx)

    for {
        select {
        case event := <-events:
            fmt.Fprintf(w, "data: %s\n\n", event.JSON())
            flusher.Flush()
        case <-ctx.Done():
            return
        }
    }
}
# Python: async generator for streaming responses.
async def stream_events(request: Request) -> StreamingResponse:
    async def event_generator():
        async for event in event_store.subscribe():
            yield f"data: {event.json()}\n\n"

    return StreamingResponse(event_generator(), media_type="text/event-stream")

67. Iterator Abstraction Patterns Link to heading

Abstract over paginated data sources with an iterator that fetches pages lazily, so that the consumer sees a simple Next() or __anext__() regardless of how the pagination works underneath.

type Iterator[T any] struct {
    fetch  func(cursor string) ([]T, string, error) // returns items, next cursor, error
    buffer []T
    cursor string
    done   bool
}

func (it *Iterator[T]) Next() (T, error) {
    var zero T
    if len(it.buffer) == 0 {
        if it.done {
            return zero, io.EOF
        }
        items, nextCursor, err := it.fetch(it.cursor)
        if err != nil {
            return zero, err
        }
        it.buffer = items
        it.cursor = nextCursor
        if nextCursor == "" {
            it.done = true
        }
    }
    item := it.buffer[0]
    it.buffer = it.buffer[1:]
    return item, nil
}

68. State Machine Modeling Link to heading

Model entities with complicated lifecycles as explicit state machines. The valid transitions are declared, and the machine rejects any others, which does away with if status == "pending" checks scattered throughout the codebase.

type State string

const (
    StatePending    State = "pending"
    StateProcessing State = "processing"
    StateCompleted  State = "completed"
    StateFailed     State = "failed"
)

var validTransitions = map[State][]State{
    StatePending:    {StateProcessing, StateFailed},
    StateProcessing: {StateCompleted, StateFailed},
    // Completed and Failed are terminal — no outgoing transitions.
}

func (o *Order) TransitionTo(next State) error {
    allowed, ok := validTransitions[o.Status]
    if !ok {
        return fmt.Errorf("no transitions from %s", o.Status)
    }
    for _, s := range allowed {
        if s == next {
            o.Status = next
            return nil
        }
    }
    return fmt.Errorf("invalid transition: %s → %s", o.Status, next)
}

Part XI: Caching, Pooling, and Performance Link to heading

69. Generic Typed Cache / Store Patterns Link to heading

Type-safe caching does away with the interface{} casts that litter untyped cache implementations; with generics, the cache enforces its types at compile time.

type Cache[K comparable, V any] struct {
    mu    sync.RWMutex
    items map[K]cacheEntry[V]
    ttl   time.Duration
}

type cacheEntry[V any] struct {
    value     V
    expiresAt time.Time
}

func (c *Cache[K, V]) Get(key K) (V, bool) {
    c.mu.RLock()
    defer c.mu.RUnlock()

    entry, ok := c.items[key]
    if !ok || time.Now().After(entry.expiresAt) {
        var zero V
        return zero, false
    }
    return entry.value, true
}

func (c *Cache[K, V]) Set(key K, value V) {
    c.mu.Lock()
    defer c.mu.Unlock()
    c.items[key] = cacheEntry[V]{value: value, expiresAt: time.Now().Add(c.ttl)}
}

70. Object Pooling with sync.Pool Link to heading

Reuse objects that are expensive to allocate (byte buffers, JSON encoders, protobuf marshalers) instead of creating and garbage-collecting them on every request. sync.Pool provides reuse that is safe across goroutines.

var bufferPool = sync.Pool{
    New: func() any {
        return new(bytes.Buffer)
    },
}

func Marshal(v any) ([]byte, error) {
    buf := bufferPool.Get().(*bytes.Buffer)
    defer func() {
        buf.Reset()
        bufferPool.Put(buf)
    }()

    if err := json.NewEncoder(buf).Encode(v); err != nil {
        return nil, err
    }
    return bytes.Clone(buf.Bytes()), nil
}

71. Copy-on-Write Configuration / State Link to heading

Let concurrent readers access the current state without locking. When a writer updates the state, it makes a new copy and swaps the pointer atomically, so readers always see a consistent snapshot.

type RouteTable struct {
    routes atomic.Pointer[map[string]Handler]
}

func (rt *RouteTable) Lookup(path string) (Handler, bool) {
    routes := *rt.routes.Load()
    h, ok := routes[path]
    return h, ok // lock-free read
}

func (rt *RouteTable) Update(path string, handler Handler) {
    for {
        old := rt.routes.Load()
        newRoutes := make(map[string]Handler, len(*old)+1)
        for k, v := range *old {
            newRoutes[k] = v
        }
        newRoutes[path] = handler
        if rt.routes.CompareAndSwap(old, &newRoutes) {
            return
        }
    }
}

72. Read-Through / Write-Through Caching Link to heading

A read-through cache loads missing entries from the source of truth on its own, and a write-through cache updates both the cache and the source in a single operation. Either way, the consumer doesn’t have to manage the cache’s coherence.

type ReadThroughCache[K comparable, V any] struct {
    cache  Cache[K, V]
    loader func(ctx context.Context, key K) (V, error)
}

func (c *ReadThroughCache[K, V]) Get(ctx context.Context, key K) (V, error) {
    if val, ok := c.cache.Get(key); ok {
        return val, nil // cache hit
    }

    // Cache miss — load from source.
    val, err := c.loader(ctx, key)
    if err != nil {
        var zero V
        return zero, err
    }

    c.cache.Set(key, val)
    return val, nil
}

73. Memory Reuse via Buffer Pools Link to heading

Allocate buffers from a pool sized for the common case, which reduces pressure on the garbage collector along hot paths where allocation throughput matters.

// Pool of 4KB buffers — sized for typical HTTP request bodies.
var smallBufPool = sync.Pool{
    New: func() any {
        buf := make([]byte, 0, 4096)
        return &buf
    },
}

func ReadBody(r *http.Request) ([]byte, error) {
    bufp := smallBufPool.Get().(*[]byte)
    defer func() {
        *bufp = (*bufp)[:0]
        smallBufPool.Put(bufp)
    }()

    buf := *bufp
    n, err := io.ReadFull(r.Body, buf[:cap(buf)])
    if err != nil && err != io.ErrUnexpectedEOF {
        return nil, err
    }
    return bytes.Clone(buf[:n]), nil
}

74. Immutable Event / Message Structs Link to heading

Events and messages should be immutable value types, unchangeable once created. That removes an entire class of concurrency bugs, since a struct that can’t change can be shared across goroutines without any synchronization.

// OrderPlaced is a value type — all fields are exported,
// but there are no setter methods and no pointer receivers.
type OrderPlaced struct {
    OrderID    string
    CustomerID string
    Total      Money
    Items      []OrderItem // slice of values, not pointers
    OccurredAt time.Time
}

// Constructor validates and freezes.
func NewOrderPlaced(order Order) OrderPlaced {
    items := make([]OrderItem, len(order.Items))
    copy(items, order.Items) // defensive copy — caller cannot mutate

    return OrderPlaced{
        OrderID:    order.ID,
        CustomerID: order.CustomerID,
        Total:      order.Total,
        Items:      items,
        OccurredAt: time.Now(),
    }
}

Part XII: Observability and Operations Link to heading

75. Structured Logging (Key/Value Logging) Link to heading

Log messages are for people, and log fields are for machines. Structured logging emits key-value pairs that can be searched, filtered and aggregated, unlike free-text messages that need a regular expression to pick apart.

// Go: slog (standard library since 1.21).
slog.Info("order processed",
    "order_id", order.ID,
    "customer_id", order.CustomerID,
    "total", order.Total,
    "duration_ms", time.Since(start).Milliseconds(),
    "items_count", len(order.Items),
)
// Output: {"time":"...","level":"INFO","msg":"order processed","order_id":"abc","duration_ms":42}
# Python: structlog for structured key/value logging.
import structlog

logger = structlog.get_logger()

logger.info("order processed",
    order_id=order.id,
    customer_id=order.customer_id,
    total=str(order.total),
    duration_ms=elapsed_ms,
    items_count=len(order.items),
)

76. Health Check and Readiness Probe Patterns Link to heading

Health checks report whether the service is alive; readiness probes report whether it can take traffic. A service that is alive but not ready (still warming caches or running migrations) should receive no traffic from the load balancer.

func (s *Server) healthHandler(w http.ResponseWriter, r *http.Request) {
    w.WriteHeader(http.StatusOK) // alive
}

func (s *Server) readyHandler(w http.ResponseWriter, r *http.Request) {
    checks := map[string]error{
        "database": s.db.PingContext(r.Context()),
        "cache":    s.cache.Ping(r.Context()),
    }

    for name, err := range checks {
        if err != nil {
            slog.Error("readiness check failed", "check", name, "error", err)
            http.Error(w, fmt.Sprintf("%s: unhealthy", name), http.StatusServiceUnavailable)
            return
        }
    }
    w.WriteHeader(http.StatusOK)
}

77. Metrics Instrumentation at Boundaries Link to heading

Instrument at the service’s boundaries: where requests come in, where external calls go out, and where messages are consumed. That captures the data that matters for SLOs, namely request rate, error rate and the distribution of latencies.

func MetricsMiddleware(next http.Handler) http.Handler {
    requestDuration := prometheus.NewHistogramVec(
        prometheus.HistogramOpts{
            Name:    "http_request_duration_seconds",
            Buckets: prometheus.DefBuckets,
        },
        []string{"method", "path", "status"},
    )

    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        start := time.Now()
        wrapped := &statusRecorder{ResponseWriter: w, status: 200}

        next.ServeHTTP(wrapped, r)

        requestDuration.WithLabelValues(
            r.Method,
            r.URL.Path,
            strconv.Itoa(wrapped.status),
        ).Observe(time.Since(start).Seconds())
    })
}

78. OpenTelemetry Tracing Propagation Link to heading

Distributed tracing depends on propagating trace context across service boundaries. Each service extracts the trace ID from incoming requests and injects it into outgoing calls, which produces a view of the whole request path from end to end.

func TracingMiddleware(next http.Handler) http.Handler {
    tracer := otel.Tracer("api")
    propagator := otel.GetTextMapPropagator()

    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        // Extract trace context from incoming request headers.
        ctx := propagator.Extract(r.Context(), propagation.HeaderCarrier(r.Header))
        ctx, span := tracer.Start(ctx, r.URL.Path)
        defer span.End()

        span.SetAttributes(
            attribute.String("http.method", r.Method),
            attribute.String("http.url", r.URL.String()),
        )

        next.ServeHTTP(w, r.WithContext(ctx))
    })
}

79. Observable, Operable Service Design Link to heading

Observability isn’t something you add on afterward; it’s a design constraint from the start. Every service should expose health endpoints, emit structured logs, publish metrics, propagate traces and surface its configuration at runtime. An operable service can answer “what is happening right now?” without anyone needing SSH access or a log-scraping expedition.

func NewOperableServer(cfg Config, app http.Handler) *http.ServeMux {
    mux := http.NewServeMux()

    // Application routes.
    mux.Handle("/", app)

    // Operations routes — always available, not behind auth.
    mux.HandleFunc("/healthz", healthHandler)
    mux.HandleFunc("/readyz", readyHandler)
    mux.Handle("/metrics", promhttp.Handler())
    mux.HandleFunc("/debug/pprof/", pprof.Index)
    mux.HandleFunc("/config", configHandler(cfg)) // expose non-secret config

    return mux
}

Part XIII: Testing Link to heading

80. Table-Driven Tests Link to heading

Define test cases as data in a slice, each entry giving the inputs, the expected outputs and a descriptive name. The test loops over the cases, so adding a new scenario never means duplicating the structure of the test.

func TestParseAmount(t *testing.T) {
    tests := []struct {
        name    string
        input   string
        want    int64
        wantErr bool
    }{
        {name: "whole dollars", input: "42", want: 4200},
        {name: "cents", input: "42.50", want: 4250},
        {name: "zero", input: "0", want: 0},
        {name: "negative", input: "-10", want: -1000},
        {name: "invalid", input: "abc", wantErr: true},
        {name: "empty", input: "", wantErr: true},
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            got, err := ParseAmount(tt.input)
            if tt.wantErr {
                require.Error(t, err)
                return
            }
            require.NoError(t, err)
            assert.Equal(t, tt.want, got)
        })
    }
}
# Python: pytest.mark.parametrize achieves the same pattern.
import pytest

@pytest.mark.parametrize("input_str, expected", [
    ("42", 4200),
    ("42.50", 4250),
    ("0", 0),
    ("-10", -1000),
])
def test_parse_amount(input_str: str, expected: int) -> None:
    assert parse_amount(input_str) == expected

@pytest.mark.parametrize("input_str", ["abc", ""])
def test_parse_amount_invalid(input_str: str) -> None:
    with pytest.raises(ValueError):
        parse_amount(input_str)

81. Subtests with t.Run() Link to heading

Group related assertions under named subtests. Each subtest runs independently, so a failure in one doesn’t skip the others, and its name appears in the output, which tells you precisely what failed.

func TestUserService(t *testing.T) {
    svc := setupTestService(t)

    t.Run("create", func(t *testing.T) {
        user, err := svc.Create(ctx, validUser)
        require.NoError(t, err)
        assert.NotEmpty(t, user.ID)
    })

    t.Run("create_duplicate_email", func(t *testing.T) {
        _, err := svc.Create(ctx, validUser) // same email
        require.Error(t, err)
        assert.True(t, errors.Is(err, ErrConflict))
    })

    t.Run("get_nonexistent", func(t *testing.T) {
        _, err := svc.GetByID(ctx, "nonexistent")
        require.Error(t, err)
        assert.True(t, errors.Is(err, ErrNotFound))
    })
}

82. Golden-File Testing Link to heading

Compare output against a known-good reference file, and when the output changes on purpose, update the reference with a flag. The pattern is especially good for testing serialization, code generation and template rendering.

var update = flag.Bool("update", false, "update golden files")

func TestRenderTemplate(t *testing.T) {
    got := renderInvoice(testOrder)

    goldenPath := filepath.Join("testdata", t.Name()+".golden")

    if *update {
        os.WriteFile(goldenPath, []byte(got), 0644)
        return
    }

    want, err := os.ReadFile(goldenPath)
    require.NoError(t, err)
    assert.Equal(t, string(want), got)
}
// Run with: go test -update to regenerate golden files.

83. Fake / Mock Implementations Over Mocking Frameworks Link to heading

Write explicit fake implementations rather than generating mocks with a framework. Fakes are readable and debuggable, and they behave like the real thing: they keep state, enforce invariants and can be shared across tests.

// Fake — a complete in-memory implementation.
type FakeUserRepo struct {
    mu    sync.Mutex
    users map[string]User
}

func NewFakeUserRepo() *FakeUserRepo {
    return &FakeUserRepo{users: make(map[string]User)}
}

func (f *FakeUserRepo) GetByID(_ context.Context, id string) (User, error) {
    f.mu.Lock()
    defer f.mu.Unlock()
    user, ok := f.users[id]
    if !ok {
        return User{}, ErrNotFound
    }
    return user, nil
}

func (f *FakeUserRepo) Save(_ context.Context, user User) error {
    f.mu.Lock()
    defer f.mu.Unlock()
    f.users[user.ID] = user
    return nil
}

// Usage in tests — no mocking framework, no generated code.
func TestUserService_GetUser(t *testing.T) {
    repo := NewFakeUserRepo()
    repo.Save(context.Background(), User{ID: "1", Email: "test@example.com"})

    svc := NewUserService(repo, noopLogger{})
    user, err := svc.GetByID(context.Background(), "1")
    require.NoError(t, err)
    assert.Equal(t, "test@example.com", user.Email)
}
# Python: same principle — hand-written fakes, not mock.patch.
class FakeUserRepo:
    def __init__(self) -> None:
        self._users: dict[str, User] = {}

    async def get_by_id(self, user_id: str) -> User | None:
        return self._users.get(user_id)

    async def save(self, user: User) -> None:
        self._users[user.id] = user

async def test_get_user() -> None:
    repo = FakeUserRepo()
    await repo.save(User(id="1", email="test@example.com"))

    svc = UserService(repo=repo)
    user = await svc.get_by_id("1")
    assert user.email == "test@example.com"

84. Test Helper Packages and Fixtures Link to heading

Extract shared test setup (database connections, fixture loading, assertion helpers) into dedicated helper packages. They cut down on duplication and let each test file concentrate on behavior rather than infrastructure.

// testutil/db.go — shared test infrastructure.
package testutil

func NewTestDB(t *testing.T) *sql.DB {
    t.Helper()
    db, err := sql.Open("postgres", os.Getenv("TEST_DATABASE_URL"))
    require.NoError(t, err)
    t.Cleanup(func() { db.Close() })

    // Run migrations.
    require.NoError(t, migrate.Up(db))

    // Clean state between tests.
    t.Cleanup(func() { truncateAll(t, db) })

    return db
}

func LoadFixture(t *testing.T, name string) []byte {
    t.Helper()
    data, err := os.ReadFile(filepath.Join("testdata", name))
    require.NoError(t, err)
    return data
}

Part XIV: Architecture and Package Design Link to heading

85. Single-Responsibility Packages Link to heading

A package should do one thing. A package named utils or helpers is a code smell, a junk drawer that grows without limit. Name packages after what they provide: auth, invoice, postgres, retry.

// Bad: everything in one package.
pkg/
├── utils/
│   ├── string_helpers.go
│   ├── http_client.go
│   ├── retry.go
│   └── database.go

// Good: each package has a single responsibility.
pkg/
├── retry/          // retry logic
│   └── retry.go
├── httputil/       // HTTP client helpers
│   └── client.go
├── postgres/       // PostgreSQL client
│   └── client.go
└── auth/           // authentication
    ├── token.go
    └── middleware.go

86. Dependency Boundaries via internal Packages Link to heading

Go’s internal/ directory restricts imports to the parent module. Use it to mark packages as private to a service, so that other services in the monorepo can’t depend on its implementation details.

apps/
├── orders-api/
│   ├── cmd/main.go
│   └── internal/        # only orders-api can import these
│       ├── handlers/
│       ├── services/
│       └── repos/
├── payments-api/
│   ├── cmd/main.go
│   └── internal/        # only payments-api can import these
│       ├── handlers/
│       └── services/

pkg/                     # shared code — importable by any service
├── auth/
├── retry/
└── logging/

87. Package-Oriented Architecture Over Layered Monoliths Link to heading

Organize by domain rather than by technical layer. A user/ package that holds the handler, service, repository and models for user operations is more cohesive than user logic scattered across handlers/, services/ and models/. (Readers of The Ideal Repository Structure will notice some tension with the layered model I argue for there. The two coexist more happily than you might think, with domains inside the layers, but I won’t pretend the tension isn’t real.)

// Layer-oriented (vertical slicing required to understand any feature):
handlers/
├── user_handler.go
├── order_handler.go
services/
├── user_service.go
├── order_service.go
models/
├── user.go
├── order.go

// Package-oriented (each feature is self-contained):
user/
├── handler.go
├── service.go
├── repository.go
├── model.go
├── user_test.go
order/
├── handler.go
├── service.go
├── repository.go
├── model.go
├── order_test.go

88. Hexagonal / Ports-and-Adapters-Lite Architecture Link to heading

The domain layer defines ports (interfaces), and adapters implement them for particular technologies. The domain never imports an adapter package, so dependencies flow inward.

// Port: defined in the domain package.
package order

type Repository interface {
    Save(ctx context.Context, order Order) error
    GetByID(ctx context.Context, id string) (Order, error)
}

type NotificationSender interface {
    SendOrderConfirmation(ctx context.Context, order Order) error
}

// Adapter: implements the port with PostgreSQL.
package postgres

type OrderRepository struct{ db *sql.DB }

func (r *OrderRepository) Save(ctx context.Context, o order.Order) error {
    // PostgreSQL-specific implementation
}

// Adapter: implements the port with SendGrid.
package sendgrid

type Notifier struct{ client *sendgrid.Client }

func (n *Notifier) SendOrderConfirmation(ctx context.Context, o order.Order) error {
    // SendGrid-specific implementation
}

89. Domain-Specific Package Naming Link to heading

Name packages after the domain concept they represent rather than the technology they use. A package named invoice survives a change of technology; one named mongoInvoice ties its name to an implementation detail.

// Bad: technology in the package name.
mongorepo/
├── user.go
├── order.go
rediscache/
├── session.go

// Good: domain names, technology is an implementation detail.
user/
├── repository.go      // interface
├── postgres.go        // PostgreSQL implementation
├── mongo.go           // MongoDB implementation (if needed)
order/
├── repository.go
├── postgres.go

90. Thin Orchestration / Fat Domain Logic Link to heading

Controllers and workflows should be thin layers of orchestration that delegate to domain services, while the domain holds the business rules, validation and state transitions. If a controller has an if that checks business state, the logic is in the wrong place.

// Thin controller — no business logic.
func (h *Handler) PlaceOrder(w http.ResponseWriter, r *http.Request) {
    var req PlaceOrderRequest
    if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
        http.Error(w, "invalid request", http.StatusBadRequest)
        return
    }
    order, err := h.service.PlaceOrder(r.Context(), req.toDomain())
    if err != nil {
        writeError(w, err)
        return
    }
    writeJSON(w, http.StatusCreated, toOrderResponse(order))
}

// Fat domain — all business rules live here.
func (s *Service) PlaceOrder(ctx context.Context, order Order) (Order, error) {
    if err := order.Validate(); err != nil {
        return Order{}, err
    }
    if err := s.inventory.Reserve(ctx, order.Items); err != nil {
        return Order{}, fmt.Errorf("reserve inventory: %w", err)
    }
    order.Status = StatusConfirmed
    order.ConfirmedAt = time.Now()
    return s.repo.Save(ctx, order)
}

91. Boundary-Focused Validation Link to heading

Validate at the system’s boundaries, where data enters from outside. Once data has crossed the boundary and become a domain entity, trust it; internal calls shouldn’t re-validate what the boundary has already checked.

// Validate at the API boundary.
func (h *Handler) CreateUser(w http.ResponseWriter, r *http.Request) {
    var req CreateUserRequest
    if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
        http.Error(w, "invalid JSON", http.StatusBadRequest)
        return
    }

    // Boundary validation — exhaustive.
    if err := req.Validate(); err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    // Domain layer trusts the input — no re-validation.
    user, err := h.service.CreateUser(r.Context(), req.toDomain())
    // ...
}

func (r CreateUserRequest) Validate() error {
    var errs []error
    if r.Email == "" {
        errs = append(errs, errors.New("email is required"))
    }
    if len(r.Name) < 2 {
        errs = append(errs, errors.New("name must be at least 2 characters"))
    }
    return errors.Join(errs...)
}

Part XV: Generics and Modern Go Link to heading

92. Generics-Based Reusable Utilities Link to heading

Generics, since Go 1.18, do away with per-type utility functions; a single Map, Filter or Contains works for every type.

func Map[T, U any](items []T, fn func(T) U) []U {
    result := make([]U, len(items))
    for i, item := range items {
        result[i] = fn(item)
    }
    return result
}

func Filter[T any](items []T, predicate func(T) bool) []T {
    var result []T
    for _, item := range items {
        if predicate(item) {
            result = append(result, item)
        }
    }
    return result
}

func Contains[T comparable](items []T, target T) bool {
    for _, item := range items {
        if item == target {
            return true
        }
    }
    return false
}

// Usage: type-safe, no casting.
names := Map(users, func(u User) string { return u.Name })
active := Filter(users, func(u User) bool { return u.Active })

93. Functional Slice/Map/Filter Helpers with Generics Link to heading

Building on that, a small collection of generic slice and map operations composes naturally and replaces many common loops with declarative expressions.

func GroupBy[T any, K comparable](items []T, key func(T) K) map[K][]T {
    groups := make(map[K][]T)
    for _, item := range items {
        k := key(item)
        groups[k] = append(groups[k], item)
    }
    return groups
}

func Reduce[T, U any](items []T, initial U, fn func(U, T) U) U {
    acc := initial
    for _, item := range items {
        acc = fn(acc, item)
    }
    return acc
}

func Keys[K comparable, V any](m map[K]V) []K {
    keys := make([]K, 0, len(m))
    for k := range m {
        keys = append(keys, k)
    }
    return keys
}

// Usage: group orders by status, sum totals.
byStatus := GroupBy(orders, func(o Order) string { return string(o.Status) })
total := Reduce(orders, Money(0), func(sum Money, o Order) Money { return sum + o.Total })

Part XVI: Remaining Patterns Link to heading

94. Controller-Runtime Reconcile Contracts Link to heading

In Kubernetes controller development, the reconcile function is the central contract. It takes the desired state (from the custom resource) and moves the actual state toward it, while the runtime handles requeueing, retries and leader election.

type Reconciler struct {
    client client.Client
}

func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
    var resource MyResource
    if err := r.client.Get(ctx, req.NamespacedName, &resource); err != nil {
        return ctrl.Result{}, client.IgnoreNotFound(err)
    }

    // Converge actual → desired.
    if resource.Status.Phase != resource.Spec.DesiredPhase {
        if err := r.converge(ctx, &resource); err != nil {
            return ctrl.Result{RequeueAfter: 30 * time.Second}, err
        }
    }

    return ctrl.Result{}, nil
}

95. Shared Informer Cache Patterns Link to heading

In Kubernetes operators, shared informers watch API resources and keep a local cache of them, and several controllers share one informer to avoid loading the API server twice.

func setupInformer(ctx context.Context, client kubernetes.Interface) cache.SharedIndexInformer {
    factory := informers.NewSharedInformerFactory(client, 30*time.Second)
    podInformer := factory.Core().V1().Pods().Informer()

    podInformer.AddEventHandler(cache.ResourceEventHandlerFuncs{
        AddFunc:    func(obj interface{}) { handlePodAdd(obj.(*v1.Pod)) },
        UpdateFunc: func(old, new interface{}) { handlePodUpdate(old.(*v1.Pod), new.(*v1.Pod)) },
        DeleteFunc: func(obj interface{}) { handlePodDelete(obj.(*v1.Pod)) },
    })

    factory.Start(ctx.Done())
    factory.WaitForCacheSync(ctx.Done())
    return podInformer
}

96. Eventual Consistency Reconciliation Link to heading

In distributed systems, services will disagree for a while. A reconciliation process periodically compares state across services and resolves the discrepancies, moving the system toward consistency.

func (r *Reconciler) reconcileInventory(ctx context.Context) error {
    // Compare order service state with inventory service state.
    orders, _ := r.orderStore.ListReserved(ctx)
    inventory, _ := r.inventoryStore.ListAllocations(ctx)

    orderSet := toSet(orders, func(o Order) string { return o.ID })
    allocSet := toSet(inventory, func(a Allocation) string { return a.OrderID })

    // Orders with reservations but no allocation — create allocation.
    for id := range orderSet {
        if _, ok := allocSet[id]; !ok {
            slog.Warn("missing allocation, creating", "order_id", id)
            _ = r.inventoryStore.Allocate(ctx, id)
        }
    }

    // Allocations with no matching order — release.
    for id := range allocSet {
        if _, ok := orderSet[id]; !ok {
            slog.Warn("orphaned allocation, releasing", "order_id", id)
            _ = r.inventoryStore.Release(ctx, id)
        }
    }
    return nil
}

97. Event-Driven Processing Pipelines Link to heading

Process events through a series of handlers, each of which can transform, filter, enrich or route them. The pipeline decouples the production of events from their consumption.

type EventHandler func(ctx context.Context, event Event) (Event, error)

type EventPipeline struct {
    handlers []EventHandler
}

func (p *EventPipeline) Process(ctx context.Context, event Event) error {
    current := event
    for _, handler := range p.handlers {
        next, err := handler(ctx, current)
        if err != nil {
            return fmt.Errorf("pipeline stage: %w", err)
        }
        current = next
    }
    return nil
}

// Compose pipelines from focused handlers.
pipeline := &EventPipeline{
    handlers: []EventHandler{
        validateEvent,
        enrichWithMetadata,
        filterDuplicates,
        routeToDestination,
    },
}

98. Stream Processing Worker Patterns Link to heading

Long-running workers that consume from Kafka, Redis Streams or SQS need consistent lifecycle management: connect, consume, process, acknowledge, and shut down cleanly.

type StreamWorker struct {
    consumer Consumer
    handler  func(ctx context.Context, msg Message) error
}

func (w *StreamWorker) Run(ctx context.Context) error {
    for {
        msg, err := w.consumer.Receive(ctx)
        if err != nil {
            if ctx.Err() != nil {
                return nil // clean shutdown
            }
            slog.Error("receive failed", "error", err)
            continue
        }

        if err := w.handler(ctx, msg); err != nil {
            slog.Error("handle failed", "error", err, "msg_id", msg.ID)
            msg.Nack()
            continue
        }
        msg.Ack()
    }
}

99. Zero-Allocation Hot-Path Optimization Link to heading

On performance-critical paths, avoid allocations by reusing buffers, using arrays on the stack, and avoiding the interface conversions that make values escape to the heap. Profile first, though, and optimize only what the profiler says is hot.

// Stack-allocated array for small, known-size operations.
func formatKey(prefix, id string) string {
    // Avoid fmt.Sprintf — it allocates.
    var buf [128]byte // stack-allocated
    n := copy(buf[:], prefix)
    n += copy(buf[n:], ":")
    n += copy(buf[n:], id)
    return string(buf[:n])
}

// Pre-allocated slice for known-capacity operations.
func collectIDs(users []User) []string {
    ids := make([]string, 0, len(users)) // pre-allocate capacity
    for _, u := range users {
        ids = append(ids, u.ID)
    }
    return ids
}

100. Consistent Hashing and Work Distribution Link to heading

For distributed work queues and caches, consistent hashing assigns keys to workers so that very little moves when a worker joins or leaves, and each worker is responsible for a predictable slice of the keyspace.

type ConsistentHash struct {
    ring     []uint32
    nodes    map[uint32]string
    replicas int
}

func (ch *ConsistentHash) Add(node string) {
    for i := range ch.replicas {
        hash := hashKey(fmt.Sprintf("%s-%d", node, i))
        ch.ring = append(ch.ring, hash)
        ch.nodes[hash] = node
    }
    sort.Slice(ch.ring, func(i, j int) bool { return ch.ring[i] < ch.ring[j] })
}

func (ch *ConsistentHash) Get(key string) string {
    hash := hashKey(key)
    idx := sort.Search(len(ch.ring), func(i int) bool {
        return ch.ring[i] >= hash
    })
    if idx == len(ch.ring) {
        idx = 0
    }
    return ch.nodes[ch.ring[idx]]
}

Summary Link to heading

I should end with a caveat. Patterns are not rules; they are tools, each one appropriate in some contexts and counterproductive in others. A worker pool is pure overhead for a CLI that processes ten items, and a circuit breaker is unnecessary for a service that only ever calls itself. The judgment lies in knowing when a pattern earns its complexity.

What the hundred have in common, as far as I can see, is a single orientation, which is to make the implicit explicit: dependencies injected explicitly rather than hidden in globals, errors classified explicitly rather than matched on strings, lifecycles managed explicitly rather than left to fire-and-forget goroutines, and interface contracts stated explicitly rather than left to duck typing and hope.

The difference between a codebase that scales and one that collapses under its own weight is rarely the cleverness of any single solution. It’s the consistency of the patterns holding the system together. When every service handles errors, shuts down and reports its health in the same way, the system becomes navigable, and a new engineer can read one service and understand all of them.

So build the vocabulary and apply it consistently. The patterns will compound in your favor, much as their absence compounds against you.

Q.E.D.  ∎

a.j.djalali

About the author

AJ Djalali is Research Faculty and Senior Research Engineer at the Georgia Institute of Technology. He graduated Cum Laude from Northwestern University and was a Fulbright Fellow at the University of Amsterdam before earning his PhD in Linguistics from Stanford, where he worked on formal semantics and mathematical linguistics.

In industry, he built the natural language technology behind Tableau's Ask Data, holds 9 patents in natural language interfaces for data visualization, and led cross-cloud AI strategy at Salesforce. In 2023 he co-founded PerceptivePanda, an AI startup acquired by Zapier in 2025.

Outside of work, he rides a 2015 Indian Scout, collects guitars, and hunts for rare vinyl pressings.