100,000 Lines of C Later: Re-architecting Enterprise Postgres Backups in Go
Tuesday, September 08 at 14:40–15:30
pgBackRest is the de facto standard for enterprise PostgreSQL backup. It's a decade of bug fixes, fault-tolerance lessons, and production-hardened invariants, packed into nearly 100,000 lines of dense C with custom networking, manual memory management, and bespoke wire protocols. In April 2026, after losing corporate backing, the original maintainer archived the project. It now survives, but the close call made the underlying problem clear: a codebase that size in that language inevitably draws contributors from a small pool, and critical infrastructure deserves alternatives that more people can actually contribute to.
So I started writing pgSafe: a spiritual successor in Go, with every modern tool at my disposal. I studied pgBackRest's source; pgSafe shares its concepts and operational rules, and none of its code. The goal is functional parity for the common deployment patterns: full and incremental backups, point-in-time recovery, and five storage backends (POSIX, S3, Azure Blob, GCS, SFTP) across PostgreSQL 13 through 18.
The interesting result is how much smaller a greenfield reimplementation actually is. Roughly two-thirds of pgBackRest's C is accidental complexity that a modern toolchain simply absorbs: macros that imitate higher-level features in C, hand-rolled data structures and types, custom HTTP/TLS clients and JSON parser, and the worker coordination machinery that errgroup collapses to a few dozen lines. PG 17's WAL summarizer and the backup_manifest JSON format absorb several thousand more lines of backup engine logic. What remains, the actual backup orchestration, fits in roughly 20,000 lines of Go.
The talk covers four areas:
- The "10 invariants" of PostgreSQL backups: the strict operational rules pgBackRest learned from a decade of production incident post-mortems, and how each one maps idiomatically into Go.
- The design tenets that constrain the rewrite: PostgreSQL never knows pgSafe exists; standards-and-libraries first; no storage credentials on the database host's disk.
- Three operating modes: libpq-only, remote-parallel, and same-host/SSH worker, balancing operational simplicity, throughput, and security.
- The parts that genuinely need careful design: primarily the resume protocol, the WAL bracket guarantees, and the credential narrowing model. These don't shrink, no matter what language you write them in.
You'll leave this talk with a clear picture of what makes a PostgreSQL backup tool performant and reliable, a working frame for evaluating "rearchitect, don't port" projects in your own stack, and maybe a little inspiration to take on a piece of important-but-aging infrastructure of your own, just for fun.
pgSafe is open source and welcoming contributors; if PostgreSQL backups are your kind of fun, come help build the alternative.