I’m almost done with this year’s crazy conference season schedule, just Brazil’s PG.BR next week left. All of my recent presentations are now available at our talks page. You’ll also find many of the talks from our CHAR(11) conference this summer there for the first time. Those were unavailable for a while due to an unfortunately timed web site change.
I like to do some original research for my talks, and this year that included a look into Synchronous Replication and Durability Tuning in PostgreSQL 9.1, specifically the performance side. At last week’s PG.EU I gave an updated version of this talk (with Simon Riggs), including a bit more info than I had available during the Postgres Open presentation on the same topic.
The good news/bad news performance results are nicely summarized by the “Group commit magic” graph on slide 17. There I was replicating across the Atlantic Ocean, crossing from my home here in Baltimore over to the conference site in Amsterdam. When doing synchronous replication, the speed of light determines how fast any single client can commit. You can only reach about 50% of that round trip time given current (and expected future) technology here. The efficiency rate of the fiber-optic cables used is not perfect, and tasks like routing add some overhead too.
With that sort of distance, the round trip time is at least 100 milliseconds. What this means is that no one client can commit more than 10 times per second. This shows just how difficult sync rep is to run well, the maximum rate drops fast if you expect to leave your local data center to commit. Light turns out to be really slow compared with what many people expect.
The great thing I proved in that slide is that the efficiency when multiple clients are committing at the same time scales almost linearly. If you want 1000 transactions/second over this sort of distance, you can get that–but you’ll need just over 125 clients going at once to do it. Each one of those clients will be seeing 10 commits/second and 100 millisecond latencies (and higher for a small percentage, peak latency in the test was closer to 600ms). But each commit reply will be acknowledging a pile of clients at once, just by sending a small packet with an updated “committed up to this point” response.
When the speed of light turns out to be your bottleneck, there’s not much you can do about that attacking directly. Some people who want to reach higher rates might architect their systems with many smaller clients, as I’ve shown in this example. I was able to reach my goal of 2000 INSERT commits/second here, but it took 275 clients to get there.
The other great thing about how PostgreSQL implements sync rep is that it’s controlled per transaction. Once you see how expensive the commits are, if that’s too high for some of your data, you can always tweak that for higher performance just by disabling sync rep for some transactions. Having such fine-grained control over synchronous commits is a unique feature to PostgreSQL, allowing something like a Quality of Service suggestion to the database. The PostgreSQL code as of 9.1 really has an unprecedented range of trade-offs here. You can go for faster but not very durable at all (with unlogged tables), locally durable but not guaranteed to a remote data center at a medium speed, all the way up to fully synchronous and very expensive to commit. It’s possible to argue that other database choices are better at one end of this range. You might use MongoDB for higher speeds at the low durability range, and Oracle for their better tested sync rep capabilities (I saw better tested simply because the PostgreSQL 9.1 code is very new relative to Oracle’s implementation).
PostgreSQL started in the middle here, and with 9.1 it’s expanded nicely toward both ends of the spectrum at once. It’s now providing options for higher and lower durability at the same time, in one database, and with the speed/durability trade-off adjustable for every transaction. Building one size fits all software is really hard, and the new features in 9.1 nicely push out capabilities here for several popular use cases, all at the same time, and only when you want to pay for them.