https://bonesmoses.org/tags/performance/ Recent content in Performance on BonesMoses.org Hugo -- gohugo.io en-us Fri, 09 Dec 2016 12:07:14 +0000 https://bonesmoses.org/2016/pg-phriday-planner-pitfalls/ Fri, 09 Dec 2016 12:07:14 +0000 https://bonesmoses.org/2016/pg-phriday-planner-pitfalls/ Recently a coworker asked me this question: Should I expect variance between minutes and hours for the same query? And I was forced to give him this answer: Potentially, but not commonly. Query planning is an inexact science, and regardless of the query being the “same query,” the data is not the “same data.” This isn’t generally the case, but on occasion, changes in data can affect the query execution path. https://bonesmoses.org/2016/pg-phriday-primal-planner-prep/ Fri, 18 Nov 2016 14:22:16 +0000 https://bonesmoses.org/2016/pg-phriday-primal-planner-prep/ The Postgres query planner is house of cards built upon the ever-shifting sand of our data. It has the utterly impossible mission of converting our ridiculous and inane requests into a logical series of fetch, filter, sort, join, and other instructions. Then the resulting steps must be ruthlessly efficient or the execution phase could very well saturate every hardware resource available; Set Theory isn’t very forgiving. Forewarned is forearmed is very apt when applied to database query planners. https://bonesmoses.org/2016/pg-phriday-broken-parts/ Fri, 21 Oct 2016 11:48:49 +0000 https://bonesmoses.org/2016/pg-phriday-broken-parts/ Partitioning tables in Postgres can be an extremely risky endeavor. Unfortunately on many larger systems, it’s also essentially a requirement; the maximum size of a Postgres table is 32TB. This isn’t just because converting an existing table to a series of partitions is expensive or time consuming. We must consider how the query planner will react to the partitioned version of a table. There’s also the very real risk we will (or already have) implement flaws in the trigger or constraint logic. https://bonesmoses.org/2016/pg-phriday-pesky-partition-plans/ Fri, 07 Oct 2016 14:05:51 +0000 https://bonesmoses.org/2016/pg-phriday-pesky-partition-plans/ For all of those warehouse queries that never seem to complete before the heat death of the universe, there’s often a faster version. Sometimes this is due to a fundamental misunderstanding of how queries work, or how Postgres specifically functions. The trick is knowing when to back away slowly from an ugly but efficient query, and when to inject a flurry of predicates to fully illustrate the original intent of the query so the planner makes better decisions. https://bonesmoses.org/2016/pg-phriday-exceptional-performance/ Fri, 01 Jul 2016 11:41:45 +0000 https://bonesmoses.org/2016/pg-phriday-exceptional-performance/ Like any programming language, the PL/pgSQL Postgres procedural language has plenty of handy control structures. Among those thankfully modern accoutrements is the humble EXCEPTION block. While not the more prevalent try/catch methodology, we can use BEGIN anywhere to start an embedded section for the same effect. Knowing this is the case, what are the costs of actually using them? Postgres is fairly streamlined, and as such, can take several shortcuts when there are no exception blocks within a chunk of PL/pgSQL. https://bonesmoses.org/2015/pg-phriday-parallel-sequence-scans/ Fri, 13 Nov 2015 11:49:43 +0000 https://bonesmoses.org/2015/pg-phriday-parallel-sequence-scans/ A couple days ago, Robert Haas announced that he checked in the first iteration of parallel sequence scans in the Postgres 9.6 branch. And no, that’s not a typo. One of the great things about the Postgres devs is that they have a very regimented system of feature freezes to help ensure timely releases. Thus even though 9.5 just released its second beta, they’re already working on 9.6. So what is a sequence scan, and why does this matter? https://bonesmoses.org/2015/pg-phriday-massively-distributed-operation/ Fri, 30 Oct 2015 12:12:46 +0000 https://bonesmoses.org/2015/pg-phriday-massively-distributed-operation/ Postgres has been lacking something for quite a while, and more than a few people have attempted to alleviate the missing functionality multiple times. I’m speaking of course, about parallel queries. There are several reasons for this, and among them include various distribution and sharding needs for large data sets. When tables start to reach hundreds of millions, or even billions of rows, even high cardinality indexes produce results very slowly. https://bonesmoses.org/2015/pg-phriday-parallel-o-postgres/ Fri, 16 Oct 2015 11:58:11 +0000 https://bonesmoses.org/2015/pg-phriday-parallel-o-postgres/ I wasn’t able to write an article last week due to an unexpected complication regarding tests I was running to verify its contents. So this week, it’s going to be extra special! Also long. What’s the fastest way to load a Postgres table? If you believe the documentation, the COPY command is the best way to unceremoniously heave data into a table. Fortunately after all of our talk about partitions, our minds are primed and ready to think in chunks. https://bonesmoses.org/2015/pg-phriday-when-partitioning-goes-wrong/ Fri, 02 Oct 2015 16:33:10 +0000 https://bonesmoses.org/2015/pg-phriday-when-partitioning-goes-wrong/ I’ve been talking about partitions a lot recently, and I’ve painted them in a very positive light. Postgres partitions are a great way to distribute data along a logical grouping and work best when data is addressed in a fairly isloated manner. But what happens if we direct a basic query at a partitioned table in such a way that we ignore the allocation scheme? Well, what happens isn’t pretty. Let’s explore in more detail. https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-sex-offenders/ Fri, 24 Jul 2015 12:20:16 +0000 https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-sex-offenders/ We’re finally at the end of the 10-part Postgres (PostgreSQL) performance series I use to initiate new developers into the database world. To that end, we’re going to discuss something that affects everyone at one point or another: index criteria. Or to put it another way: Why isn’t the database using an index? It’s a fairly innocuous question, but one that may have a surprising answer: the index was created using erroneous assumptions. https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-indexing-the-world/ Fri, 17 Jul 2015 11:42:28 +0000 https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-indexing-the-world/ An easy way to give Postgres (PostgreSQL) a performance boost is to judiciously use indexes based on queries observed in the system. For most situations, this is as simple as indexing columns that are referenced frequently in WHERE clauses. Postgres is one of the few database engines that takes this idea even further with partial indexes. Unfortunately as a consequence of insufficient exposure, most DBAs and users are unfamiliar with this extremely powerful functionality. https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-maximized-value/ Fri, 10 Jul 2015 12:46:10 +0000 https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-maximized-value/ I apologize for putting this series on a short hiatus last week for the 4th of July. But worry not, for this week is something special for all the developers out there! I’m going to try to make your life easier for a change. Screw the database! As a Postgres (PostgreSQL) DBA, it’s easy to get tied up in performance hints, obscure syntax, and mangled queries, but it’s really all about the people. https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-in-the-loop/ Fri, 26 Jun 2015 11:18:29 +0000 https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-in-the-loop/ As a database, Postgres (PostgreSQL) is fairly standard in its use of SQL. Developers of all colors however, might have trouble switching gears and thinking in set operations, since so many language constructs focus on conditionals and looping. Last week in the performance pitfalls series, we discussed a bit of Set Theory, and how ignorance of its implications can be disastrous. But what about the more mundane? What happens, for instance, when we treat a database like a programming language? https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-functionally-bankrupt/ Fri, 19 Jun 2015 12:01:23 +0000 https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-functionally-bankrupt/ Functions are great. Having cut my teeth on a database that didn’t even provide the ability to define functions, I’ve come to almost take them for granted in Postgres (PostgreSQL). However, with this kind of ubiquity, sometimes they can be overused in ways that don’t seem to be part of the common programmer lexicon. In this week’s PG Phriday series on performance-killing missteps, I’m going to talk a bit about set theory, and how a certain amount of familiarity is necessary to properly interact with a database. https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-out-of-order/ Fri, 12 Jun 2015 12:14:34 +0000 https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-out-of-order/ There are a lot of database engines out there. As such, a developer or DBA will naturally have varying levels of experience with each, and some of this might conflict with how Postgres (PostgreSQL) operates. These kinds of embedded misunderstandings can cause potential issues by themselves, but in this particular case, corrective action is fairly simple. So this week, I’d like to talk about indexes. Many people treat them as a “make query faster” button, and this often results in egregious misuse. https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-cast-away/ Fri, 05 Jun 2015 12:31:32 +0000 https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-cast-away/ Unlike a lot of programming languages that have loose typing, databases are extremely serious about data types. So serious in fact, many patently refuse to perform automatic type conversions in case the data being compared is not exactly equivalent afterwards. This is the case with Postgres (PostgreSQL) and surprisingly often, queries will suffer as a result. Fortunately this is something we can address! There are easier ways to demonstrate this, but in the interests of making this more of an adventure, let’s use a join between two tables: https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-in-sanity/ Fri, 29 May 2015 12:17:32 +0000 https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-in-sanity/ When working with a database, sometimes performance problems are both far more subtle, and much worse than a query itself might suggest. The topic of this week’s Postgres (PostgreSQL) performance killers article concerns the use of the IN clause, and how misusing it can catastrophically obliterate the database in mysterious ways. To that end, we’ll use a slightly revised single-table test case since it’s served us pretty well so far: https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-forgetting-it-exists/ Fri, 22 May 2015 18:41:33 +0000 https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-forgetting-it-exists/ For the second of my ten part series on hidden Postgres (PostgreSQL) performance killers, I’m going to talk about something called an anti-join. It’s not a well-known approach outside of the database world, but due to how it works, it can impart almost magical plan revisions that drastically improve query performance in the right scenario. Developers can add it to a growing bag of tricks when working on database-driven content, since it comes in handy more often than you might expect. https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-forgetting-to-explain/ Fri, 15 May 2015 14:47:12 +0000 https://bonesmoses.org/2015/pg-phriday-10-ways-to-ruin-performance-forgetting-to-explain/ Yesterday I gave the developers at my company what I call a DBA Chat. It’s something I try to do every month to keep them apprised on various features, caveats, performance considerations, and so on. I find that educating the folks who regularly work with the database does wonders for application performance and my sanity. The benefit of this long format is that I can go over more information than a time constrained set of slides. https://bonesmoses.org/2015/pg-phriday-functions-and-performance-attributes/ Fri, 08 May 2015 15:56:42 +0000 https://bonesmoses.org/2015/pg-phriday-functions-and-performance-attributes/ Users both old and new are likely aware that Postgres has functions. Some lucky readers may have even written one or two. For those without much experience in this area, or are thinking about contributing a few functions to your database for the first time, there are a couple things you should know. This week, I want to cover a few of the function declaration decorators. If you have no idea what I’m talking about, then this is probably something you’ll want to read regardless of your experience level. https://bonesmoses.org/2015/pg-phriday-lets-talk-about-data-storage/ Fri, 24 Apr 2015 11:48:06 +0000 https://bonesmoses.org/2015/pg-phriday-lets-talk-about-data-storage/ As a DBA, I strive not to live in an isolated ivory tower, away from the developers that are destined to fill our databases with volumes of data from a myriad of applications. It’s important, I think, to occasionally come back to the basics. So I’d like to discuss one of the core elements that Postgres DBAs might be intimately familiar with, but comes up often enough that some clarification is warranted. https://bonesmoses.org/2015/a-short-examination-of-pg_shard/ Thu, 12 Mar 2015 16:28:01 +0000 https://bonesmoses.org/2015/a-short-examination-of-pg_shard/ For part of today, I’ve been experimenting with the new-ish pg_shard extension contributed by CitusData. I had pretty high hopes for this module and was extremely excited to try it out. After screwing around with it for a while, I can say it has a lot of potential. Yet I can’t reasonably recommend it in its current form. The README file suggests quite a few understandable caveats, but it’s the ones they don’t mention that hurt a lot more. https://bonesmoses.org/2015/pg-phriday-materialized-views-revisited/ Fri, 06 Mar 2015 11:32:08 +0000 https://bonesmoses.org/2015/pg-phriday-materialized-views-revisited/ Materialized views are a great improvement to performance in many cases. Introduced in PostgreSQL 9.3, they finally added an easy method for turning a view into a transient table that could be indexed, mined for statistics for better planner performance, and easily rebuilt. Unfortunately, refreshing a materialized view in PostgreSQL 9.3 caused a full exclusive lock, blocking any use until the process was complete. In 9.4, this can finally be done concurrently, though there are still a couple caveats. https://bonesmoses.org/2015/pg-phriday-postgresql-select-filters/ Fri, 27 Feb 2015 11:54:29 +0000 https://bonesmoses.org/2015/pg-phriday-postgresql-select-filters/ Long have CASE statements been a double-edged sword in the database world. They’re functional, diverse, adaptive, and simple. Unfortunately they’re also somewhat bulky, and when it comes to using them to categorize aggregates, something of a hack. This is why I wanted to cry with joy when I found out that PostgreSQL 9.4 introduced a feature I’ve always wanted, but found difficult to express as a need. I mean, CASE statements are fine, right? https://bonesmoses.org/2014/friends-dont-let-friends-use-loops/ Fri, 25 Jul 2014 12:12:06 +0000 https://bonesmoses.org/2014/friends-dont-let-friends-use-loops/ Programming is fun. I love programming! Ever since I changed my career from programming to database work, I’ve still occasionally dabbled in my former craft. As such, I believe I can say this with a fair amount of accuracy: programmers don’t understand databases. This isn’t something small, either; there’s a fundamental misunderstanding at play. Unless the coder happens to work primarily with graphics, bulk set-based transformations are not something they’ll generally work with. https://bonesmoses.org/2014/foreign-keys-are-not-free/ Wed, 14 May 2014 16:40:23 +0000 https://bonesmoses.org/2014/foreign-keys-are-not-free/ PostgreSQL is a pretty good database, and I enjoy working with it. However, there is an implementation detail that not everyone knows about, which can drastically affect table performance. What is this mysterious feature? I am, of course, referring to foreign keys. Foreign keys are normally a part of good database design, and for good reason. They inform about entity relationships, and they verify, enforce, and maintain those relationships. Yet all of this comes at a cost that might surprise you.