Lazy NRR is Not NRR. Accept No Imitations or Subtitutes.

The other day I was looking at an ARR bridge [1] with a young finance ace.  He made a few comments and concluded with, “and net revenue retention (NRR) is thus 112%, not bad.”

I thought, “Wait, stop!  You can’t calculate NRR from an ARR bridge [2].”  It’s a cohort-based measure.  You need to look at the year-ago cohort of customers, get their year-ago ARR, get that same group’s current ARR, and then divide the current ARR by the year-ago.

“Yes, you can,” he said.  “Just take starting ARR, add net expansion, and divide by starting ARR.  Voila.”

Expecto patronum!  Protect me from this dark magic.  I don’t know what that is, I thought, but that’s not NRR.

Then I stewed on it for a bit.  In some ways, we were both right.

• Under the right circumstances, I think you can calculate NRR using an ARR bridge [3]. But the whole beauty of the metric is to float over that definitional swamp and just divide two numbers — so I inherently don’t want to.
• My friend’s definition, one I suspect is common in finance whiz circles, was indeed one shortcut too short. But, under the right circumstances, you can improve it to work better in certain cases.

The Trouble with Churn Rates
For a long time, I’ve been skeptical of calculations related to churn rates.  While my primary problems with churn rates were in the denominator [4], there are also potential problems with the numerator [5].  Worse yet, once churn rates get polluted, all downstream metrics get polluted along with them – e.g., customer lifetime (LT), lifetime value (LTV), and ergo LTV/CAC.  Those are key metrics to measure the value of the installed base — but they rely on churn rates which are easily gamed and polluted.

What if there were a better way to measure the value of the installed base?

There is.  That’s why my SaaStr 2019 session title was Churn is Dead, Long Live Net Dollar Retention [6].  The beauty of NRR is that it tells you want you want to know – once you acquire customers, what happens to them? – and you don’t have to care which of four churn rates were used.  Or how churn ARR itself was defined.  Or if mistakes were made in tracking flows.

You just need to know two things:  ARR-now and ARR-then for “then” cohort of customers [7].

A Traditional ARR Bridge
To make our point, let’s review a traditional ARR bridge.

Nothing fancy here.  Starting ARR plus new ARR of two types:  new logo customers (aka, new logo ARR) and existing customers (aka, expansion ARR).  We could have broken churn ARR into two types as well (shrinkage and lost), but we didn’t need that breakout for this exercise.

Now, let’s add my four favorite rows to put beneath an ARR bridge [8]:

Here’s a description:

• Net new ARR = New ARR – churn ARR. How much the water level increased in the SaaS leaky bucket of ARR.  Here in 1Q21, imagine we spent $2,250K in S&M in the prior quarter.  Our CAC ratio would be a healthy 1.0 on a new ARR basis, but a far less healthy 2.1 on a net new ARR basis.  That’s due to our quarterly churn of 8%, which when annualized to 32%, flies off the charts.
• Expansion as a percent of new ARR = expansion ARR / new ARR. My sense is 30% is a magic number for an established growth-phase startup.  If you’re only at 10%, you’re likely missing the chance to expand your customers (which will also show up in NRR).  If you’re at 50%, I wonder why you can’t sell more new logo customers.  Has something changed in the market or the salesforce?
• Net expansion = expansion ARR – churn ARR. Shows the net expansion or contraction of the customer base during the quarter.  How much of the bucket increase was due to existing (as opposed to new) customers?
• Churn rate, quarterly. I included this primarily because it raises a point we’ll hit when discussing lazy NRR.  Many people calculate this as = churn ARR / starting ARR (quarter).  That’s what I call “simple quarterly,” and you’ll note that it’s always lower than just “quarterly,” which I define as = churn ARR / starting ARR (year) [9].  The trace-precedents arrows below show the difference.

Lazy NRR vs. Cohort-Based NRR
With that as a rather extensive warm-up, let’s discuss what I call lazy NRR.

Lazy NRR is calculated as described above = (starting ARR + net expansion) / starting ARR.  Lazy NRR is a quarterly expansion metric.

Let’s look at a detailed example to see what’s really being measured.

This example shows the difference between cohort-based NRR and Lazy NRR:

• Cohort-based NRR, a year-over-year metric that shows expansion of the two year-ago customers (customers 1 and 2).  This is, in my book, “real NRR.”
• Lazy NRR, simple quarterly, which compares net expansion within the current quarter to starting ARR for that quarter.

The point of the trace-precendents arrows shows you that while the result coincidentally might be similiar (and in this case it is not), that they are measuring two completely different things.

Let’s talk about the last row, lazy NRR, cohort-based approximation, which takes starting ARR from year-ago customers, then adds (all) net expansion over the year and divides by the year-ago starting ARR. The problem?  Customer 3.  They are not in the year-ago cohort, but contribute expansion to the numerator because, with only an ARR bridge, you can’t separate year-ago cohort net expansion from new-customer net expansion.  To do that, you’d need to have ARR by customer [10].

Lazy NRR is not NRR.  NRR is defined as snapshot- and cohort-based.  Accept no substitutes or imitations.  Always calculate NRR using snapshots and cohorts and you’ll never go wrong.

Layer Cakes Tell No Lies
While I’m usually quite comfortable with tables of numbers and generally prefer receiving them in board reports, this is one area where I love charts, such as this layer cake that stacks annual cohorts atop each other.  I like these layer cakes for several reasons:

• They’re visual and show you what’s happening with annual cohorts.
• Like snapshot- and cohort-based NRR, they leave little to no room for gaming.  (They’re even harder to survivor bias as you’d have to omit the prior-period ARR.)
• Given my now-distant geophysics background, they sometimes remind me of sedimentary rock.  (Hopefully yours don’t look like that, as unmetamorphized, sedimentary rock represents an NRR of only 100%!)

The spreadsheet for this post is available here.

(The post was revised a few times after initial publication to fix mistakes and clarify points related to the cohort-based approximation.  In the end, the resultant confusion only convinced me more to only and always calcuate NRR using cohorts and snapshots.)

# # #

Notes
Edited 10/8/22 to replace screenshots and fix spreadsheet bug in prior version.

[1] Starting ARR + new ARR (from new logo and expansion) – churn ARR (from shrinkage and lost) = ending ARR

[2] I probably should have said “shouldn’t.”  Turns out, I think you can, but I know you shouldn’t.  We’ll elaborate on both in this post.

[3] Those conditions include a world where customers expand or contract only on an annual basis (as you are unable to exclude expansion or contraction from customers signed during the year since they’re not sepearated in an ARR bridge) and, of course, a clear and consistent definition of churn, playing fairly with no gaming designed understate churn or overstate expansion, and avoidance of mistakes in calculations.

[4] Churn rates based off the whole ARR pool can halve (or more than halve) those based on the available to renew (ATR) pool, for example if a company’s mean contract duration is 2 or 3 years.  ARR churn rates are probably better for financial calculations, but ATR churn rates are a better indicator of customer satisfaction

[5] Examples of potential problems, not all strictly related to calculation of churn ARR, but presented for convenience.

• Expansion along the way. Consider a customer who buys 100-unit contract, expands to 140 the next quarter (without signing a new one-year agreement that extends the contract), and then at the annual renewal renews for 130.  The VP of CS wants to penalize the account’s CSM for 10 units of churn whereas the CFO wants to tell investors its 30 units of expansion.  Which is it?  Best answer IMHO is 40 units of expansion in the second quarter and 10 units of churn at the renewal, but I’ve seen people/systems that don’t do it that way.   NRR sees 130% rate regardless of how you count expansion and churn.

• Potential offsets and the definition of customer – division 1 has 100 units and shrinks to 80 at renewal while a small 40-unit new project starts at division 2. Is that two customers, one with 20 units of churn and one new 40-unit customer or is it one customer with 20 units of expansion?  NRR sees either 80% rate or 120% rate as function of customer definition, but I’d hope the NRR framing would make you challenge yourself to ask:  was division 2 really a customer and ergo belong in the year-ago cohort?

• Potential offsets and the definition of product – a customer has 100 units of product A, is unhappy, and shrinks to A to 60 units while buying your new product B for 40. Did any churn happen?  In most systems, the answer is no because churn is calculated at the account level.  Unless you’re also tracking product-level churn, you might have trouble seeing that your new product is simply being given away to placate customers unhappy with your first one.  NRR is inherently account-level and doesn’t solve this problem – unless you decide to calculate product-level NRR, to see which products are expanding and which are shrinking.

• Adjustments.  International companies need to adjust ARR for fluctuations in exchange rates.  Some companies adjust ARR for bad debt or non-standard contracts.  Any and all of these adjustments complicate the calculation of churn ARR and churn rates.

• Gaming.  Counting trials as new customers and new ARR, but excluding customers <$5K from churn ARR calculations (things won’t foot but few people check).  Renewing would-be churning customers at $1 for two years to delay count-based churn reporting (ARR churn rates and NRR will see through this).  Survivor biasing calculations by excluding discontinuing customers.  Deferring ARR churn by renewing would-be churning customers with net 360 payables and a handshake (e.g., side letter) to not collect unless thing XYZ can be addressed (NRR won’t see through this, but cash and revenue won’t align).

[6] Since I now work frequently with Europe as part of my EIR job with Balderton Capital, I increasingly say “NRR” instead of “NDR” (net dollar retention), because for many of the companies I work with it’s actually net Euro retention.  The intent of “dollar” was never to indicate a currency, but instead to say:  “ARR-based, not count-based.”  NRR accomplishes that.

[7] Some companies survivor bias their NRR calculation by using the now-value and then-value of the now cohort, eliminating discontinuing customers from the calculation.   Think:  of the mutual funds we didn’t shut down, the average annual return was 12%.

[8] If you download the spreadsheet and expand the data groups you can see some other interesting rows as well.

[9] The flaw in “simple quarterly” churn is that, in a world that assumes pure annual contracts, you’re including people who were not customers at the start of the year and ergo cannot possibly churn in the calculations.  While you use the same numerator in each case, you’re using an increasing denominator and with no valid reason for doing so.  See here for more.

[10] In which case you might as well calculate NRR as defined, using the current and year-ago snapshots.