The Mathematics of Difficulty in Shadowdark

The other day I was having a discussion in the Shadowdark Discord about dice maths, so I decided to dive in and take a look. Because of the d20 vs DC core mechanic, the maths turns out quite different to other OSR games.

In Shadowdark there are two main “dials” the game master can use to adjust the difficulty of a check. Those are:

• Adjusting the DC between:
  • Easy (DC 9)
  • Normal (DC 12)
  • Hard (DC 15)
  • Extreme (DC 18)
• Granting advantage or disadvantage

These adjustments are made on top of the bonuses & penalties the PCs already have from their stats, class talents, and magic items.

I have thus set out to answer the following question: What is the mathematical difference between adjusting a DC, and granting advantage/disadvantage

Generating Player Character Statistics

Before we can determine the difference between adjusting DCs or granting advantage/disadvantage, we first have to work out what effects the different methods of character generation in Shadowdark have on player statistics and their bonuses & penalties.

Shadowdark has 6 core stats: strength, dexterity, constitution, intelligence, wisdom & charisma. In shadowdark, as in many other OSR games, these stats are generated by rolling 3d6 six times and placing them, in order, into the six core stats. This is what I’m going to call the traditional method.

Beyond the traditional method, Shadowdark also offers an optional rule which allows players to discard all of their stats & roll again if all six of them are below 14. This is what I’m going to call the at least 14 method.

I want to compare the outcomes of these two methods of stat generation, to identify the following:

• What you can expect the average stat of a PC to be.
• What you can expect a PC’s highest stat to be
• What you can expect a PC’s lowest stat to be

Traditional Method

For the traditional method, this is pretty easy to work out. We just need to calculate the following:

• Average Stat: Expected value of 3d6
• Highest Stat: The expected value of the highest die across every permutation of the roll 6×(3d6)
• Lowest Stat: The expected value of the lowest die across every permutation of the roll 6×(3d6)

AnyDice can make this sort of calculation easy. In the end we get a curve that looks like this:

output 6@(6d(3d6)) named "Average lowest stat"
output 3d6 named "Average stat"
output 1@(6d(3d6)) named "Average highest stat"

This gives us a result that shows the following means for each stat:

At least 14 Method

The at least 14 method is actually significantly harder to calculate, as we need to be able to exclude from our averages any result where every stat is under 14. It turns out this is still possible to do in anydice with custom functions, and the resulting curve is as follows:

function: validatedaverage D:s {
  if (1@D <= 13) {
    result: d{}
  }
  result: d D
}

function: validatedmax D:s {
  if (1@D <= 13) {
    result: d{}
  }
  result: 1@D
}

function: validatedmin D:s {
  if (1@D <= 13) {
    result: d{}
  }
  result: 6@D
}

output [validatedmin 6d(3d6)] named "Average lowest stat (at least one stat ≥ 14)"
output [validatedaverage 6d(3d6)] named "Average stat (at least one stat ≥ 14)"
output [validatedmax 6d(3d6)] named "Average highest stat (at least one stat ≥ 14)"

This gives us mean stats that look like:

While this table makes the results look pretty similar to those generated with the traditional method, there are actually still some really important differences:

• A PC’s highest stat is always 14 or higher. While this still gives us an average of +2 for a PC’s highest stat, it increases their chances of having at least a +3 from 24.75% with the traditional method, to 37.86% with the at least 14 method — that’s an increase of 13.11%.
• Because of the extra peak in the curve for the average stat, a player is more likely to get a 14 for any given stat than they are a 12 or 13. Accordingly, their probability of getting a +1 in any given stat from 37.5% to 43.9% — a 6.4% bump.

Adjusting DCs vs Advantage & Disadvantage

Now that we know what we can expect the average PC’s stats to be, we can use those stats (-2, +0, +2) as a baseline to calculate the probability of them succeeding against the standard DCs.

Flat rolls vs Standard DCs

Shadowdark’s 4 standard DCs are:

• Easy (DC 9)
• Normal (DC 12)
• Hard (DC 15)
• Extreme (DC 18)

And the probability of success vs each of those DCs with the bonuses & penalties from earlier is shown below.

loop A over {9, 12, 15, 18} {
  loop B over {-2, 0, +2} {
    output 1d20 + B >= A named "[B] vs DC [A]"
  }
}

Adding in Advantage & Disadvantage

It is often said that advantage is roughly equal to a +5 bonus, and disadvantage a -5 penalty; while this sometimes gets close, the reality is actually far more complicated. The benefit or drawback of advantage & disadvantage are highly situational, and have wildly different effects on the probability depending on the stat bonuses already in play as well as the DC in question.

There are two ways to look at the impacts of applying advantage or disadvantage:

1. Direct Probabilities. We can look directly at the probability of success vs a DC, with bonuses; or
2. Change in Probabilities. We can look at the change in chances of success vs a DC, with bonuses

We will look at direct probabilities first, because they are needed to calculate the change in probabilities.

Direct Probabilities

We can, again, use AnyDice to help us calculate these probabilities, and get the following results:

When you have advantage, stepping up & down between DCs does not result in a linear change in probability; instead probability changes on a curve that drops off faster and faster as DCs increase.

Interestingly, the curve is different depending on the static bonuses & penalties being applied — the worse a PC’s penalty, the faster their chances of success drop when stepping up through DCs. This means that the worse a PC’s penalty, the less impactful advantage is for them. On the other hand, the greater a PC’s bonus, the more impactful advantage is for them.

For example, given an Easy (DC 9) check with advantage, a +2 bonus only increases your chances of success by 7%. Without advantage, that +2 would increase your chances by 10%. Conversely, the same +2 bonus vs an Extreme (DC 18) check confers a whopping 16% increase in your chances of success.

When we take a look at disadvantage, we see the exact same pattern just in reverse; PCs with big penalties are more negatively impacted, and PCs with big bonuses are less negatively impacted.

Change in Probabilities

Instead of looking at the probability of success given advantage or disadvantage, we can instead look at how your probability of success changes, when compared to a flat roll without advantage or disadvantage. We can then use that change in probability to approximate an equivalent bonus to advantage at different DCs — This isn’t to say that advantage should be equated to bonuses, but it does help us get a better understanding of the impacts of advantage.

Here we see the same pattern we saw earlier — the higher your bonus, the more impactful advantage is for you. We also continue to see that advantage becomes less and less helpful the higher the DC gets. Given a bonus of +0 vs an Easy (DC 9) check, advantage increases a PCs chances of success by 24% (roughly equivalent to a +5 in a flat roll), but against an Extreme (DC 18) check, advantage only increases the PC’s chances by 12.75% (Roughly equivalent to a +3 bonus).

This effect becomes even more pronounced if you start with a -2 penalty instead of a +0 bonus. In this case, advantage only gives a 4.75% increase to the PC’s chances of success vs an Extreme (DC 18) check — less than a +1 bonus.

This pattern exists because Advantage & Disadvantage increase your chances of getting a higher or lower number, but they never increase the maximum or minimum number that you can roll. As the DC increases for a check, and a PC gets greater and greater penalties, their chances of success even if they roll really well go down. For example, if a PC has a penalty of -4 vs an Extreme (DC 18) check then even a 20 on the dice cannot beat the DC, and thus advantage is useless in this situation. The same is true if you go the other way too, the lower the DC and the higher the bonus, the less impact advantage has — we just don’t see this as clearly in Shadowdark as there are no DCs under 9 (but you can still see the beginnings of this in the chart between DC 9 and DC 12, where the effectiveness of advantage seems to flip)

And for disadvantage, we once again see the exact same patterns but mirrored:

Given all of this information, we can calculate a rough equivalence between advantage & disadvantage for different DCs, and different static bonuses/penalties:

Evidently, while a lot of these numbers do hover around the ±5 mark, it is a lot more complicated than that.

loop A over {9, 12, 15, 18} {
  loop B over {-2, 0, +2} {
    output 1@2d20 + B >= A named "Advantage [B] vs DC [A]"
    output 2@2d20 + B >= A named "Disadvantage [B] vs DC [A]"
  }
}

Conclusion

So what is the mathematical difference between adjusting a DC, and granting advantage/disadvantage?

Adjusting DCs

Adjusting DCs is a clean, linear way to increase or decrease a PC’s chances at success. Each step along the standard DCs goes up or down by 15%, and your chances increase by that 15% across the board, regardless of any other bonuses you have.

A change in DC represents a change in difficulty that benefits or hinders everyone equally, regardless of skill.

Advantage & Disadvantage

Fundamentally, advantage & disadvantage aren’t fair — Advantage helps players with high stats far more than it helps players with low stats, and disadvantage hinders players with low stats far more than it hinders players with high stats.

Advantage & disadvantage are a poor representation of repeatable skill, and come much closer to representing situational edge or obstacle, where a PCs ability to capitalise on that edge, or work around that obstacle is tempered by their pre-existing stat bonuses.

How can we use this?

There’s been a fair amount of discussion about when to use advantage/disadvantage and when to adjust DCs up or down a step, and I think this really helps give some guidance on this. The way I interpret the maths here is as follows:

• Pre-existing skill or some innate competence is best represented by adjusting DCs.
• Situational leverage, or a circumstantial leg-up that still requires skill to capitalise on is best represented by advantage.