[Psyche] How to think like a Bayesian: it pays to be able to think clearly about probabilities.

with a guest appearance by James Harden and a relevant example pertaining to DEI

How to think like a Bayesian
In a world of few absolutes, it pays to be able to think clearly about probabilities. These five ideas will get you started

https://psyche.co/guides/how-to-think-like-a-bayesian-and-make-better-decisions

excerpt:

Bayesians work a lot with conditional probabilities. Conditional probability arises when you consider how common some trait is among a subgroup of the population, instead of considering the population as a whole. If you pick a random American, they’re very unlikely to enjoy pizza made with an unleavened crust, topped with Provel cheese, and cut into squares. But conditional on the assumption that they grew up in St Louis, the probability that they’ll enjoy such a monstrosity is much higher.

Conditional probabilities can behave quite counterintuitively. Simple principles that one would think should be obvious can fail in spectacular fashion. The clearest example of this is Simpson’s Paradox.

Hopefully all of us have learned in our lives not to draw broad generalisations from a single example, or to assume that a small group is representative of the whole. A foreigner who judged American pizza preferences by visiting only St Louis would be seriously misled. By carelessness or sheer bad luck, we can stumble into a subpopulation that is unlike the others, and so bears traits that aren’t reflected by the population in general.

But Simpson’s Paradox demonstrates something much weirder than that: sometimes every subpopulation of a group has a particular trait, but that trait still isn’t displayed by the group as a whole.

In the 2016-17 NBA season, James Harden (then of the Houston Rockets) made a higher percentage of his two-point shot attempts than DeMar DeRozan (of the Toronto Raptors) made of his two-point shots. Harden also sank a higher percent of his three-point attempts than DeRozan. Yet DeRozan’s overall field-goal percentage – the percent of two-pointers and three-pointers combined that he managed to sink – was higher than Harden’s. Harden did better on both two-pointers and three-pointers, and those are the only kinds of shots that factor into the field-goal percentage, yet DeRozan was better overall. How is that possible?

Pro hoops aficionados will know that, for any player, two-point shots are easier to hit than three-pointers, yet Harden stubbornly insists on making things difficult for himself. In the 2016-17 season, he attempted almost the same number of each kind of shot (777 three-pointers versus 756 two-pointers), while DeRozan attempted more than 10 times as many two-pointers as three-pointers. Even though Harden was better at each kind of shot, DeRozan made the strategic decision to take high-percentage shots much more often than low-percentage ones. So, he succeeded at an overall higher rate.

The same phenomenon appeared when graduate departments at the University of California, Berkeley were investigated for gender bias in the 1970s. In 1973, 44 percent of male applicants were admitted to Berkeley’s graduate school, while only 35 per cent of female applicants succeeded. Yet a statistical study found that individual departments (which actually made the admissions decisions) were letting in men and women at roughly equal rates, or even admitting women more often. The trouble was that some departments were much more difficult than others to get into (for all applicants!), and women were applying disproportionately to more selective fields.

Of course, that doesn’t eliminate all possibilities of bias; a study found that women were applying to more crowded fields because they weren’t given the undergraduate mathematical background to study subjects that were better-funded (and therefore could admit more students). But the broader point about conditional probabilities stands: you can’t assume that an overall population reflects trends in its subpopulations, even if those trends occur in all the subpopulations. You also have to consider the distribution of traits across subpopulations.
​

more at the link