There seems to be some tension in the thought that both Normies and Presties can be equally qualified, and that the quality of Presties is more legible, and I think this shows that the category 'Normie' at ambiguous at some points from 'students at Normie institutions' to 'students at Normie institutions who are equally qualified'. Our priors are presumably formed by taking 'students' as the relevant reference class and assigning a low probability to any given student being sufficiently qualified. You're right that if I don't get as many markers about the Normies, then I don't come to think they're qualified even if they are. But from my perspective, it will also be true that, of most students at Normie institutions about whom I have similar evidence, they are, in fact, less qualified. It's only when we use 'Normie' to select 'Normie who is equally qualified' that we notice any unfairness. It's important to note that Presties only have more legible markers of competence because they are, as a group, in fact more qualified than students at normie institutions generally, and I worry that claims that Presties and Normies are equally qualified obscures this fact, one which is important to know because it affects other evidence we can acquire about quality, and the likelihood of various interventions being successful. Saying Normies are in fact just as qualified as Presties sounds like saying there are $100 bills lying on the ground - since everyone competes for Presties, I should be able to easily find some Normies, gather more evidence until their quality is equally legible, and hire the best qualified ones, leading to my institution becoming more presigious over time. If this doesn't happen, this is some evidence that either the Normies aren't as qualified, or that this extra legibility cannot be gained by hiring committees.
You get really dramatic versions of this kind of result in information cascade games. Imagine each person gets to flip each coin once, but also sees which coin every prior person has bet on. After a while, rationality requires that one do whatever everyone else has done whatever the result of the flip. What can one more data point do to what is, in expectation, a huge body of evidence behind the group choice.
Here's my overarching question: how (if at all) do our obligations to remediate Bayesian injustice differ from our obligations to remediate other sorts of injustice?
More concrete example. You present a stylized case in which the whole admissions gap between Presties and Normies is driven by differences in legibility (they have the same underlying distribution of qualifications). Imagine another stylized case in which the whole gap is driven by differences in the underlying distribution of qualifications; while there's still uncertainty, they have the same degree of legibility. (In reality I think we've got both differences, and they interact--I'll explain below--but bear with me).
So in this other case, what should we do when we find ourselves admitting more Presties? Some possibilities:
1. Nothing. Admissions gaps driven by differences in the underlying distribution of qualifications aren't unfair.
2. Level down. Instead of picking the candidates we think are most qualified, pick some threshold of estimated candidate quality, and then have a lottery among the candidates above that threshold. (You see people arguing for stuff like this in college admissions.)
3. Level up. Do stuff to increase the qualifications of the Normies. (Easy to think summer philosophy programs like PIKSI are in part aimed at this kind of thing, though maybe hard to distinguish attempting to increase Normie qualifications from attempting to increase Normie legibility.)
Which of the three responses do you think are better or worse motivated depending on what is driving the admissions gaps? Clearly you don't like 1 when the gap is driven by differences in legibility, but might you go with 1 when it's driven by differences in the underlying distribution of qualifications? (If so, I do think the distinction requires explanation--I think drawing that distinction is probably only attractive if you think leveling up/down are better motivated in the legibility case.)
For 2, do you think there's any weaker a case for leveling down in response to differences in the underlying qualifications than in response to differences in legibility? I don't really see the difference myself. Though maybe if you think the signals driving differences in legibility are also really weak (but still somewhat informative) signals of quality, maybe that's a reason why you can level down without losing out much when the gap is driven by differences in legibility. Whereas if the gap is driven by substantial differences in average underlying ability, leveling down will require a much bigger sacrifice in terms of average candidate quality. So is that what you think? The signals driving differences in legibility are also really weak signals of quality, so we can level down without sacrificing much?
Onto leveling up. You might think leveling up is just much more feasible when the gaps are driven by differences in legibility. But I'm not really sure about that. I think a big part of what drives the legibility difference is the difference in the underlying distribution of abilities. Candidates from Presties who stand out stand out in a stronger field. When a letter says someone is the strongest undergrad they've taught in X years, and you trust them, that means a lot more when they've taught a lot of very impressive undergrads in X years, and on average that will differ between letter writers from Presties and from Normies. Likewise with winning thesis prizes, and stuff like that.
And if that's a big part of what's driving the legibility gap, then leveling up is gonna be really hard--the extra informative signals that Prestie letter writers can send aren't plausibly something you could counteract just by looking closer at writing samples from Normies. (In my judgment, at least.)
You might well know this, but your argument is interestingly related to Luc Bovens' argument *in favour* of affirmative action at the shortlisting stage, which hinges precisely on those minority candidates being less legible: https://philarchive.org/rec/BOVSUU-2
Thanks! Had come across that paper ages ago but couldn't find it in writing this actually; Lionel Page pointed it out to me. Wish we'd cited that one properly!
This sounds like an argument for not accepting letters of recommendation, and requiring GREs.
There seems to be some tension in the thought that both Normies and Presties can be equally qualified, and that the quality of Presties is more legible, and I think this shows that the category 'Normie' at ambiguous at some points from 'students at Normie institutions' to 'students at Normie institutions who are equally qualified'. Our priors are presumably formed by taking 'students' as the relevant reference class and assigning a low probability to any given student being sufficiently qualified. You're right that if I don't get as many markers about the Normies, then I don't come to think they're qualified even if they are. But from my perspective, it will also be true that, of most students at Normie institutions about whom I have similar evidence, they are, in fact, less qualified. It's only when we use 'Normie' to select 'Normie who is equally qualified' that we notice any unfairness. It's important to note that Presties only have more legible markers of competence because they are, as a group, in fact more qualified than students at normie institutions generally, and I worry that claims that Presties and Normies are equally qualified obscures this fact, one which is important to know because it affects other evidence we can acquire about quality, and the likelihood of various interventions being successful. Saying Normies are in fact just as qualified as Presties sounds like saying there are $100 bills lying on the ground - since everyone competes for Presties, I should be able to easily find some Normies, gather more evidence until their quality is equally legible, and hire the best qualified ones, leading to my institution becoming more presigious over time. If this doesn't happen, this is some evidence that either the Normies aren't as qualified, or that this extra legibility cannot be gained by hiring committees.
Nice post!
You get really dramatic versions of this kind of result in information cascade games. Imagine each person gets to flip each coin once, but also sees which coin every prior person has bet on. After a while, rationality requires that one do whatever everyone else has done whatever the result of the flip. What can one more data point do to what is, in expectation, a huge body of evidence behind the group choice.
http://veconlab.econ.virginia.edu/cas/cas.php#:~:text=Information%20Cascade%20Experiment%3A%20Introduction&text=Predictions%20are%20made%20in%20sequence,turns%20out%20to%20be%20correct.
Really nice post!
Here's my overarching question: how (if at all) do our obligations to remediate Bayesian injustice differ from our obligations to remediate other sorts of injustice?
More concrete example. You present a stylized case in which the whole admissions gap between Presties and Normies is driven by differences in legibility (they have the same underlying distribution of qualifications). Imagine another stylized case in which the whole gap is driven by differences in the underlying distribution of qualifications; while there's still uncertainty, they have the same degree of legibility. (In reality I think we've got both differences, and they interact--I'll explain below--but bear with me).
So in this other case, what should we do when we find ourselves admitting more Presties? Some possibilities:
1. Nothing. Admissions gaps driven by differences in the underlying distribution of qualifications aren't unfair.
2. Level down. Instead of picking the candidates we think are most qualified, pick some threshold of estimated candidate quality, and then have a lottery among the candidates above that threshold. (You see people arguing for stuff like this in college admissions.)
3. Level up. Do stuff to increase the qualifications of the Normies. (Easy to think summer philosophy programs like PIKSI are in part aimed at this kind of thing, though maybe hard to distinguish attempting to increase Normie qualifications from attempting to increase Normie legibility.)
Which of the three responses do you think are better or worse motivated depending on what is driving the admissions gaps? Clearly you don't like 1 when the gap is driven by differences in legibility, but might you go with 1 when it's driven by differences in the underlying distribution of qualifications? (If so, I do think the distinction requires explanation--I think drawing that distinction is probably only attractive if you think leveling up/down are better motivated in the legibility case.)
For 2, do you think there's any weaker a case for leveling down in response to differences in the underlying qualifications than in response to differences in legibility? I don't really see the difference myself. Though maybe if you think the signals driving differences in legibility are also really weak (but still somewhat informative) signals of quality, maybe that's a reason why you can level down without losing out much when the gap is driven by differences in legibility. Whereas if the gap is driven by substantial differences in average underlying ability, leveling down will require a much bigger sacrifice in terms of average candidate quality. So is that what you think? The signals driving differences in legibility are also really weak signals of quality, so we can level down without sacrificing much?
Onto leveling up. You might think leveling up is just much more feasible when the gaps are driven by differences in legibility. But I'm not really sure about that. I think a big part of what drives the legibility difference is the difference in the underlying distribution of abilities. Candidates from Presties who stand out stand out in a stronger field. When a letter says someone is the strongest undergrad they've taught in X years, and you trust them, that means a lot more when they've taught a lot of very impressive undergrads in X years, and on average that will differ between letter writers from Presties and from Normies. Likewise with winning thesis prizes, and stuff like that.
And if that's a big part of what's driving the legibility gap, then leveling up is gonna be really hard--the extra informative signals that Prestie letter writers can send aren't plausibly something you could counteract just by looking closer at writing samples from Normies. (In my judgment, at least.)
Anyway, again, great post!
You might well know this, but your argument is interestingly related to Luc Bovens' argument *in favour* of affirmative action at the shortlisting stage, which hinges precisely on those minority candidates being less legible: https://philarchive.org/rec/BOVSUU-2
You probably know this, but I think this is called "screening discrimination" in the econ lit, e.g. https://www.journals.uchicago.edu/doi/abs/10.1086/262033
Thanks! Had come across that paper ages ago but couldn't find it in writing this actually; Lionel Page pointed it out to me. Wish we'd cited that one properly!