Yeah, I think definitely an argument in favor—though given how important letters are to the process, I'd think that one's probably outweighed by other factors, given that this'll likely show up even without letters. I'd probably be in favor of the GRE point, though. (Though I know that's controversial.)
There seems to be some tension in the thought that both Normies and Presties can be equally qualified, and that the quality of Presties is more legible, and I think this shows that the category 'Normie' at ambiguous at some points from 'students at Normie institutions' to 'students at Normie institutions who are equally qualified'. Our priors are presumably formed by taking 'students' as the relevant reference class and assigning a low probability to any given student being sufficiently qualified. You're right that if I don't get as many markers about the Normies, then I don't come to think they're qualified even if they are. But from my perspective, it will also be true that, of most students at Normie institutions about whom I have similar evidence, they are, in fact, less qualified. It's only when we use 'Normie' to select 'Normie who is equally qualified' that we notice any unfairness. It's important to note that Presties only have more legible markers of competence because they are, as a group, in fact more qualified than students at normie institutions generally, and I worry that claims that Presties and Normies are equally qualified obscures this fact, one which is important to know because it affects other evidence we can acquire about quality, and the likelihood of various interventions being successful. Saying Normies are in fact just as qualified as Presties sounds like saying there are $100 bills lying on the ground - since everyone competes for Presties, I should be able to easily find some Normies, gather more evidence until their quality is equally legible, and hire the best qualified ones, leading to my institution becoming more presigious over time. If this doesn't happen, this is some evidence that either the Normies aren't as qualified, or that this extra legibility cannot be gained by hiring committees.
Fair points! I absolutely think it's implausible that (at least in many cases) Normies and Presties are going to be equally qualified—as you say, there's plenty of selective and market pressures for differences to emerge, etc.
But the post was written under the hypothetical: supposing they ARE equally qualified, we get this sort of result. Clearly if they're not equally qualified, this illegibility-asymmetry will still be active—so it'll be an important aspect of the problem, even if differences in distribution are another important aspect. It'll be gnarly as hell to figure out how much of what cause is contributing, of course, which is why it's helpful to look at how extreme things can get even when we suppose equal qualifications.
(Moreover, I think having this sort of explantion on the table is useful when it's contests just how much of a difference in the distributions of qualifications there is—as is often contested in concrete cases.)
You get really dramatic versions of this kind of result in information cascade games. Imagine each person gets to flip each coin once, but also sees which coin every prior person has bet on. After a while, rationality requires that one do whatever everyone else has done whatever the result of the flip. What can one more data point do to what is, in expectation, a huge body of evidence behind the group choice.
Interesting! Hadn't connected this to info cascades. Is the thought that this could explain Matthew, rich-get-richer effects?—"if everyone else has said this candidate is great, who am I to disagree?", which leads to the list of people saying positive things just getting longer.
One amendment: if I understand them, often the way information cascades work is really that the others' actions stop providing much evidence at all—everyone knows that everyone would follow the crowd regardless of what their signal was, so the fact that they follow the crowd doesn't provide more evidence. I think that's supposed to be part of why cascades can be fragile. This case might be a bit different. Although I guess your point is that rarely do we know that NO MATTER the private signal, people will follow the crowd—so it's still a little bit informative each time.
Here's my overarching question: how (if at all) do our obligations to remediate Bayesian injustice differ from our obligations to remediate other sorts of injustice?
More concrete example. You present a stylized case in which the whole admissions gap between Presties and Normies is driven by differences in legibility (they have the same underlying distribution of qualifications). Imagine another stylized case in which the whole gap is driven by differences in the underlying distribution of qualifications; while there's still uncertainty, they have the same degree of legibility. (In reality I think we've got both differences, and they interact--I'll explain below--but bear with me).
So in this other case, what should we do when we find ourselves admitting more Presties? Some possibilities:
1. Nothing. Admissions gaps driven by differences in the underlying distribution of qualifications aren't unfair.
2. Level down. Instead of picking the candidates we think are most qualified, pick some threshold of estimated candidate quality, and then have a lottery among the candidates above that threshold. (You see people arguing for stuff like this in college admissions.)
3. Level up. Do stuff to increase the qualifications of the Normies. (Easy to think summer philosophy programs like PIKSI are in part aimed at this kind of thing, though maybe hard to distinguish attempting to increase Normie qualifications from attempting to increase Normie legibility.)
Which of the three responses do you think are better or worse motivated depending on what is driving the admissions gaps? Clearly you don't like 1 when the gap is driven by differences in legibility, but might you go with 1 when it's driven by differences in the underlying distribution of qualifications? (If so, I do think the distinction requires explanation--I think drawing that distinction is probably only attractive if you think leveling up/down are better motivated in the legibility case.)
For 2, do you think there's any weaker a case for leveling down in response to differences in the underlying qualifications than in response to differences in legibility? I don't really see the difference myself. Though maybe if you think the signals driving differences in legibility are also really weak (but still somewhat informative) signals of quality, maybe that's a reason why you can level down without losing out much when the gap is driven by differences in legibility. Whereas if the gap is driven by substantial differences in average underlying ability, leveling down will require a much bigger sacrifice in terms of average candidate quality. So is that what you think? The signals driving differences in legibility are also really weak signals of quality, so we can level down without sacrificing much?
Onto leveling up. You might think leveling up is just much more feasible when the gaps are driven by differences in legibility. But I'm not really sure about that. I think a big part of what drives the legibility difference is the difference in the underlying distribution of abilities. Candidates from Presties who stand out stand out in a stronger field. When a letter says someone is the strongest undergrad they've taught in X years, and you trust them, that means a lot more when they've taught a lot of very impressive undergrads in X years, and on average that will differ between letter writers from Presties and from Normies. Likewise with winning thesis prizes, and stuff like that.
And if that's a big part of what's driving the legibility gap, then leveling up is gonna be really hard--the extra informative signals that Prestie letter writers can send aren't plausibly something you could counteract just by looking closer at writing samples from Normies. (In my judgment, at least.)
Thanks for the thoughts! Sorry to be slow. These are great questions and I'd hoped to have more time to think about them this week, but alas.
You're pressing on something that we hadn't thought too much about, and it looks like without good reason. I think, at a first pass, we were mostly driven by dialectical issues related to (1), thinking that it's much harder to make the case for leveling up or leveling down to the skeptics if the different is driven by underlying distributions. There's some merit to that, I think: even though different distributions will still make it so that Presties/Normies at the same level of qualifications get treated differently, these equal-levels will be less common—so insofar as the commonality of the unfair treatment is part of what makes it an injustice, it'll be less common (or better(?): a smaller proportion of people in the disadvantaged group will be subject to it) and so less pressing.
Still, it's still clearly unfair to the Normies. I suppose we think leveling up is easier in the legibility case, even if it's not really doable to completely get rid of the problem. Boven's 2016 paper on affirmative actions at shortlisting stage is I think a good explanation of why: if we're in a position to get lots more information about a subset of candidates, then it makes sense to shortlist those we have most uncertainty about—which often will be the illegible candidates and so will be a form of leveling up.
More broadly, we're thinking that one reason these debates can be intractable is that people don't have the possibility of this sort of illegibility as an explanation on the table. Group 1 looks at admissions processes and says they're biased; group 2 says otherwise. Group 1 hears that as saying that the disadvantaged groups are less qualified, and takes it to be a form of prejudice. Group 2 thinks group 1 isn't characterizing their position fairly, and thinks group 1 isn't sensitive to the hard truths about root causes of qualification-discrepancies.
In that rather toxic dialectic, we think illegibility as an explanation can be something that both sides can agree is happening and plays part of a role, and is the sort of thing that seems worth doing something about.
I think you're pressing us on whether those dispositions (that people will be more open to illegibility-style explanations) are well-founded. Hard question!
Here's a quick intuition pump to the effect that 1 isn't where the buck stops. Imagine a world where all the disparities between Presties and Normies are driven by differences in underlying ability, *but* bringing Normies up to the level of Presties would be really easy/cheap. I think the case for leveling up in that scenario--as opposed to resting context content with supposedly fair inequalities (since based on underlying ability differences)--would be very strong.
Contrast that with a world where all the differences are driven by differences in legibility, *but* we stipulate that there's no feasible way of mitigating these legibility differences. Then I think just accepting differences is all you can do.
So the idea that gaps due to differences in legibility are unfair, while gaps due to differences in avg underlying ability are not, really depends on the background assumption that legibility differences are much more mitigable than ability differences. Or so it seems to me.
For my own part, i agree that I'm the actual world legibility differences are *more" mitigable than underlying ability differences. But I'm not sure about *much* more, for the reasons I'm the previous comment.
You might well know this, but your argument is interestingly related to Luc Bovens' argument *in favour* of affirmative action at the shortlisting stage, which hinges precisely on those minority candidates being less legible: https://philarchive.org/rec/BOVSUU-2
Yeah, I love that argument! Very good point about how to counteract this sort of effect. Mulligan has an interesting reply to Bovens (https://philarchive.org/rec/MULUIH-2) which appeals to this sort of statistical discrimination effect, and I've never really got to the bottom of the back and forth or to what degree they actually disagree.
Thanks! Had come across that paper ages ago but couldn't find it in writing this actually; Lionel Page pointed it out to me. Wish we'd cited that one properly!
This sounds like an argument for not accepting letters of recommendation, and requiring GREs.
Yeah, I think definitely an argument in favor—though given how important letters are to the process, I'd think that one's probably outweighed by other factors, given that this'll likely show up even without letters. I'd probably be in favor of the GRE point, though. (Though I know that's controversial.)
There seems to be some tension in the thought that both Normies and Presties can be equally qualified, and that the quality of Presties is more legible, and I think this shows that the category 'Normie' at ambiguous at some points from 'students at Normie institutions' to 'students at Normie institutions who are equally qualified'. Our priors are presumably formed by taking 'students' as the relevant reference class and assigning a low probability to any given student being sufficiently qualified. You're right that if I don't get as many markers about the Normies, then I don't come to think they're qualified even if they are. But from my perspective, it will also be true that, of most students at Normie institutions about whom I have similar evidence, they are, in fact, less qualified. It's only when we use 'Normie' to select 'Normie who is equally qualified' that we notice any unfairness. It's important to note that Presties only have more legible markers of competence because they are, as a group, in fact more qualified than students at normie institutions generally, and I worry that claims that Presties and Normies are equally qualified obscures this fact, one which is important to know because it affects other evidence we can acquire about quality, and the likelihood of various interventions being successful. Saying Normies are in fact just as qualified as Presties sounds like saying there are $100 bills lying on the ground - since everyone competes for Presties, I should be able to easily find some Normies, gather more evidence until their quality is equally legible, and hire the best qualified ones, leading to my institution becoming more presigious over time. If this doesn't happen, this is some evidence that either the Normies aren't as qualified, or that this extra legibility cannot be gained by hiring committees.
Fair points! I absolutely think it's implausible that (at least in many cases) Normies and Presties are going to be equally qualified—as you say, there's plenty of selective and market pressures for differences to emerge, etc.
But the post was written under the hypothetical: supposing they ARE equally qualified, we get this sort of result. Clearly if they're not equally qualified, this illegibility-asymmetry will still be active—so it'll be an important aspect of the problem, even if differences in distribution are another important aspect. It'll be gnarly as hell to figure out how much of what cause is contributing, of course, which is why it's helpful to look at how extreme things can get even when we suppose equal qualifications.
(Moreover, I think having this sort of explantion on the table is useful when it's contests just how much of a difference in the distributions of qualifications there is—as is often contested in concrete cases.)
Hope that helps! Good points.
Nice post!
You get really dramatic versions of this kind of result in information cascade games. Imagine each person gets to flip each coin once, but also sees which coin every prior person has bet on. After a while, rationality requires that one do whatever everyone else has done whatever the result of the flip. What can one more data point do to what is, in expectation, a huge body of evidence behind the group choice.
http://veconlab.econ.virginia.edu/cas/cas.php#:~:text=Information%20Cascade%20Experiment%3A%20Introduction&text=Predictions%20are%20made%20in%20sequence,turns%20out%20to%20be%20correct.
Interesting! Hadn't connected this to info cascades. Is the thought that this could explain Matthew, rich-get-richer effects?—"if everyone else has said this candidate is great, who am I to disagree?", which leads to the list of people saying positive things just getting longer.
One amendment: if I understand them, often the way information cascades work is really that the others' actions stop providing much evidence at all—everyone knows that everyone would follow the crowd regardless of what their signal was, so the fact that they follow the crowd doesn't provide more evidence. I think that's supposed to be part of why cascades can be fragile. This case might be a bit different. Although I guess your point is that rarely do we know that NO MATTER the private signal, people will follow the crowd—so it's still a little bit informative each time.
Thanks!
Really nice post!
Here's my overarching question: how (if at all) do our obligations to remediate Bayesian injustice differ from our obligations to remediate other sorts of injustice?
More concrete example. You present a stylized case in which the whole admissions gap between Presties and Normies is driven by differences in legibility (they have the same underlying distribution of qualifications). Imagine another stylized case in which the whole gap is driven by differences in the underlying distribution of qualifications; while there's still uncertainty, they have the same degree of legibility. (In reality I think we've got both differences, and they interact--I'll explain below--but bear with me).
So in this other case, what should we do when we find ourselves admitting more Presties? Some possibilities:
1. Nothing. Admissions gaps driven by differences in the underlying distribution of qualifications aren't unfair.
2. Level down. Instead of picking the candidates we think are most qualified, pick some threshold of estimated candidate quality, and then have a lottery among the candidates above that threshold. (You see people arguing for stuff like this in college admissions.)
3. Level up. Do stuff to increase the qualifications of the Normies. (Easy to think summer philosophy programs like PIKSI are in part aimed at this kind of thing, though maybe hard to distinguish attempting to increase Normie qualifications from attempting to increase Normie legibility.)
Which of the three responses do you think are better or worse motivated depending on what is driving the admissions gaps? Clearly you don't like 1 when the gap is driven by differences in legibility, but might you go with 1 when it's driven by differences in the underlying distribution of qualifications? (If so, I do think the distinction requires explanation--I think drawing that distinction is probably only attractive if you think leveling up/down are better motivated in the legibility case.)
For 2, do you think there's any weaker a case for leveling down in response to differences in the underlying qualifications than in response to differences in legibility? I don't really see the difference myself. Though maybe if you think the signals driving differences in legibility are also really weak (but still somewhat informative) signals of quality, maybe that's a reason why you can level down without losing out much when the gap is driven by differences in legibility. Whereas if the gap is driven by substantial differences in average underlying ability, leveling down will require a much bigger sacrifice in terms of average candidate quality. So is that what you think? The signals driving differences in legibility are also really weak signals of quality, so we can level down without sacrificing much?
Onto leveling up. You might think leveling up is just much more feasible when the gaps are driven by differences in legibility. But I'm not really sure about that. I think a big part of what drives the legibility difference is the difference in the underlying distribution of abilities. Candidates from Presties who stand out stand out in a stronger field. When a letter says someone is the strongest undergrad they've taught in X years, and you trust them, that means a lot more when they've taught a lot of very impressive undergrads in X years, and on average that will differ between letter writers from Presties and from Normies. Likewise with winning thesis prizes, and stuff like that.
And if that's a big part of what's driving the legibility gap, then leveling up is gonna be really hard--the extra informative signals that Prestie letter writers can send aren't plausibly something you could counteract just by looking closer at writing samples from Normies. (In my judgment, at least.)
Anyway, again, great post!
Thanks for the thoughts! Sorry to be slow. These are great questions and I'd hoped to have more time to think about them this week, but alas.
You're pressing on something that we hadn't thought too much about, and it looks like without good reason. I think, at a first pass, we were mostly driven by dialectical issues related to (1), thinking that it's much harder to make the case for leveling up or leveling down to the skeptics if the different is driven by underlying distributions. There's some merit to that, I think: even though different distributions will still make it so that Presties/Normies at the same level of qualifications get treated differently, these equal-levels will be less common—so insofar as the commonality of the unfair treatment is part of what makes it an injustice, it'll be less common (or better(?): a smaller proportion of people in the disadvantaged group will be subject to it) and so less pressing.
Still, it's still clearly unfair to the Normies. I suppose we think leveling up is easier in the legibility case, even if it's not really doable to completely get rid of the problem. Boven's 2016 paper on affirmative actions at shortlisting stage is I think a good explanation of why: if we're in a position to get lots more information about a subset of candidates, then it makes sense to shortlist those we have most uncertainty about—which often will be the illegible candidates and so will be a form of leveling up.
More broadly, we're thinking that one reason these debates can be intractable is that people don't have the possibility of this sort of illegibility as an explanation on the table. Group 1 looks at admissions processes and says they're biased; group 2 says otherwise. Group 1 hears that as saying that the disadvantaged groups are less qualified, and takes it to be a form of prejudice. Group 2 thinks group 1 isn't characterizing their position fairly, and thinks group 1 isn't sensitive to the hard truths about root causes of qualification-discrepancies.
In that rather toxic dialectic, we think illegibility as an explanation can be something that both sides can agree is happening and plays part of a role, and is the sort of thing that seems worth doing something about.
I think you're pressing us on whether those dispositions (that people will be more open to illegibility-style explanations) are well-founded. Hard question!
Here's a quick intuition pump to the effect that 1 isn't where the buck stops. Imagine a world where all the disparities between Presties and Normies are driven by differences in underlying ability, *but* bringing Normies up to the level of Presties would be really easy/cheap. I think the case for leveling up in that scenario--as opposed to resting context content with supposedly fair inequalities (since based on underlying ability differences)--would be very strong.
Contrast that with a world where all the differences are driven by differences in legibility, *but* we stipulate that there's no feasible way of mitigating these legibility differences. Then I think just accepting differences is all you can do.
So the idea that gaps due to differences in legibility are unfair, while gaps due to differences in avg underlying ability are not, really depends on the background assumption that legibility differences are much more mitigable than ability differences. Or so it seems to me.
For my own part, i agree that I'm the actual world legibility differences are *more" mitigable than underlying ability differences. But I'm not sure about *much* more, for the reasons I'm the previous comment.
You might well know this, but your argument is interestingly related to Luc Bovens' argument *in favour* of affirmative action at the shortlisting stage, which hinges precisely on those minority candidates being less legible: https://philarchive.org/rec/BOVSUU-2
Yeah, I love that argument! Very good point about how to counteract this sort of effect. Mulligan has an interesting reply to Bovens (https://philarchive.org/rec/MULUIH-2) which appeals to this sort of statistical discrimination effect, and I've never really got to the bottom of the back and forth or to what degree they actually disagree.
I didn't know about the reply, so thanks, I'll check it out!
You probably know this, but I think this is called "screening discrimination" in the econ lit, e.g. https://www.journals.uchicago.edu/doi/abs/10.1086/262033
Thanks! Had come across that paper ages ago but couldn't find it in writing this actually; Lionel Page pointed it out to me. Wish we'd cited that one properly!