Broad men and narrow women: the perils of soundbite science

Last week a few people asked me what I made of a new study that was generating some interest on social media. At the time I hadn’t read it: I only knew Nature had reported it under a headline–‘Male researchers’ “vague” language more likely to win grants’–that made it sound both baffling (why would scientists get points for being vague?) and infuriating (as usual, it seemed to be men who were benefiting and women who were losing out). So I decided to investigate further, and then share my conclusions in this post.

The study was conducted by researchers at the US National Bureau of Economic Research (NBER), and their write-up is available as an NBER Working Paper. The data they analysed consisted of 6794 grant applications submitted to the Bill and Melinda Gates Foundation, which operates a policy of anonymous reviewing. Because reviewers weren’t told whether applicants were men or women, the researchers assumed that any gender differences in success rates could not be the result of direct discrimination. Whatever was leading reviewers to favour men must be contained in the application itself. And since most of a grant application consists of words, they decided to look for gender-differentiated patterns of word-use.

What their analysis revealed was a tendency for reviewers to give higher scores to applications that contained ‘broad’ words and lower scores to those that used ‘narrow’ words. Since broad words were used more frequently in men’s proposals, while narrow words appeared more often in women’s, this preference for broad over narrow words was also a preference for male- over female-authored applications. The researchers found no reason to think that broad words were associated with better proposals. When they looked at what applicants had gone on to achieve, the words used in their proposals appeared to be a poor predictor of research quality. Overall, then, the study’s conclusion was as infuriating as the Nature headline suggested: men whose research was objectively no better than women’s were receiving more funding from the Gates Foundation because reviewers preferred a particular style of grant writing.

One question the researchers didn’t attempt to answer was why men and women writing grant proposals might favour, respectively, ‘broad’ and ‘narrow’ words. But many people who commented on their findings thought the answer was obvious: simply and bluntly put, men–or at least a higher proportion of men–are bullshitters. Whereas women offer specific, realistic accounts of what they think their research can deliver, men have fewer inhibitions about making sweeping, grandiose claims.

This take is an example of a common interpretive strategy. If you present people with a generalization about language and gender—especially one whose significance isn’t immediately obvious—they will often try to make sense of it by invoking some other, more generic gender stereotype. In this case what they did was map the alleged linguistic difference (‘men use broad words, women use narrow words’) onto a higher-level, more familiar male-female opposition: ‘men are over-confident, women are over-cautious’.

You might ask: what’s wrong with that? Stereotypes aren’t always false: there’s plenty of other research you could cite in support of the thesis that men are over-confident (for instance, experimental studies showing that male test-takers consistently overestimate how well they’ve done, or the fact that men are more likely than women to apply for jobs when they don’t meet the advertised criteria). I don’t dispute any of that: in fact, I agree that ‘men are over-confident and women are over-cautious’ captures a real and significant cultural tendency. But there are, nevertheless, some problems with using it to explain the findings of this study.

One general problem is that you can use the same interpretive strategy to explain pretty much any set of findings, including made up ones. Suppose I told you the study had found that men use narrow words and women use broad words (i.e., the opposite of what it actually found). You’d be able to come up with an equally plausible explanation for that (non) finding just by switching to a different gender stereotype. Instead of ‘men use broad words because they’re overconfident bullshitters’ you might suggest that ‘women use broad words because they’re more attuned to their readers’ needs’; or ‘men use narrow words to show off their expert knowledge’. Since the supply of gender stereotypes is inexhaustible, there’s no statement of the form ‘men do x and women do y’ that can’t be slotted into this explanatory frame.

In the case of the NBER study, though, there’s a more specific problem with explaining men’s use of broad words as a linguistic manifestation of their over-confidence. When the researchers use the terms ‘broad’ and ‘narrow’, they don’t mean what people have assumed they mean (i.e., what the words would mean in ordinary English).

By way of illustration, here’s a list of six words taken from the study: three of them were classified as ‘broad’ and the other three as ‘narrow’. Which do you think are which?

  1. bacteria
  2. brain
  3. community
  4. detection
  5. health
  6. therapy

My guess is that you defined words as ‘broad’ if they were just basic, everyday vocabulary, and ‘narrow’ if they were a bit more abstract and technical. On that basis you probably categorised ‘health, ‘brain’ and ‘community’ as broad and ‘bacteria’, ‘detection’ and ‘therapy’ as narrow. That wasn’t, however, what the researchers did. Their definition wasn’t based on the characteristics of the words themselves, but on their frequency and distribution in the sample. Broad words were those that occurred in proposals on a wide range of different research topics; narrow words were restricted to proposals on a particular topic. By those criteria, ‘bacteria’, ‘detection’ and ‘therapy’ were broad, whereas ‘brain, community’ and ‘health’ were narrow.

If you think these definitions are confusing, I agree: the researchers might have done better to choose a different pair of terms (like, say, ‘core words’ and ‘peripheral words’). But once you’ve understood how they made their broad/narrow distinction and looked at the words in each category, it becomes difficult to argue that what’s behind the gender difference is men’s propensity for writing grandiose bullshit and women’s dogged attention to detail. (Is ‘health’ more precise than ‘bacteria’? Is ‘therapy’ vaguer than ‘brain’, or more grandiose than ‘community’?)

The fact that so much discussion revolved around the question of explanation suggests that most people had simply accepted the findings themselves at face value. This always bothers me: in my view, any claim that men use language in one way and women use it in another should be approached with a degree of scepticism. And that’s especially true if what you’re basing your assessment on is a report in the media. For obvious reasons, the media pay most attention to studies whose findings will make an eye-catching headline or a killer soundbite; this means they have a bias towards research which makes bold rather than cautious claims (stories like ‘men and women fairly similar, study shows’, or ‘we looked, but we didn’t find anything’, are not exactly clickbait). But for feminist sceptics it’s always worth asking whether the finding everyone’s talking about is supported by any other evidence. Have other researchers found the same thing? Or have they asked similar questions and come up with different answers?

There is, in fact, other research investigating the influence of writing style on grant decisions. Earlier this year, the Journal of Language and Social Psychology published an analysis of the language used in a sample of nearly 20,000 abstracts taken from research proposals submitted to the US National Science Foundation. This study considered only successful applications, taking the amount of funding applicants had been awarded as a measure of how positively their proposals had been assessed. It found there was a relationship between the funding researchers received and the language used in their proposal abstracts, but the linguistic features which made a difference were not the same ones the NBER study identified. The NSF gave more money to applicants whose abstracts were longer than average, contained fewer common words, and were written with ‘more verbal certainty’.

But I’m not just lamenting the uncritical reception of the NBER findings on general scientific principles. It also bothers me because I know how easy it is to propagate myths about the way men and women use language. ‘Men use broad words and women use narrow words’ is exactly the sort of thing that gets mythologized–detached from its original context (a study in which, as I’ve already pointed out, it meant something completely different from what most people thought) and repeated without elaboration in dozens of other sources, until eventually it turns into one of those zombie facts–like ‘Eskimos have a lot of words for snow’, or ‘women utter three times as many words per day as men’–that refuse to die no matter how many times they’re debunked.

If it does become part of our collective folk-wisdom on this subject, there’s every chance that ‘men use broad words, women use narrow words’ will also be filtered through the kind of deficit thinking which sees whatever women do with language as a problem in need of remedial intervention. Using ‘narrow’ words could join over-apologizing, hedging and tilting your head on the list of bad habits which are said to hold women back, and which it then becomes women’s responsibility to fix. (I can already imagine the TED talks exhorting women to ‘think broad’, and the workshops for female grant applicants on ‘choosing the right words’.)

To be fair to the authors of the NBER study, that isn’t what they think should happen. As they see it, it’s the reviewers who need training: their bias towards certain ways of writing elevates style over substance and leads to less than optimal funding decisions. But it’s hard for researchers to control what people make of, or what they do with,  findings that have entered the public domain. Even a study that was intended to be part of the solution can end up becoming part of the problem.

This is a dilemma for everyone who researches or writes about language and gender, myself included. Whenever I criticise some questionable claim or mistaken belief, I’m aware that I could be amplifying it just by giving it airtime. Though I’m only repeating it to explain the arguments against it, those arguments won’t necessarily be what people take away. But as you’ll have noticed, that hasn’t caused me to retreat into silence. I do believe that knowledge can set us free–but only if we’re willing to interrogate it critically.

 

Advertisements

A memo to my co-workers

To: All staff

From: Jane Demure

Here at Words, Inc.—an acknowledged global leader in the verbal communication industry—we value diversity and strive to be inclusive. That goes without saying, which is why we say it so often. But some of us feel excluded by a left-leaning culture which only values certain points of view, while deeming others illegitimate. Important issues are not being addressed because people are afraid to speak openly about them. We should not be intimidated or shamed into silence by groupthink and political correctness. Honest discussion with those who disagree can highlight our blind spots and help us grow, which is why I wrote this document.

In the field of verbal communication we are regularly told that men have not achieved parity with women because of bias, stereotyping and rigid gender norms. I’m willing to believe there’s some truth in that, but it is far from being the whole story.

On average, men and women differ biologically in numerous ways. These differences are not just socially constructed: they are universal across human cultures, can often be related to the effects of prenatal testosterone, and are exactly what we would predict from the perspective of evolutionary psychology.

Of course, we must remember we’re dealing with averages here. I’m not suggesting that all male individuals exhibit the same characteristics, or denying there’s overlap between men and women.  But at the level of whole populations, their preferences and abilities do differ, in part due to biological causes, and these differences may explain why men are basically rubbish at verbal communication under-represented at Words, Inc.

On average, women are more verbal than men: they talk more, and attach more importance to talking. This is exactly what an evolutionary psychologist would expect, since it reflects what must have been the case while our species was evolving many thousands of years ago. Early human females had plenty of time to gossip with each other as they foraged, whereas men, the hunters, needed to stalk their prey in silence. We modern humans have inherited our prehistoric ancestors’ genes: our lifestyles may be different from theirs, but human nature doesn’t change.

Women favour (on average, of course) a co-operative, rapport-building style of verbal interaction, one which puts group consensus and conflict-avoidance above the pursuit of individual advantage.  Men’s preferred style is more aggressive, competitive and status oriented. This is why a lot of men are not very good at listening, and why they enjoy shouty arguments more than women.

All employees at Words, Inc. have highly developed verbal skills—this world-leading company does not hire the merely average—but we still need to recognise that women have a natural advantage. A large body of research confirms that girls and women on average outperform boys and men on a range of measures of verbal ability. The difference may only be very slight (some experts suggest it’s equivalent to about one tenth of one standard deviation), but when you’re a world-leading verbal communication company you can’t afford to overlook anything that might give you a competitive edge. Lowering the bar so we can hire or promote more men is bad for business, and also demoralising for the company’s female majority.

If we take proper account of all the relevant data, we will surely have to acknowledge that the marginal position occupied by men at Words, Inc. has very little, if anything, to do with anti-male bias. Apart from the fact that more women than men meet the company’s exacting standards, there’s also the question of their differing preferences and life-goals.  A lot of men just don’t want to work with words all day: it’s not where their talents and their interests lie. Of course there are some men who are capable communicators, but even they might well think that it’s easier and less stressful to do something more traditionally male, be it writing computer code or biting the heads off chickens.

Unfortunately, we as a community have allowed our left-leaning biases to cloud our thinking on this issue. In addition to the affinity we feel for underdogs in general, there’s a strong tendency among humans (especially female ones, whose brains tend to be wired more for empathy than logical thinking) to defer to men’s wishes, protect their feelings and overestimate their accomplishments. This tendency most likely evolved because men have a lot of testosterone in their system, and they’re apt to beat the shit out of anyone they think is disrespecting them.

I know some of you will find these observations distasteful or even shocking, but I’m not going to apologise for using the B-word, nor for paying attention to what research has revealed about the unalterable biological differences between the sexes. On average, men just aren’t as good as women at the kinds of things we do here—nor as passionate about the work. Maybe the reason they’re not getting ahead is because they aren’t driven to put in the long hours and the unstinting effort which success at this company requires. No one should blame men for having other priorities—that’s only natural—but nor should the rest of us be blamed for discriminating against them when it’s actually all about their own aptitudes and choices.

Another thing I’m not going to apologise for is circulating this memo to thousands of my co-workers, some of whom will inevitably be men. If you’re a man reading this, I’d just like to remind you that you shouldn’t take it personally, because that would be completely unreasonable. I recognise that some of you may have experienced discrimination in the past: ten years ago, the women who worked at Words, Inc. didn’t even pretend to think that most men were anything but inarticulate gibbons. But today we are far more enlightened. We even force everyone to spend at least two hours a year examining their unconscious biases in diversity training workshops.

That’s a good thing, of course (I value diversity, did I mention that?) but it’s possible to take it too far. Valuing diversity shouldn’t mean abandoning the principles of meritocracy, free speech and common sense. And it certainly shouldn’t mean ignoring the insights offered by science, which deals in objective facts rather than prejudices or feelings. Once we acknowledge that not all differences are socially constructed or due to discrimination, we open our eyes to a more accurate view of the human condition which is necessary if we want to solve problems.

I know a lot of people here at Words, Inc., agree with me, they just don’t dare to say so. I too have felt the fear, but I’ve decided to do it anyway. If it all goes pear-shaped, at least I’ll be remembered as someone who broke out of the ideological echo-chamber and had the courage to stand up for her unfashionable beliefs. [Presses Send and thinks of Galileo]

_________________________________________________

Note: this post was inspired by (and in places is directly lifted from) a memo sent to his co-workers by the (now ex-) Google employee James Damore, the text of which is available here.  All the claims made in the post about sex-differences in linguistic behaviour are taken from the published (and sometimes peer-reviewed) work of real scientists, though the phrasing is my own. However, the fact that I repeat certain claims for the purposes of satire should not be taken to mean that I endorse them. (If you want to know what I really think about this body of work, this article lays it out: it also contains references for the sources I’ve used here without attribution. There’s also a (shorter and less ‘academic’) discussion in my book The Myth of Mars and Venus.)

The Google memo has prompted many non-satirical responses: among those I’ve read, the ones I’ve found most enlightening are this piece by the physicist Chanda Prescod-Weinstein, and this one by the computer scientist Cynthia Lee.

Sexism on the brain

In 2008, a group of researchers at Yale did an experiment which they wrote up and published under the title ‘The seductive allure of neuroscience explanations’.  Their subjects, a mixture of experts and non-experts, were presented with descriptions of psychological phenomena, each followed by an explanation, and asked to indicate how satisfying they found the explanation. The researchers had included both good and bad explanations, and in half the explanations of each type they had also inserted a logically irrelevant reference to neuroscience. The experts were not impressed, but the non-experts were: they rated explanations with neuroscience more convincing than those without. The effect was particularly striking with bad explanations, where the inclusion of irrelevant information about the brain seemed to stop people from noticing quite basic logical flaws.

The ‘seductive allure’ of neuroscience has been harnessed for all kinds of purposes by all kinds of people. For every scientist doing her best to communicate the complexity of contemporary brain research, there are a hundred non-scientists—self-help gurus, life-coaches, marketing consultants—churning out what has been labelled ‘neurobollocks’, a species of discourse that purports to be scientific, but is actually, in the words of one article on the subject, ‘self-help books dressed up in a lab coat’.

One flourishing branch of neurobollocks (dubbed ‘neurosexism’ by the psychologist Cordelia Fine) deals with the perennially popular topic of differences between men and women, which it explains with reference to the idea that human brains come in two distinct varieties, male and female. According to this literature, it’s because they have female brains that most women are rubbish at parking and maths, but great at multi-tasking (unless, presumably, they’re attempting to park a car while solving equations). And it’s also because they have female brains that women are better with language than men.

The language connection explains why over the years I have felt obliged to read such classics of neurosexism as Why Men Don’t Iron, which proclaimed on its cover in 1999 that ‘men’s brains are built for action and women’s for talking: men do, women communicate’; and The Female Brain, a bestseller in 2006, whose author was so convinced that women’s brains are built for talking, she reproduced the invented statistic that men on average utter 7000 words a day whereas women on average utter 20,000.  (As I explained in an earlier post, real research shows that women don’t talk more than men: where there’s a difference, it usually goes in the other direction.)

But the King and Queen of brain-sex bollocks are the husband-and-wife team Allan and Barbara Pease, authors of a series of advice books for heterosexual couples with titles like Why Men Don’t Listen and Women Can’t Read Maps. The Peases specialize in taking familiar self-help platitudes and mashing them up with exactly the kind of irrelevant (and in their case usually garbled) neuroscience which the Yale researchers used in their experiment.

In Pease-world you encounter some quite remarkable claims. At one point in Why Men Don’t Listen they inform readers that

When a male talks, MRI scans show that his entire left hemisphere becomes active as it searches to find a centre for speaking, but is unable to find much. Consequently, men aren’t much good at talking.

Women, on the other hand, are endowed with conversational superpowers:

With a greater flow of information between left and right hemispheres and specific brain locations for speech, most women can talk about several subjects simultaneously—sometimes in a single sentence.

At a guess, what’s being referenced here is the (much disputed) claim that language functions are more strongly lateralized (i.e. concentrated in a single hemisphere, typically the left one) in male brains than in female ones, along with the (even more disputed) claim that the corpus callosum, a structure which connects the two hemispheres, is larger in female brains than male ones. This is supposed to explain why females have more advanced verbal skills than males. (Or as the Peases prefer to put it, with impeccable scientific rigour, why ‘men aren’t much good at talking’.)

But I said ‘at a guess’ because the Peases’ garbled rendition of whatever research findings they might be alluding to bears little resemblance to any kind of science. Taken literally, the statements I’ve quoted are so ludicrous, they make me laugh every time I read them—until I remember that there are US states where this kind of weapons-grade bollocks has provided a rationale for teaching boys and girls separately, using methods that supposedly suit their brains (like having girls share their feelings about the laws of physics, and not expecting boys to read much literature). In Britain, too, I’ve met school teachers who’ve encountered a less extreme version of the same ideas in professional development courses—and who often didn’t realize there was anything scientifically questionable about them, since they’re always presented as the real deal.

But researchers are now challenging not only the nonsense peddled by popular writers like the Peases, but the scientific consensus it is (however loosely) based on. At the end of last month, the mainstream media were full of headlines like ‘Scans prove there’s no such thing as a “male” or “female” brain’ and ‘Men are from Mars, women are from Venus? New brain study says not’.

What occasioned these headlines was a research study which looked at a large number of structural features on MRI scans of over 1400 people’s brains, and found that only a small minority of those brains displayed consistently ‘male’ or ‘female’ characteristics. The majority were a mixture: they showed some of the characteristics previous research has associated more with male than female subjects, and some of the characteristics that previous research has associated more with female than male subjects. The conclusion the researchers drew was that if you examine the brain as a whole, there aren’t two distinct types that could sensibly be described as ‘male’ and ‘female’.

If this eventually becomes the consensus among scientists, what will become of the people who have made lucrative careers out of describing brains as ‘male’ and ‘female’? Will they feel obliged to admit they got it wrong, and either change their approach or shut up shop?

Maybe they should, but I very much doubt they will, because this is not the kind of popular science that’s written for laypeople with an interest in science. As the article quoted earlier observes, it’s more like self-help in a lab coat. Rather than starting from current debates in neuroscience, writers begin with familiar gender stereotypes (things like ‘men don’t listen’ and ‘women talk all the time’), and then cherry-pick a few studies whose results appear to support the argument they want to make (that these behaviours are ‘hard-wired’ in the brain).

It’s often assumed that the reason writers do this is because they are hacks who just don’t understand the science. But while some, like the Peases, may be genuinely clueless, many if not most of the books in my collection were written by people who have no such excuse. The Female Brain, for instance, is the work of a practising neuropsychiatrist, who presumably knows her way around an MRI scan.

But what these writers also know is their audience. Readers who buy books with titles like Why Men Don’t Listen and Women Can’t Read Maps are not looking for a nuanced, scientific discussion of sex and gender. They’re looking for a story that confirms their beliefs about how men and women are different, and reassures them that men and women will always be different no matter how much feminists shout and scream. It’s not about the science, it’s about the politics.

At the moment, the story these readers find most compelling comes from neuroscience, but in the past many other scientific stories were pressed into service to explain why men and women were both different and unequal. Genetics, evolutionary science, neurology, endocrinology and psychoanalysis have all had their moments—and let’s not forget gynaecology, whose practitioners warned women in the 19th century that stressful activities like higher education and public speaking would cause their reproductive organs to shrivel.

Every generation of scientific sexists disclaims the errors and biases of its predecessors and assures us that today’s science is different. Yet in one fundamental respect it isn’t different at all: contemporary scientists may be offering a new explanation for sex-differences, but the differences they’re trying to explain are the same old collection of stereotypes and myths. Occasionally one of these does fade into obsolescence (no one today suggests that education shrivels the ovaries); but many are in the category of ‘zombie facts’ which have been around forever (sometimes they’re older than science itself), have never been supported by good evidence, and still refuse to die.

The belief that women are the ‘more verbal’ sex is a case in point. Every time I encounter yet another discussion of what neuroscience might have to tell us about this (and such discussions appear in the scholarly literature as well as the popular bollocks), I feel as if I’m reading an account of how unicorns evolved. How compelling I find the explanation is beside the point: there are no unicorns, and women don’t talk more than men.

That’s why I’m cautious about hailing the ‘no such thing as a male/female brain’ study as a great leap forward, politically as well as scientifically. I do think the findings of the study are interesting, and I’m glad to see research evidence casting doubt on the idea of brain-sex. But I don’t think that gets to the root of the problem. The beliefs that are most damaging to women are not beliefs about the brain as such, they’re beliefs about sex-specific abilities and behaviour (like ‘women are no good at maths’ or ‘men can’t express their feelings’) which at the moment are often justified by appealing to supposed facts about the brain. Those beliefs may be reinforced by ‘the seductive allure of neuroscience explanations’, but they existed long before those explanations became available, and they could survive if those explanations were discredited.

So, yes, it’s important for feminists to challenge neurosexism. But if we only focus on what’s wrong with the story it tells about the brain, we’re in danger of conceding too much to the story it presupposes about the way men and women think, feel and behave. Without that other story, the brain story would cease to serve any purpose. Because the core of neurosexism isn’t the neuro, it’s the sexism.