Testing people’s reactions to language

Linguistic corpora are wonderful things: vast catalogues of examples of the ways people use language. Well-compiled corpora are great academic research tools and they’re essential for informing dictionaries and other resources such as usage guides. These guides – at least, the sensible ones – pay attention to what people say and write to inform their recommendations.

But there’s another side to the story. Language is as much about receiving as transmitting, and while a corpus tells us what the speakers and writers are doing, it doesn’t cover the reactions of listeners and readers. A certain word or phrase or grammatical construction may be commonly used in a certain way by plenty of people, but that fact doesn’t tell us whether other people understand it consistently or find it off-putting.

To unscientifically survey

With this in mind, the other day I carried out a silly little straw poll doomed to have completely unrepresentative results. I asked people on Twitter to guess what percentage of adult native English speakers dislike split infinitives – a classic staple of popular grammar debates. I also asked people whether they personally were part of that group. There were 22 people kind enough to answer.

Five of the 22 said they did dislike split infinitives (23%). The other 17 (77%) said they didn’t, although three (14%) volunteered reservations about using them.

Only 18 people made a numerical guess about how common this dislike is. Guesses ranged from 0.00001% to 32% of the population, neither of which I took terribly seriously, with 11 of the 18 (61%) saying 5% or less and four (22%) saying more than 10%.

Among the comments people offered was Michael Regnier (whose writing I have the regular pleasure of copyediting) saying “5% of adult native English speakers have a reflex reaction against split infinitives; 0.05% actually ‘dislike’ them”. Whatever the numbers, this feels right: the number of passionate believers is far exceeded by the number of people with a vague sense of what they ‘ought’ to do but who aren’t really bothered. William Brett seemed to be a case in point, giving his attitude as “trained to dislike them but secretly don’t”.

I’m not going to get into an argument about split infinitives. My business here is different.

Being more rigorous

How can we properly find out about people’s reactions to split infinitives, or other (more interesting) aspects of usage?

One way would be doing a larger, professional survey along the lines of my straw poll: get a market research company to ask a representative sample of the public (or any target demographic) what they think about split infinitives. This has obvious limits: plenty of people won’t know what a split infinitive is. Others will say what they think they ‘ought’ to say rather than reporting their real attitudes.

A smarter way would be to look at what effect split infinitives have when people come across them. You could set up a test like this:

Give a group of people the task of rating hypothetical candidates for a job. Explain that the candidates’ skills, experience and so on are very evenly matched, so the choice has to be based on their personal statements. These would be short passages supposedly written by them. You’d try to make them roughly as good as each other, but some would contain a few split infinitives and some a few unsplit infinitives. Get each person to rate how good they think each candidate would be. You could also ask a follow-up question about how well-written each statement was.

You give a second group of people the same statements but with the split/unsplit infinitives reversed.

Then you take the average ratings across the two groups and see what difference, if any, tinkering with the infinitives has made.

A similar method would be to present arguments (on a not-too-controversial topic) containing split/unsplit infinitives and asking people how much they agree or disagree. Another would be for the text to be a factual description of something; once people have read it, you take it away and ask them questions to test how well they understood it. You could also time how long it takes people to finish reading, as a measure of ease of comprehension.

This sort of method could be used to test all sorts of aspects of language: the effect of spelling mistakes, sentence length, exclamation marks, technical terms, dangling modifiers, varying levels of formality, the passive voice, first vs third person, metaphors, expletives…

In conclusion, further research is needed

I’ve hardly seen any studies in this vein: for instance, this one found that concrete language is more convincing than abstract language. But they seem few and far between, so I guess it’s a bit of a niche area.

What we need is an eccentric billionaire willing to bankroll a load of these studies. Anyone?

Advertisements
Both comments and trackbacks are currently closed.

Comments

  • Jason Preater  On March 3, 2013 at 10:17 pm

    I think you need to tell some recuitment agency that this will be the way to properly check (sic) their candidates and then let them do the research for you.

  • Jonathon Owen  On March 4, 2013 at 5:24 am

    I’ve seen a few studies that measured readers’ reactions to various errors, but none that I’ve seen ever included split infinitives. It seems like a curious oversight.

  • CaitlinO  On March 6, 2013 at 1:44 pm

    Written language garners a very different reaction than spoken. I would be more interested in seeing the same experiment observing people’s reactions to applicants who spoke using controversial grammar, dialects, or accents.

%d bloggers like this: