A dinner-time phone call from a Tory pollster sparks some thoughts about the wording of political questions and the biases that appear between the words

I received a curious phone call last night. Actually, I received a Curia phone call last night; David Farrar's polling company wanted to know whether I'd spare five minutes to take part in a political survey. Never one to miss a chance to research the researchers, I naturally said yes.

I was asked the questions you'd expect – who I'd vote for if an election was held tomorrow, whether I could be convinced to change my mind, which issue was most important to me and who should be Prime Minister. I was asked which party I trusted most, had the best team behind the leader, and thought they had the best interests of the country at heart.

And they wanted to know about just two National ministers – John Key and Bill English. I had to find them either favourable or unfavourable (or very), there was no middle 'meh' option.

They were direct questions, requiring one word answers – yes or no, a party name, or a favorability ranking. I can understand the clarity that gives, and I'm sure over the years those yes or no answers to certain questions have become well-known weather vanes for trends or voters' willingness to be won over.

But when I was asked whether I thought New Zealand was going in the right or wrong direction, I had to stop myself replying, 'in what area?'

I was particularly intrigued by the wording used by my pollster.

I can't remember word for word, so I'll refrain from using full quote marks. But I remember my eye-brows going up to a few questions. Such as the one above about the best interests of New Zealand.

As I remember it, I wasn't asked which party I thought was best for New Zealand, but which party thought it was best for New Zealand, as if testing the parties' confidence levels rather than my views.

The pollster took me through a range of portfolio areas, asking which party I thought would do best in each one... Health, education, tax, foreign policy, welfare...

But the question changed subtly, but significantly for a couple, and in a way that seemed somewhat leading. I assume it was on purpose, and therefore am left wondering whether that doesn't undermine the use of the data.

On the environment, I was asked not which party would do best, but which was most balanced. Now, I can't remember the precise wording, but it was a question constructed differently from the others. I remember thinking, 'balanced is a word so heavily used by National that it points in that party's direction'....

What I do remember exactly was being asked which party would be "toughest on crime". Not which party would make us safer, or reduce crime or address the causes of crime. Which would be "toughest". As if being tough on crime equated to being the best party to deal with crime.

I'm someone whose heart sinks when I read that our prison population has topped 200 per 100,000. I see that as a failure of our country, not something to be proud of. I drive past that prison rearing over Auckland's southern motorway and wonder why all the money being spent there isn't going into education or innovation. I think the four prisons Labour built during its three terms represents a craven cop-out.

So I told the pollster which party I thought "toughest", but they learnt nothing about which party I'd support on law and order issues. Which presumably was what they really wanted to know.

The climax of the interview were questions about the two marquee policies announced by the major parties this year. It was Labour's Tax-free zone vs National's partial privatisation – interesting what most concerns DPF and his clients.

Again, the questions had a noticeable lean. Asking about Labour's policy, the pollster told me that critics say it's unaffordable while supporters say it can be funded by increasing taxes on the rich (a simplistic description, but no more than Labour deserves given its vague outline). However National's privatisation plans were proposed to simply 'pay off debt'. There was no talk of critics, such as those who complain at the loss of dividends or foreign encroachment, or indeed at other reasons to support the plan, such as strengthening our stock exchange and access to capital.

Now DPF can ask whatever questions he wants, but by asking elading questions, don't you get less accurate results? Aren't people more likely to tell you what you want to hear, rather than the views that will actually drive them when they're standing in the booth on election day?

David, feel free to respond. Or perhaps our own Rob Salmond might have thoughts - Rob? Anyway, thanks for the call, David!

 

 

Comments (22)

by Danyl Mclauchlan on February 15, 2011
Danyl Mclauchlan

Curia tried to poll my wife a while back - she wouldn't answer which was wise, I think. I wouldn't put it past DPF to be gradually buliding up a database of poll results obtained from political journalists.

by Rob Hosking on February 15, 2011
Rob Hosking

I was polled on Sunday -  by UMR.

I was dying to know hear their questions, but they said they wanted males aged 18-40.

Like a damn honest fool, I said '46' when they asked my age.

Still pissed off about it, actually.

 

 

by Richard on February 15, 2011
Richard

So I told the pollster which party I thought "toughest", but they learnt nothing about which party I'd support on law and order issues. Which presumably was what they really wanted to know.

I'm not certain that they do really want to know what party you support.

They just want to create a poll result which they can point to and say "the majority of people support National's position on law and order". Which will then be duly reported in the media without close analysis.

It's just part of a cynical attempt to create misinformation to seduce "undecided voters" into voting in a particular way.

by Raymond A Francis on February 15, 2011
Raymond A Francis

Danyl, I think your wife was wise to not answer but you have to be careful about paranoia, sometimes it is just coincidence

Do you have to answer the questions honestly, who knew, who does.

by The Falcon on February 15, 2011
The Falcon

I am familiar with Curia's polling so can hopefully help clear things up.

And they wanted to know about just two National ministers – John Key and Bill English. I had to find them either favourable or unfavourable (or very), there was no middle 'meh' option.

Generally Curia gives 5 options, ranging from very favourable to very unfavourable, with "neutral" in the middle. It is possible they have changed their questions recently, but more likely the pollster just couldn't be bothered reading out the middle option.

But when I was asked whether I thought New Zealand was going in the right or wrong direction, I had to stop myself replying, 'in what area?'

Polling companies can't take detailed comments from respondents, it's impractical. This question is likely designed to gauge whether NZ currently has a "positive mood" - not very scientific, but can be useful to know.

As I remember it, I wasn't asked which party I thought was best for New Zealand, but which party thought it was best for New Zealand, as if testing the parties' confidence levels rather than my views.

A huge number of people get confused about this question, which is surprising. I believe the question is "which party [do you believe] will do what's right for NZ?" But yes you're not the only one to find it unclear.

Generally, I find Curia's questions fairly balanced. As you mentioned, there is no incentive to write biased questions, because that defeats the purpose of polling.

Curia tried to poll my wife a while back - she wouldn't answer which was wise, I think. I wouldn't put it past DPF to be gradually buliding up a database of poll results obtained from political journalists.

I think that's unlikely. As far as I know, the poll results are kept confidential, and names are not matched to responses in any kind of database.

They just want to create a poll result which they can point to and say "the majority of people support National's position on law and order". Which will then be duly reported in the media without close analysis.

Nice conspiracy theory, but no. Curia's research is rarely published in the media.

by Danyl Mclauchlan on February 15, 2011
Danyl Mclauchlan

<i>I think that's unlikely. As far as I know, the poll results are kept confidential, and names are not matched to responses in any kind of database.</i>

I know DPF pretty well. I'm not saying he's doing this, but I wouldn't say it's unlikely.

by Richard on February 15, 2011
Richard

Curia's research is rarely published in the media.

Except when it is.

Which is when the results suit the organisation commisioning it.

by Andrew Geddis on February 15, 2011
Andrew Geddis

"I know DPF pretty well. I'm not saying he's doing this, but I wouldn't say it's unlikely."

I'd say this is very unlikely, for three reasons:

(1) It would be of limited use. So you find out that Journalist X is pro-Greens while Journalist Y is pro-National. Does this automatically mean one will be a more useful idiot than the other? Isn't there a separation between a journalist's professional  work and how they vote? And if there isn't, do you really need to poll them to discover their preferences (witness Fran O'Sullivan vis-a-vis Bernard Rudman)?

(2) I'm pretty sure it would be in breach of the Privacy Act ... but happy for others to opine more on that. (Paging Graeme Edgeler or Stephen Price ...)

(3) If it ever came out that DPF was using Curia to do so, I think it would be the end of his business. Which, political animal though he is, would be a high price to pay.

by Graeme Edgeler on February 15, 2011
Graeme Edgeler

(2) I'm pretty sure it would be in breach of the Privacy Act ... but happy for others to opine more on that. (Paging Graeme Edgeler or Steven Price ...)

I could construct an argument, but knowing that Curia, like other polling companies, has an opening spiel about how your answers are completely confidential, it wouldn't apply.

Information may only be retained for the purpose it was collected. If it was collected for the purposes of collating an anonymised poll result, it couldn't be retained or used for targetting journalists with sympathetic messages, etc.

by Phil Lyth on February 15, 2011
Phil Lyth

What Graeme said, and people have to know it is being collected. They have to know that they have the right to access it, and to correct any mistakes. And information must not be retained for longer than necessary for the purpose for which it is collected. Doesn't sound as though any of that was disclosed in the call.

Speaking as someone who has tangled more than once with the Privacy Act.

The Privacy Commissioner has a pretty good website, including this page: http://privacy.org.nz/information-privacy-principles

by Danyl Mclauchlan on February 15, 2011
Danyl Mclauchlan

Just to emphasise that I have no proof for my rather defamatory theory and could very easily be wrong. But the risk would be very low, and the information would be useful in countless ways (looking at hiring a new press secretary? which gallery journalists support our party?) as well as just being interesting for its own sake.

by Andrew Geddis on February 15, 2011
Andrew Geddis

"Just to emphasise that I have no proof for my rather defamatory theory ..."

That's OK ... DPF is highly unlikely to take action. Or, not without looking a bit hypocritical.

by Tim Watkin on February 15, 2011
Tim Watkin

Danyl, it was an interesting theory but I can put it to bed. They didn't ask my name, and so have no idea which were my answers. I thought about how much to reveal in this post, but figuring they tried hundreds last night, good luck to them guessing which response was mine.

And my trump card... I was actually at someone else's house (I won't say who to protect my anonymity), and they handed the phone to me. So there's absolutely no way of knowing who I was.

Falcon, thanks for the insight. They definitely didn't give the middle option in my phone call because I said, "there's no middle option?" and there wasn't. And I don't think the question about what's best for NZ was as clear as your quote, because (at risk of giving myself away) I answered "all of them", based on the idealistic assumption that they all think they're doing the best for the country.

by Tim Watkin on February 15, 2011
Tim Watkin

Rob, damn. It'd be fascinating to know what Labour wanted to know... Interesting that they're targeting males 18-40. I'm guessing that would be one of their weakest demographics.

by Rob Hosking on February 16, 2011
Rob Hosking

Colmar Brunton used to - and may well still do - ask screening questions about whether you or a close relative is involved in the political process.

Some years ago my mum was polled and she said no, not a member of a political party, not involved in the political process, but she had a son working in the press gallery.

That disqualified her.

Given refusal rates (reputed to be quite high), pollsters may now be a bit less picky.

 

by Bruce Thorpe on February 16, 2011
Bruce Thorpe

I know somebody who always lies about her age when asked to complete polls and surveys,  because  they never drop off the youngies, probably because most of them are mobile only.

by Danyl Mclauchlan on February 16, 2011
Danyl Mclauchlan

Danyl, it was an interesting theory but I can put it to bed. They didn't ask my name, and so have no idea which were my answers. I thought about how much to reveal in this post, but figuring they tried hundreds last night, good luck to them guessing which response was mine.

And my trump card... I was actually at someone else's house (I won't say who to protect my anonymity), and they handed the phone to me. So there's absolutely no way of knowing who I was.

Oh, you are so naive.

by Tim Watkin on February 16, 2011
Tim Watkin

Danyl, can they recognise my voice now? Or are those cameras in the street really following my every move? Heck, is that someone behind that bush over there?! That DPF is everywhere!

by Rob Hosking on February 16, 2011
Rob Hosking

Tim:  You can have my tinfoil hat, I'm done with it.

by on September 06, 2011
Anonymous
Info PR: 4 I: 15,600 L: error LD: error I: n/a Rank: 570214 Age: n/a I: 0 whois source Robo: yes Sitemap: no Rank: 976345 Price: 63 Density

/* Style Definitions */ table.MsoNormalTable {mso-style-name:普通表格; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin:0cm; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;}

moncler jackets is the famous brand which headquarters located in France Grenoble,We manufactured original and top quality moncler jacket.In addition, the moncler coats White inside filled with 90% white duck down and 10% feather,welcome to our moncler outlet ,all Moncler Sale with low price now.

by on September 28, 2011
Anonymous
by on March 07, 2012
Anonymous

Over the years, from a at most of the selected pursuits, it can inevitably see several real strength of the burberry mens bags .Then that shoe in that situation ought to improve their own individual innovative strength Rehe, working on building its own brand.

Post new comment

You must be logged in to post a comment.