Bad survey design and how words work (brought to you by @NatCen)

The British Social Attitudes survey is an annual survey of about 3,000 people it asks people hundreds of questions on all kinds of topics

One of the questions it asks is about transgender people using public toilets in which 47% of women say they are v. comfortable with a "transgender woman" using a womans toilet and only 41% of men saying they are v. comfortable with a "transgender man"

Which is kind of odd when you think about - are men really less comfortable about sharing their space with "female to male" transitioners than women are about "male to female"?

Do they really understand the terms? People answering the survey are given an intro screen which explains what a "transgender person" is

So you click enter to say you understand that it means a person who has gone thru all or part of a process which ranges from thoughts to using a different name, to putting on clothing, to taking hormones or having surgery.

(remember: this is in the midst of lots of other Qs)

Then men get a question about "transgender men" in the mens loos and women get a question about "transgender women"

This begs the question - at what point on the spectrum from thought to clothing and makeup to wearing fake breasts to genital surgery does a man become a woman?

The survey doesn't tell you so people have to mentally fill in the blanks on the word "woman" and "man"

I suspect women are answering the question based on some subset of the spectrum where they consider a transwoman has put in so much effort it would be mean to say no

I suspect men are answering a completely different question in their minds -- how comfortable do you feel about a transvestite or MtF transsexual in the mens loos?

I doubt there are many men who understand a "transgender man" in the question is supposed to mean a female person

Its also worth noting that people being interviewed are answering this question after having been asked do you think prejudice against transgender people is wrong, which of course most people answer in the negative.

So faced with an ambiguous question about whether they are comfortable with "transgender women" using women's toilets they say "no".

But it seems highly unlikely they are applying this to the whole spectrum of people covered by the definition they agreed to.

Natcen then report this as people being comfortable with people using toilets "according to their gender identity".

This is not what the question asked.

We have no way of knowing what women really think about people with penises using women's toilets because the survey is so badly designed.

How did that happen? Well questions in the survey are sponsored by interest groups.

@NatCen sometimes says who funds particular questions, but I haven't been able to find the information on this one.

@Nancy_M_K recently moved from Deputy Director of NatCen to head up Stonewall.

I am going to ask Natcen who funded this question

A bit more on the Natcen survey (HT @SeaWitchMelody) - so apparently they do repeat and make clearer the definition in the Q.

I still think this is leading "all or part of the process to become a woman" is vague and assumes a process by which a man can become a woman...

To which people will fill in the gaps - what is that process?

Is someone who is "pre-op" and intending to have surgery different from someone who has no intention of surgery?

In practice how can we tell the difference?

Also notable the question was designed to "further examine a person's level of prejudice".

This was not a question designed from a neutral point of view that it might be legitimate for women to feel uncomfortable with male people in their toilets.

I have found who sponsored the questions in the survey it was @GEOgovuk ....