(no subject)

[gaudior]

I was talking to a (moderate, politics-wise) friend this evening about gay marriage. He advanced the idea that everyone, straight and gay, should have civil unions, which include all the legal rights and responsibilities of marriage, but don't use the word. Instead, the word "marriage" should be reserved for religious institutions, which can then decide for themselves whether they will perform same-sex marriages.

This is not a new idea. This is, in fact, an idea which I myself advanced early in the debate, and someone told me, "No, that's a bad idea-- no-one will vote for it."

But it just keeps coming up. I've heard the idea proposed half-a-dozen times over the last few years, always by relatively moderate people, all of whom seem to think that they've come up with it on their own.

And I do know a number of arguments as to why people want to insist on actual legalized gay marriage (not the least of which is that it seems entirely likely to happen, and soon)(and all the difficulties involved in things like, say, Catholic hospitals not wanting to allow unionized queer couples visitation and adoption on their premises). But I find myself curious about the numbers.

So, o my (admittedly biased) sample:

[Poll #1452891]

And this raises the question-- if it turned out that a majority did favor this plan, should same-sex marriage advocates change tactics? Or not?

Discuss!

--R

(Upon request, I have added the following questions, for people who like the status quo in New Jersey, here. That's, civil unions for queers, legal marriages for straight people. Unfortunately, lj will not allow me to revise a poll, or add another poll to a pre-existing entry-- otherwise, I would edit this into the original.)



ETA: Okay, so, on further reflection, I clearly did not think this entry through very thoroughly at all. It is, as many people have pointed out, a bad poll, poorly worded, and unlikely to get results which are in any way representative of the general population. If I were being a real social scientist, this would have been my test-run, in which I found out all the things wrong with the poll before revising it, running it by another test pool, and then taking it to a large, anonymous, randomized sample, preferably with multiple methods of reaching participants of a good range of demographics.

Which was clearly not my intent. Honestly, I just wanted an ideas-check-- "Hey, I've heard this idea from a bunch of people, but I don't see any moves towards it-- howcome? Is it a bad idea, and if so, why, so that the next half-dozen times someone proposes it to me, I'll have ideas about what to say?" Or it might have been possible that it was a good idea, which for some reason no-one had proposed, in which case, I might have wanted to take more action. But I didn't have a real agenda besides finding out what people thought, and looking for more ideas.

So, my apologies for taking so long to respond to people's interesting and insightful comments-- I was somewhat overwhelmed by just how many responses I got! But. Onwards.

--R

Comments

[eredien]

I think that lignota is arguing this:

Gaudior's poll included options x and y, but lignota noticed that gaudior had entirely overlooked an option z.

Gaudior's poll seems to have been intended to capture the entire range of opinions, but from the very start it didn't and couldn't do that, becuase it didn't include the entire range of opinions in the first place.

Lignota--correct me if I'm wrong--seems to think that gaudior left option z out of the poll so that the poll could seem to be unbiased while actually containing a clear bias. I don't agree with that; based on gaudior's reaction, I think gaudior left option z out of the poll because she didn't realize she should have included it.

However, the reason why gaudior left out option z isn't the point, whether she forgot it, didn't realize she should have included it in the first place, or deliberately chose not to include it.

The point is that leaving out any option at all for any reason would still leave any poll with a bias--but because the poll includes statements that the poll's author disagrees with, the casual poll-taker will assume that that fact alone makes the poll unbiased. (In other words, if the questions present in a poll seem unbiased, most people will assume that makes the poll unbiased, when in fact leaving off questions means that the poll will be biased in favor of the questions present and against the questions that are absent. It's hard to see absence instead of presence. Since most people aren't looking for absence and you're polling them on the presence of options, most people won't even think that something is missing). This technique is well-known and is often used to make polls that are deliberately biased, so people will answer the question in a certain way.

Adding option z to the poll retroactivley doesn't fix the problem, and indeed introduces new problems. Introducing it now means only that option z is now available to new survey-takers, but not people who have already taken the survey. This means that the data you are getting from the poll is skewed, such that any conclusions you might try to draw from said data are going to be automatically faulty.

That is one of the reasons why I am not voting in the poll. Another reason is that I am not clear about some wording choices in gaudior's explanation. Another reason is that I am sick of having disagreements with friends right now. Since this is an issue I know I am likely to have arguments about, the debate around terminology and law on this poll, skewed or not, isn't something I have the ability or will to deal with right now, in public or otherwise.

[eredien]

Gaudior: there seems to be a lot of subtle bias in this survey. I understand that it's not scientific and it's not meant to be, but the assumptions you are making are problematic, even if you're not meaning to make them, and even if you're not a social scientist doing hard research.

You may want to read recent discussions on surveyfail! for how other people made some of these same assumptions; because their assumptions are more blatant you can see problems with thier survey that are the same type of problems you have here. Since their assumptions are more obviously blatant, it is easier to spot the problem.

For example, the question "are you male or female?" contains bias against genderqueer, transgendered, or intersexed people. That biases your poll because those options are left out, in much the same way that leaving out option z in your poll biased your poll because an option that existed was left out.

I am not saying that the assumptions you are making are on the same level of horrible offensiveness as the assumptions that the surveyfail! people made and then had the gall to defend. But because your assumptions are less blatant doesn't mean that renders them unoffensive; unfortunately, the fact that they are less blatant means that you will likely find it harder to spot and squish them.

[zdenka]

weirdquark has correctly interpreted what I was trying to say.

Lignota--correct me if I'm wrong--seems to think that gaudior left option z out of the poll so that the poll could seem to be unbiased while actually containing a clear bias.

Not really. I don't think gaudior would play games like that.

I admit I phrased my first comment non-neutrally, because I was frustrated that (as I saw it) gaudior's ideological bias was causing her to miss something that seemed obvious to me.

I'm not actually expecting gaudior to conduct a survey with perfect scientific rigor here, and so I'm not personally that worried by the skewed-data issue. I think the comparison to SurveyFail is unfair. My concern is more that I think the implicit model behind gaudior's poll is faulty, and that if she wanted her poll to describe/capture the range of opinions about same-sex marriage and civil unions, she should have put a little more thought into it, including how to describe the range of opinions that she does not agree with but that people nonetheless hold.

[(Anonymous)]

Not really. I don't think [info]gaudior would play games like that.

I admit I phrased my first comment non-neutrally, because I was frustrated that (as I saw it) [info]gaudior's ideological bias was causing her to miss something that seemed obvious to me.

Ok, that makes sense to me. I was trying to reconcile the fact that your original comment didn't seem neutral to me with the fact that I didn't think you would think that gaudior was deliberately playing mind-games with an admittedly non-scientific survey, so that is why I asked for clarification and/or correction. Thanks for providing it!

I think the comparison to SurveyFail is unfair

Hm. Maybe my intent didn't come across clearly. I don't think that gaudior's questions themselves were comparable to the SurveyFail ones; in fact, that was part of the point I was trying to make.

I was trying to draw a comparison between the type of bias problems I saw in the surveyfail survey and gaudior's survey, and point out that I was seeing some of the same general type of bias problems (for instance, "leaving out an option") in both surveys, even though the survey topic and extent and visibility of bias were in fact, so very different in each individual survey.

Using the surveyfail example may have been a bad call on my part because it's so inflammatory, but I chose it as an example, despite its potential for drama, for a few reasons:
- I have recently been reading about it and using it as a tool to think about survey bias and bias in general in my own life and head.
- I wanted a survey where most people would agree that the example in question showed clear bias problems and little nuance, such that there wouldn't be a question of whether my example of clear bias actually contained clear bias.
- I wanted a survey that showed obvious and clear bias problems and little nuance in order to contrast it with gaudior's survey, which has subtler bias problems and a lot more nuance in its questions.
- I wanted a survey that people reading this thread would probably be somewhat familiar with.

My second-choice example of a biased survey, a survey I took on furries in July, contained some bias. However, I think the survey would have been unfamiliar to readers, the bias would not have been as obvious, I wasn't sure if I would not remember it well enough to use it accurately, and it was vetted beforehand by an IRB as part of an ongoing research project. So, I chose not to use it as my example.

I wondered why both surveys shared the same types of bias problems, despite their other differences. I realized that the more obviously biased the survey itself was, the easier it was to see that bias; but that the less obviously biased the survey was, the more difficult it became to see these very same biases. The strength of nuanced questions, their nuance, can also be a liability in terms of bias visibility.

(Perhaps this is because less nuanced questions tend to not only have large biases in the first place, but act as their own pointers to the bias contained in the question, while more nuanced questions may have fewer biases and be less able to act as their own pointers to the bias in the question? I view nuance as a sort of a narrowing-down of space for bias to exist within the question--though the possibility of bias outside of the question or the perniciousness of the bias itself is another matter entirely.)

You can probably learn a lot about how to avoid bias from surveys that have obvious bias problems and little nuance: if you've seen enough obvious instances of the same type of bias on a large scale, you can start seeing less obvious instances of the same bias on a smaller scale, since you already have some idea of what you are trying to avoid.

[gaudior.livejournal.com]

I would like to say again: this was a terrible poll. It was not intended to be good science, nor intended to be presented as such. But I had not realized just how terrible a poll it was until lignota, angstnokami and weirdquark made the very salient points they've made above. And you (eredien) are quite right that the attempted changes I made were not useful either, and that it does show my biases very clearly.

Sorry, folks. No conscious intent, just lack of forethought.

Edited to make it clear I was responding to eredien.

[eredien]

It's hard to make lj polls that are useful for any kind of real--or even quick-and-dirty--data-gathering, even if the data being gathered is not important. I'm on an lj scifi book group that has had trouble using the polls for something as simple as picking the upcoming month's book from a list of options.

I want to reiterate that I don't think that gaudior presented the poll as a scientific one, and I don't think that anyone else who's commented here is assuming that it was; it's just that these are salient points for anyone who is trying to collect data via any poll, for any reason at all; I am actually planning on bookmarking this thread to look at in the future before I do any lj polls on my own journal because I think the discussion was awesome.

Side thought: I got this mental image of gaudior wearing a pristine white lab coat, holding aloft a beaker, clicking "add poll," and saying, "ha-ha, it's scientific now!" :D

[gaudior.livejournal.com]

Side thought: I got this mental image of gaudior wearing a pristine white lab coat, holding aloft a beaker, clicking "add poll," and saying, "ha-ha, it's scientific now!" :D

Laughing out LOUD, me.