Online questionnaire cut short (report)

I had to cut short the questionnaire I was running on persons’ views of faith. Something in the code got garbled so that it didn’t present properly. I had a jquery expert look at it, I re-installed the software, and there was no easy fix with either approach. Having undergone foot surgery this week, I’ve had enough else going on that I’ve decided to go with just the information that came in while it was still working.

Design and Criticisms

I am reporting these results primarily because I said I would. Simply to have aborted the questionnaire with no report would have been been wrong, in view of that commitment, and it would have looked odd to critics. I do not claim to have any grand conclusions to offer, other than the obvious (see criticism 2): Boghossian was wrong.

Aside from the serious problem of its unplanned shortening, the questionnaire is open to a couple of major criticisms. The second is more valid than the first, in my opinion.

Criticism 1

Some people criticized the design of my “survey,” but that was based on their misconception that it actually was a survey. I have described it instead (and exclusively) as a survey-like questionnaire. It came out of between Peter Boghossian and Tim McGrew, in which Boghossian said that of the “tens of thousands” of people he’s talked to about faith, all of them take it to mean belief without evidence and/or (at least in practice”>last week’s debate pretending to know what one does not know. At one point he went as far as saying that was what “billions of people” believe. I do not know where he got the evidence for that.

When McGrew disagreed with him, Boghossian said that perhaps in McGrew’s “universe” of “50 to 100 people,” mostly academics, there were some people who had a different understanding of faith. McGrew’s view of faith could be summarized, “I have trust and confidence in Jesus Christ, and I do not agree that I’m pretending and/or have no evidence for what I believe.” Respondents were asked to choose which definition they thought most closely met their own view of faith, or believers’ view.

The intent of this questionnaire was to build on one that debate host Justin Brierley had posted on Facebook, asking listeners to indicate what they believed “faith” meant. His questionnaire reported percentage, not raw numbers, and it left out one or two points that were of interest to me.

The key question was whether there were more than 50 to 100 people, including non-academics, who disagreed with Boghossian’s definitions of faith. The answer to that is simply obvious to anyone not in Boghossian’s camp, but it seemed worthwhile to give a whole bunch of people a chance to say so.

Because there was no chance of drawing from any representative sample, I had no thought of drawing statistical inferences to any larger population. In other words, a response of x% to any question on this questionnaire could not and cannot be translated to x% of any group. This was not a survey in that sense. It was, as I said in the survey’s intro, a chance for people to voice their views.

Criticism 2

The more obvious criticism is that it was completely unnecessary from the beginning, and therefore almost a ridiculous exercise from the get-go since Boghossian’s view is so obviously wrong to anyone who knows Christian belief (and probably also to persons involved with other beliefs). One after-the-fact indicator of this is that the questionnaire showed Boghossian to be wrong in multiple ways, even though I stopped it short, with only about 5 to 10 percent of the responses I was hoping to receive.

Results

So with that in mind I present to you the responses.

There were 452 respondents. Of these, 378 indicated they were persons of faith, 47 said they were not persons of faith.

Boghossian’ Count Is Wrong Among Unbelievers

Of the persons “not of faith,” 31 said that they agreed more with McGrew’s definition of “faith” than with either of Boghossian’s definitions. It seems likely that if the questionnaire had stayed active a while longer, there would have soon been more than 50-100 people siding with McGrew’s definition, from among non-believers alone. It takes only a slight bit of responsible extrapolation to conclude that Boghossian’s rash statement stands contradicted by that information.

Boghossian’s Count Is Wrong Among Believers

Among those who indicated they were believers, the vast majority (365 out of 378) indicated that they were followers of Jesus Christ. The proportions here (and following) obviously reflect a skewed sample: mostly Christians, and weighted (by means of the questionnaire’s publicity) toward those with an interest in apologetics.

There were 349 respondents who chose McGrew’s view of faith as the more accurate one. In other words, it took no effort at all to show that Boghossian’s count was wrong. Although no accurate extrapolations or inferences can be offered from this questionnaire, we could go out on a limb and suppose that it failed to represent Western Christianity by, say, two orders of magnitude. (Is that allowing enough latitude for error?)

In that case, instead of 95.6 percent of Christians saying that McGrew’s definition is the better one, only 95.6 out of 10,000 would agree. Any small midwestern town would still have more than 50-100 people agreeing with McGrew. Boghossian’s “billions” doesn’t look too well supported in that case.

But what if they were all academics who took McGrew’s side, as Boghossian said? In my small non-representative sample, 255 of those who agreed with McGrew were not currently students or faculty members.

And what if they all merely said they agreed, without actually having concrete evidence in mind for their faith? Again, in my small sample, 233 actually listed categories of information they would turn to for evidence: a sign that they have thought it through well enough not just to be clicking a box on a questionnaire form.

Summary

The questionnaire ended up being disappointingly short with a ridiculously small sample for drawing any conclusions. Nevertheless, it was sufficient to draw at least one major conclusion: Boghossian was wrong. In a strange way, maybe the message is even strengthened by how little it took to show that.