I just got invited by Dropbox, one of the largest companies on the Internet, to take part in a survey. What I found there may well be instructive to you, should you ever want to do a church survey or other organizational research.
My grad studies included a heavy research emphasis, and I’ve done quite a bit of organizational research since then. More than once I’ve been asked to crunch the numbers and produce a report on surveys others have conducted.
And more than once I’ve tried to share the lesson, Your survey was constructed poorly; I can’t give you valid results; This isn’t as easy as it looks.
Dropbox’s questionnaire included several questions asking to rate its quality, from “poor” to “very good” on how well it works for my team. But for most of the many years I’ve used Dropbox, I haven’t used it with a team.
So how should I answer? Give it a low mark because that’s the closest thing to saying, “No, I don’t use it that way”? Give it a high mark because I haven’t had any problems using it with a team?
They should have had some way to say, “Not applicable.” But they didn’t. The result is going to be that an unknown but probably substantial number of users will answer the question in a meaningless manner. They’ll have no way to separate the meaningless answers from the real ones. And that means they’ll get no valid, interpretable results on those questions.
If they can make a basic mistake like that, so can you.
I don’t often make this known, but if you need organizational research done, I can help. (Not for free; my training is worth a fee; and only for a select, limited number of groups each year.) I know how to design a questionnaire, I know enough to test it — because anyone can still make mistakes — and I know how to interpret the results.
Whatever you do — if you’re doing a church health study, an organizational climate survey, a morale check-up, or whatever — don’t go it alone. Not unless you know what you’re doing. Find qualified help.
Question: What’s the most important thing to know about gathering survey information in your organization?
Answer: You’re not just gathering information. You’re communicating information, too. You can’t survey a group without sending them a message. You’re telling them you care about what you’re asking about. You’re implying that you’re going to do something with what you learn. You may even be signaling that you don’t care what they think about questions you’re not asking.
Application: Make sure your questions are telling the truth.
(Disclaimer RE Dropbox: I gave up on the survey at this point. Maybe they asked later, “Do you use Dropbox with a team?” If so, then they’d have a way that I don’t know about to interpret the results. But that would be a wacky way to go about it. Ask that question first, so users know you’re not confused about these questions. Better yet — and this isn’t hard — design the survey so it only shows team-related questions to people who actually do use it with a team. And so on.)
Image Credit(s): Tungilik/Wikimedia Commons.
“Engaging … exhilarating! … This might be the most surprising and refreshing book you’ll read this year.” — Lee Strobel
Too Good To Be False is coming out soon! Sign up here for updates on the book and the blog, and receive a free preview chapter!