Top 10 Things About Questionnaire Design
written by Pete Cape,
About the questions:
5. Ask individual questions: trying to get more than one piece of data at time is a recipe for disaster
6. Avoid the statements in a grid style question, it is tedious, mentally exhausting, and leads to over-fast processing.
7. Be careful with Yes/No questions – acquiescence bias leads to overuse of “Yes” – present alternatives to pick from instead (respondent will not know which is your ‘right’ answer)
8. Pictures speak volumes and overwhelm the words – use sparingly not as illustrations or examples
9. Make sure your scaled questions/items are scalable and not dichotomies. Make sure your scale is appropriate to your question/item
10. If the answer is a number and they know it they will give it to you. Do not use banded options unless you already know the distribution of the answers (the central banding should contain the mean).
Top 10 Things About Questionnaire Design
written by Pete Cape,
About the questionnaire:
3. It’s a conversation, make it flow like one. End subjects and introduce new subjects with separate text pages. Take time to thank, praise and motivate.
4. Respondents will, in the main, answer any and all questions put to them, often way past their own best interest! Keep it short (<20 minutes) or provide mental breaks.
Tomorrow we will wrap up the Top 10 with the Questions. Don’t miss it!
Top 10 Things About Questionnaire Design
written by Pete Cape,
About the people:
- These are ordinary people; ask simple straightforward questions written in plain language. Avoid jargon and complexity.
- The respondent knows they are in an experiment and wants to look good to you the experimenter; everything you write and show is thought to have meaning. Do you understand the meaning of what you wrote? It is easy to manipulate public opinion – don’t!
- People are unreliable witnesses to their own behaviour; they mis-remember because it was inconsequential, they post-event justify because they are human.
Check back next week to find out About the questionaire and About the questions!
Thumb Blog
written by Keith Phillips
Recently an argument formed between me and a co-worker on whether or not a man’s dress shirt (mine in fact) should be tucked in or un-tucked at work? In order to settle the debate, we took to social media and employed the help of the Thumb App (for smartphone and tablet devices). This application is a community of members who post pictures and ask questions one at a time. Thumbs up and thumbs down touch button are used as the answers. The single question also allows for additional comments to be added as open ends. Within a few minutes you get between 50 and 100 responses and then automatically stop collecting results.
This application is interesting to me for two reasons. First, it reminds us that people are motivated to participate in surveys for reasons besides monitory incentives. There is no cash, reward, or prize for answering these questions. Yet, I can see a user who answered my question, who has answered over 20,000 questions. So why do they do it? I would speculate that it is to see what types of questions are being asked by their peers, to express their own opinions to the community, see what the results are to these questions (which you get as soon as you answer), and perhaps they want to make a community contribution so that their own questions will be answer when posed. This last point is related to the second interesting thing about this application, which is the implication it could have on market researcher’s use of social media for survey research.
Social media’s role in market research has been one of recruitment and of listening (leverage the big data). However, the Thumb App creates a social media experience around survey taking. People in the community can see how many votes are cast by members who answer their question. They can save things they have given the “Thumbs up to” and they can even give each other stars for valuable advice. It’s a research community that is not constructed by or for market researchers, but inspires survey taking. Perhaps somewhere in these philosophies lies a piece of the future for the market research industry. Listening will become more important in the future, but in order to make it successful, we have to build the right instruments to listen to.
2012: A Year in Review
Now that 2012 has come to a close, let’s take a look back and reflect on our top 5 blog posts, as measured by most views. Mixed mode, global, respondent satisfaction – all of which we’ll hear more about in 2013 and beyond…
1) Mixed Mode: Plans for 2020 Census
2) The breakneck speed of economic global change
3) National Representation: Is this What You Need?
5) How Target Figured Out A Teen Girl Was Pregnant Before Her Father Did…
Put Yourselves in the Respondents’ Shoes
written by Trish Daly
Have you ever watched TV while surfing the web? Or played a game on your smartphone while checking your Facebook account on your tablet? If the answer is “yes,” you’re not alone.
In today’s day and age with increased media and technological advances, there are even more distractions that could take a respondent’s focus away from completing an online survey. That’s why we recommend designing online surveys that are 20 minutes or less. Keeping the online surveys short, clear, and concise will help minimize “straight-lining” and respondent fatigue. The age-old saying applies to market research: we need to treat respondents like we would want to be treated!
Engaging respondents throughout their survey experience is an integral part of online research. Watch our Podcast: SSI Research Settles the Debate on Online Survey Length – Once and for All. You can also read an article about the Rules of Engagement by clicking here.
Online Surveys: Research Design is Imperative!
written by Trish Daly
Have you ever participated in an online survey and struggled to understand the question? Or perhaps you were asked a question and the list of responses didn’t apply? Do you own a dog, and were asked questions about cat food?
There are several areas that could hinder your research, which is why research and questionnaire designers need to be vigilant when crafting survey questions, and also determining the appropriate skip patterns/flow of the questions.
Watch SSI’s podcast on Tips for Effective Survey Wording, SSI’s webinar on Getting Engaged with Your Research Respondents, and take a look at a series of articles below that can help in identifying common issues with online surveys. Have you been guilty in the past of any of these issues?
http://survey.cvent.com/blog/cvent-web-surveys-blog/common-survey-pitfalls-leading-survey-questions
Top Billing on the Connecticut Ballot
written by Keith Phillips
Connecticut General Statutes > Title 9 > Chapter 147 > § 9-249a – Order of parties on the ballot label: The party whose candidate for Governor polled the highest number of votes in the last-preceding election.
In the US, each state has its own laws on how elections are run. In the state of Connecticut, the order the candidates appear on the ballot (and in the voting machines) is determined by which party the current governor of the state is in. At least that’s the way Secretary of State Denise Merrill (from the Democratic Party) interpreted the statute. However, the Republican Party sued the state, saying their party should be on top, because their gubernatorial candidate in the 2010 election, in a losing effort, received more votes on the party line. The current Connecticut governor (Dannel Malloy) was on the last election ballot for two different parties. He received 540,970 votes as a democrat and an additional 26,308 as a cross-endorsed Working Families Party candidate. Tom Foley received 560,874 votes and was only listed as the candidate for the Republican Party. Although Malloy had more votes than Foley, which won him the office, he had fewer votes on the party lines. The Republican Party brought the suit before the Connecticut Supreme Court, which has now ruled in their favor.
With the time and effort spent on this issue by the parties, one has to ask, does ballot position matter? And the answer to that question seems to be “yes it does.” According to Kelly Radar, a Yale political science professor, research has shown that candidates who are listed first on ballots tend to win “a few more percentage points.” This is especially true for unknown candidates and for voters who do not have party affiliations.
We have to keep in mind that the ballot order is not just for the presidential race, but for all the different offices up for election this November.
It seems that in the short term, the Republicans have won the battle, but perhaps this is a small price to pay for a Democratic Party whose candidate won one of the closest gubernatorial elections in Connecticut history. One has to wonder, if Malloy’s name was only on the ballot once, which party would have top line position for the upcoming presidential election. Now Malloy will need to decide whether or not his name should appear twice if he runs for re-election in 2014.
How are good Web Design and good Survey Design related?
written by Craig Vance
Attractive Surveys – More Than Just A Pretty Face?
You know a poorly designed website when you see it. It elicits the desire to cringe and avert your eyes – and never go back again! Whatever company, brand or person owns the site doesn’t have a fighting chance. But think a little more about what makes it so intolerable. Is it the organization of content, the color scheme, or maybe it’s just too difficult to read? Aesthetics are probably a big part of the reason you would leave, even if you don’t make that connection initially. Most poorly designed websites are disorganized, provide too much or too little information, use distracting or loud color schemes and images or are just plain difficult to read.
Now, think of a website that you really like. Even if you can’t exactly put your finger on what makes it attractive, you may feel much more strongly about the company or person that website belongs to. In our digital age, how good something looks and how easy it is to use is almost as valuable as what it provides to us. Web design has come a long way since the early stages of the Web, and design continues to be even more responsive as consumers change how they interact with media. Things that were only dreamed of 10 years ago are now common in our Web 2.0 world.
While web design has made very big strides over the years, Marketing Research seems to be much more resistant to change – at least from a design perspective. Web surveys, for the most part, look much the same today as they did at the “birth” of online panel research. There are online survey formats that I wish would go away and never return, for the sake of all. Many surveys are almost too ugly to read or understand, not to mention frustrating to people participating in surveys. They are boring, long, and sadly, aren’t even nice to look at.
If a poorly designed website can’t be expected to command the attention – or purchase – of a customer, why would we expect a poor web survey to keep them involved? Surely, online surveys can’t be exempt from the demands of the general public. When we serve up a poorly designed survey, we should expect the same results as those from a poorly designed website. Respondents will run away screaming and vow never to return.
So, why not make research beautiful too? And what would happen if we got more of what we really wanted out of a well-designed survey – actionable data? Maybe better survey design really is worth all the trouble.
Third Annual Quest Awards: Press Release
BDRC Continental, Qualtrics Labs, Inc. and Sinomonitor Take Top Overall Honors, While deKadt Marketing and Research and MMR Research Worldwide Earn the Highest Marks for Product Placement. Sinomonitor is named Best in QUEST.
Atlanta, GA, September 10th, 2012—SSI President and CEO Chris Fanning announced the winners of the third annual SSI QUEST™ Awards—honoring companies for creating the most engaging research experiences—at the opening session of the ESOMAR Congress in Atlanta today.
Fanning recognized three companies as overall winners for excellence in survey design: Qualtrics Labs, Inc. in the Americas, Sinomonitor in Asia-Pacific and BDRC Continental in Europe. Top honors for product placement surveys went to deKadt Marketing and Research in the Americas and MMR Research Worldwide in Europe. The highest honor, the 2012 Best in QUEST Award, went to Sinomonitor.
SSI introduced the QUEST awards in 2010 to encourage the market research industry’s creativity in developing surveys that delight participants. Self-reported satisfaction scores from people taking each survey are a key factor in determining the winning projects.
“The quality of the experience we give our participants is sometimes overlooked in all the industry discussions about quality research,” said Fanning. But to produce methodologically sound samples, we need a large pool of willing participants. We must give our participants a good survey-taking experience, or we risk losing them. This year’s SSI QUEST™ winners clearly demonstrate their commitment to high quality research results and to our industry resource of participants.”
Winning companies will receive a customized SSI QUEST™ trophy. SSI will present certificates of excellence to the project managers and programmers responsible for developing winning surveys, and make a donation in their names to the charities of their choice.
