The Power of Telephone Interviewing in the Online Research Age.
written by Daniel Somerset
Any industry with a desire to stay relevant in today’s market must innovate. Market research is not immune to this. Research was rooted in face to face interviewing but with changes in communication and technology, the industry evolved from face to face and telephone interviewing (CATI) to online and mobile research.
Often though there is a new technology is available, it doesn’t mean that the previous method is no longer useful. In the case of CATI, the opposite is quite true. While online and mobile surveys are great for wide reach and low cost, CATI is still a meaningful and affordable solution in many other situations.
One situation where CATI excels is coverage. Even though (in the US) people are transitioning away from landlines, more people are accessible through random digit dialing for landline and wireless than through online surveys; this is particularly true for hard to reach demographics. CATI can even target individuals more precisely including down to the postal code, city block or even street.
Telephone interviewing has the advantage of a live interviewer. This skilled interviewer can ease tension on a stressful survey, skillfully navigate past “gate guards” to get to executives for B2B surveys or act as a brand ambassador on your next customer satisfaction study.
Additional advantages as well as case studies can be heard at our 19 June webinar: Old Dog, New Tricks: The power of CATI in the online research age”. After the 19th, the on-demand version will be posted to surveysampling.com.
Difficult questions
written by Jackie Lorch
In another example of the power of word selection in survey questions, researchers at the University of Pittsburgh examined how physician communication affects end-of-life decisions.
The study, published in the journal Critical Care Medicine, and reported in the New York Times, tested whether patients’ relatives would be more likely to consent to or decline CPR for their loved ones depending on whether the medical order was “do not resuscitate” or “allow natural death”.
The simulated test, among a nonrandom sample of 252 designated health care surrogates found that emotional tugs, like photos of the patient, or sympathetic words from the doctor made no difference to their choice. But being told what other people had chosen did make a difference, as did the words used in the question.
61% requested CPR when the choice was between CPR or “Do not resuscitate”, but only 49% when the choice was CPR or “allow a natural death.”
As researchers know, word choice is extra important when the topic is complex and nuanced. When researching complex issues, definitions should be included so respondents have the best possible understanding of what they are being asked about.
Share and Care? The Possibility of Insights as Incentives
written by Apurv Shanker
People form the core of what makes a panel – people who are willing to participate and are responsive to undertake survey research. While the discussion in the market research sphere continues on the amount and type of monetary incentive we give respondents, we acknowledge that what makes respondents tick is for them to feel that they are an integral part of the research process.
Are we giving them enough information to keep this thirst to participate going?
A quick look at some of the panel home pages shows that respondents typically see: the surveys available to them, the number of surveys and rewards they have collected so far and how they can be redeemed. What if we take this a step further by sharing the insights that are drawn from the surveys that have been conducted? Confidentiality is a key concern and this topic definitely opens up a whole new can of worms (maybe we can share general research findings rather than exact data?), however this could be one of the factors that aids in keeping the panel churn low by captivating interest and additionally tackling the industry wide challenge with younger demographics.
What if a young adult could see that his participation has led the local government to improve road-work in his community, or that his university has introduced new course choices, because of research that was conducted on those subjects? What if we could share the new pricing changes to baby food products with the mothers on our panel, based on insights derived through them recently?
Additionally, research agencies constantly publish some of their insights on their websites – would sharing all this with respondents make them appreciate the role they play in the market research cycle? I guess we need to do a survey to find out if that’s what respondents would like!
Is Google Glass a Privacy Crystal Ball?
Google’s new wearable computing and how it may impact consumer attitudes to privacy
Written by Kristin Cavallaro
Changes in technology happen gradually and constantly, so when it comes to the impact of technology on our personal privacy, we are like the proverbial boiled frogs – unable to identify the precise moment when we lost control over our personal data.
So it’s good to have a new and attention-grabbing technology like Google Glass in the news, causing us to focus on its implications for personal privacy and assess how much has changed in a short time. In case you haven’t heard, Google Glass, currently being prototyped, is the first mainstream wearable computing device. According to Techradar, the glasses have a small display, placed just above eyeline level, and an embedded camera and microphone to record everything in view. This may sound like science fiction, but consider how much has else has changed in just the past few years.
- We are now routinely recorded by CCTV cameras as we go about our daily lives. In Britain there is one surveillance camera for every 14 people.
- Facial recognition on the fly (the ability for someone to see you on the street and know your name and personal details using data you posted on social media), has been possible for at least a couple of years.
- Similarly, apps can profile you when you walk into a bar, and broadcast an assessment of how you compare to others already in the bar – in terms of income, attractiveness — or likelihood to buy the next round perhaps.
- In the US, courts have not necessarily ruled against government requests for social media and other data to find out where someone was during certain periods of time.
How do we adjust to these developments? Do most people accept that the concept of privacy has changed, and may even cease to have any meaning in the future? Or will there be a backlash and desire to retreat into a cocoon of privacy? At the other extreme, will commonplace access to masses of personal information about millions of other people in fact make us more empathetic to our fellow human beings?
SSI’s Knowledge Team member Kristin Cavallaro has been researching changing attitudes to privacy as part of a joint project with J.D. Power & Associates. The study finds differences in attitudes by demographic, and uncovers some of the ways that people are trying to reclaim control over their personal information online.
Click here to learn more about these findings.
To join a mailing list to be alerted to this and other SSI research findings, e-mail info@surveysampling.com or tune into our Privacy Webinar: Privacy and Data Protection in the Digital Age this Thursday, May 30th at 11:00am EST.
From “Respondent” to “Retailer”: A new development in the Data Economy
written by Jackie Lorch,
Many readers will be familiar with the website Kickstarter, where friends, acquaintances or complete strangers starting their own creative enterprises request help with funding: to make a music video, create an art portfolio, expand a home business. Since 2009, 4 million people have pledged funds for over 40,000 projects. Now comes a new research-related Kickstarter twist.
Federico Zannier a student at New York University is using the site to get companies to pay for access to his own personal data.
According to an article at research-live.com, Zannier has documented his online activity and other detailed information about his habits and lifestyle, and believes it has value to advertisers.
It appears he’s correct, as he has apparently exceeded his original fund-raising goal already.
For years we’ve called the people who take our surveys “respondents” – a name that signals we think of them as passive participants in the opinion process, sharing information about themselves at the bidding of the researcher. Today, people are rejecting that role, becoming increasingly savvy about the value of their opinions and expecting something in return for their data. Zannier’s project takes this to the extreme. For $2, a business can get access to a full day’s worth of his data; $5 for a week’s worth.
Would you sell information about yourself? What information? Would you pay $2 for a day’s worth of someone’s activity? Does this move herald the start of a trend? Will it soon be the norm for individuals to package and sell vital information about themselves to business?
How does this impact the research industry?
New SSI research into what panelists really think about rewards challenges some long-held beliefs
written by Pete Cape
Since online research began nearly two decades ago, rewards for survey respondents have been an integral part of the process. Several research studies have examined the impact of different reward amounts and types on response levels and panelist loyalty.
New research from SSI challenges the overall framework of how we have thought about rewards. It finds that a surprising number of respondents — even those who say they only take surveys for the reward — don’t know what reward is being offered for a particular survey they take.
These findings suggest that respondent’s reward expectations are set at the time of recruitment into a panel or community. At that point we make a deal with the respondent that they will be rewarded for their effort. But from that point forward, many panelists barely notice or remember what reward is being offered.
The conclusion from this could be that we should think more strategically and holistically about rewards. Instead of focusing on algorithms incorporating survey length or difficulty or population scarcity to set per-survey rewards, we could start to separate the survey event from the overall “membership experience”.
What would that type of rewards approach look like? Perhaps we would reinforce the reward experience by adding an additional “thank you” reward at the point of reward redemption. Or give an unexpected reward – to celebrate the respondent’s birthday, or the anniversary of their joining the community, or as a thank you a couple of days after they complete a challenging project.
We have an opportunity for some creative thinking about this important aspect of the economics of online research — because rewards clearly impact those economics in a big way. As sample providers, we must maintain large, engaged populations of respondents; while the research market is driven to get reliable opinions as economically as possible. New thinking about the strategic use of rewards could help us make progress on each of those goals.
For more details about the results from this research, contact SSI.
DIY Research: No longer a threat to MR agencies?
written by Rosemary Ayoub
Browsing LinkedIn the other day, a discussion popped up in my news feed. Specifically: a poll on DIY research being a threat to research agencies. I’m quite sad to say I’m in the minority in terms of votes, although the comments seem to be fairly positive and full of common sense. Out of a grand total of 24 votes, only 6 were bottom 2 box (not a threat), 4 were neutral and 16 were top two box (seen as posing a threat). It surprised me a little because I really thought the world had changed a little.
Years ago, when DIY research was not available, agencies took on all projects form the massive segmentations to the three question omnibus. Nowadays it seems agencies are getting better at passing these across to data collection agencies, and admitting how much value they can or cannot add to a study. I see this quite often from some of my clients, who have a good relations hip with their clients and also a very clear idea of what they as a business are offering.
Also, I think some end clients, having been burnt by bad research have been better at knowing what to handle in house (omnibus, short straightforward surveys). I think the vast majority will always have use for agencies for strategic pieces of work, or those where an independent voice is appreciated.
I’m sure there are still studies run in-house which shouldn’t be for various (sometimes obvious reasons e.g. employee satisfaction, etc.), but on the whole, I think things have moved in the right direction.
Respondent-driven Sampling
written by Paul Johnson,
So I was excited for the AAPOR webinar on hard-to-reach populations because I really feel like this is the hardest nut to crack in the industry. Unfortunately, I left being underwhelmed probably because of a misalignment of expectations. I came in thinking that hard-to-reach is the same as hard-to-sample so I was expecting the webinar to focus on the hard-to-sample challenges. I am grateful to Dr. Tourangeau for helping me broaden my horizon. As an employee of a sampling company, sometimes I get too focused on the hard-to-sample problem and not enough on the big picture. Still, for this post I want to focus on the hard-to-sample population and open a debate on whether or not respondent-driven sampling can actually produce good estimates that can help a company make informed decisions.
Respondent-driven sampling is a type of snowball or referral sampling. It relies not on standard probability theory, but rather on network theory to estimate the probability of being selected from the population. It makes the jump to say that it doesn’t matter where in the network you start (the seeds of the study), but as long as you map out the network you can make adjustments to account for the probability of selection inside the network. That is a large claim. Taken at face value it is hard to imagine that a convenience sample of a connected convenience sample with somehow turn into a probability sample, but the math works out in theory. Whether theory turns into practice is another issue though. Dr. Tourangeau did a great job of listing the key assumptions of the theory behind respondent-driven sampling:
- The population that connects to each other. After all, if the population doesn’t connect to each other then there is no network to examine and you are not likely to get referrals anyways. This one doesn’t concern me too much as I really believe that most of these hard to reach populations are connected in some way as the whole world become more connected. However, if you are doing respondent-driven sampling and you don’t get any referrals, you have to ask yourself: “Why am I doing respondent-driven sampling anyways?” You might need to reexamine your population or realign your incentives.
- The population is in a single network. Once again, I am not as much of a skeptic here. Even when it is shown that there are distinct clusters inside the network, there is some crossover. As an example, you can look at a picture of the some of the political networks and you will see clear clusters by party but there are still some connections between the two clusters from the people who work on both sides. http://blog.magicbeanlab.com/networkanalysis/calculating-party-affiliation-us-congress/ NOTE: that this might not be the best example because a lot of Congressman did not tweet each other, but the principal of network crossover still applies.
- Each respondent’s number of network connections needs to be known. I could see some respondent reporting error in here. Frankly though while it might not be exact it should be a close enough approximation to use for our purposes. Also in the future, as social networks develop we should be able to collect some of this data passively depending on the audience we are measuring. This seems to be a solvable problem.
- Respondent need to randomly recruit from their connections in the network. This is the one that gets me. I think that some people’s attributes will make them more likely to be referred than others. At a minimum, the type of connection (friend, family, professional) would have very different chances of being referred. Even if you instructed the respondent to randomly invite the people they know I am not a big believer in the ability of the human mind to create random numbers (our brain isn’t really wired that way in my opinion, but it is a fun topic to Google and of great importance to the computer science community). I am sure that this assumption could be relaxed as long as the probabilities of being referred were known. That is the tricky part though. How do you elicit the probability of referring each of the connections in the network that you have? I don’t have the answer here, but I do think it is an interesting challenge.
All that being said, frankly with most of my work my clients just need a “fit-for-purpose” solution. Respondent-driven sampling might just be that even if it doesn’t approach a probability sample in real life as well as in theory. Look out for my colleague Kristin Cavallaro at AAPOR conference in Boston next month to see how this type of sampling compares to traditional opt-in panel sampling. When you have budget, but you want to make intelligent solutions sometimes “good enough for government work” – or in this case maybe “good enough for non-government work” – is all you need.
India: High Hope of Mobile Internet
written by Ati Sinaga
The world has witnessed India pioneering the outsourcing business a few decades ago and this business is now a significant source of foreign income to the country. Part of the success is due to the IT skills of local talent that makes people expect that India would easily tap into the internet business. But it was far beyond expectations that only 10% of residents have access to the web world. Why?
A recent article in The Economist suggests that compared to the outsourcing business, the Indian government has a higher involvement in the local internet modulation and it tends to over complicate the regulations. This problem, combined with corruption within telecom regime, says The Economist, ensures the internet industry does not move as fast as expected and it does not seem it will become resolved anytime soon.
So, where is the hope for larger segments of the Indian population to get connected with fast moving communications? Thanks to cheap smartphones and a fast wireless network, people are putting their hopes on mobile internet. Will India have another success story and lead the mobile internet? So long as government and telecoms industry work together and simplify the rules, a bright future is there.
And how would this affect the survey or data collection industry? No doubt mobile surveying is on the rise…
The Daddy Demographic
written by Melissa Geathers
Evolving parental roles are leading to new inroads to male insights when it comes to market research. Dad is no longer sole provider detached from his home environs with peripheral connections to his family. Today’s dads are working dads, stay-at-home dads, single parent dads and same-sex couple dads whose integral roles in their family’s lives have shifted away from the traditional definition of fatherhood. Dads are more involved in the day-to-day care of the children and household duties. Dads purchase the groceries and the products needed by their families. Modern day dads are also looking for brands and products that represent them and seek their insight as full and committed partners in their homes.
However, it appears that we as market researchers are overlooking the determined and developed dad. Family men in their 30s and 40s are missing from most of the product discussions for traditionally mommy purchased items, yet a recent Nielsen study notes that in 2012 men spent an average of $36.26 at the grocery store per trip, compared with $27.49 in 2004. As dads become more involved in the raising of their children, they have greater influence over dollars spent in the home on products and thus can be brand drivers for brands not usually associated with men.
Market researchers need to speak to dads as competent parents and not remove them from the discourse. It seems to me that the market research industry needs to do a little catching up. Market researchers need to be thinking about how moms buy, how dads buy, and how families buy, rather than focusing on one gender as the sole decision maker/purchaser/caregiver. In a society where we are constantly redefining what it means to be a family, we as market researchers need not only to be more inclusive in our search for insights, we need to be more creative in how we help dads discover brands that create a space for them that was influenced by them.
http://www.nytimes.com/2013/02/24/business/fathers-seek-advertising-that-does-not-ridicule.html?_r=0
http://www.bc.edu/content/bc/centers/cwf/news/TheNewDad.html
http://ssl.gstatic.com/think/docs/new-dads_research-studies.pdf
