There has been a growing interest among our U.S. clients to compare the opinions and perceptions of Americans to residents in other countries. This prompted me to recount some methodological issues discussed at a workshop I attended last spring in Switzerland.
The Workshop on Comparative Survey Design and Implementation (CSDI) was a small, international conference of survey methodologists who share a strong interest in cross-cultural, multi-national, and multi-regional research. Gathering at the University of Lausanne, the attendees were as diverse as the populations we study. Yet as we discussed the current literature and measurement issues of our global research, the cultural differences blurred while we convened to answer the same question: What are the best methods, tools, strategies, and protocols that help maximize comparability across countries, languages, and cultures?
Like nearly all fields, survey research certainly has its fair share of challenges…only to be exacerbated when conducted globally since much can be “lost in translation.” One presenter reminded us of the Chevy Nova story: a lovely car of the 60’s and 70’s that, reportedly, lacked sales in Spanish-speaking countries. Well, ‘no va’ in Spanish translates to ‘no go’ – not exactly a good marketing campaign for a car. And then, there was NASA’s $125 million orbiter that crashed into Mars when one engineering team used metric units while another used English units to estimate the distance from Earth.
No, we’re not launching rockets, but survey methodologists struggle with similar issues in international research. How do we design a question that precisely translates into multiple languages while adhering to various social norms? For example, does the term “strongly agree” elicit the same interpretation and emotion for the Chinese as it does to Americans? Well, research shows it doesn’t. How do we construct a representative sample when the accessibility, availability, and cultural expectations vary drastically across regions? A telephone survey in the United States is quite easy but definitely not as straightforward in many African…or even European countries. Given the mode of data collection affects survey data, how do we ensure that everything is comparable? And, how do we conduct sound research within reasonable costs and timeframes?
During the three day workshop, we discussed ways to test equivalence across multi-national survey instruments. In other words, methods to ensure that questions and response options, once translated, are understood and interpreted in the same way by all global survey respondents. Since quality control is challenged by the de-centralized nature of global data collection, we investigated innovative ways to implement quality assurance steps through interviewer management and monitoring, as well as strategic review of respondent data. Workshop participants also sought to better understand differential non-response across countries – that is, why survey participation and response rates may be higher in one country than in another and what can be done to increase cooperation without compromising data. We certainly didn’t solve all of the cross-cultural methodological challenges but strides were made…as were global connections and friendships.
Related Link:
https://www.csdiworkshop.org/