Keeping up with technology: What is “scientific lag” and can we proactively reduce it?

In 2011 then Census Director Robert Groves wrote on the Census Director’s Blog about the burgeoning volume of “organic data”—data that, as opposed to “designed data,” have no meaning until they are used (surveys are a primary example of the latter). He noted that finding ways to combine these two types of data to increase the “information-to-data ratio” was a challenge, but also represented the future of surveys. Using terms identified as “big data descriptors” in Groves’ piece, as well as a few other terms I think qualify, I put together the graph below to show the number of AAPOR presentation titles between 2010 and 2013 that contain a big data descriptor.1,2

One take away is the increased interest researchers have shown in big data over the past few years. An equally important lesson is that almost all of the attention big data has received from AAPOR members—at least measured by the number of presentations they’ve done—has been on social networking sites (SNS). I found only one presentation in the past four AAPOR conference programs that contained a big data descriptor for a non-social media topic—a demonstration in 2012 by Ben Waber on the use of wearable sensors for measuring behavior.

To some extent this is explained by scientific lag. Just like there is cultural lag—the time between the emergence of a new technology and when culture catches up—there is a lag time between when consumers adopt technologies and when our research methodologies catch up (i.e., scientific lag = cultural lag + time until research methodologies using those technologies are implemented). And, technologies often don’t remain static, but rather evolve making it a continuous game of catchup (development) for research methodologists. I’ll go into more detail about this in a presentation I’m giving at the AAPOR conference this year, but one quick example from the annals of survey research history is the development of computer-assisted telephone interviewing (CATI). While telephone exchanges had existed for almost a century and programmable computers emerged in the 1940s, it took until 1971 for CATI systems to be developed by market researchers, another five years for academic researchers to begin using it, and the federal government another seven years to implement its use. Certainly, cultural lag played a role. It took years for enough households to have telephones for probability based telephone sampling to make sense. In addition, it took time for the programmable computer to develop into a device usable for this purpose. But, it also took researchers time to figure out such a system was possible and the value it presented.

Now, let’s fast forward a bit. In 1997, one of the first SNSs, sixdegrees.com, was created.  It lasted until 2001. A host of other networking sites, the ones most of us are familiar with, sprang up in the early 2000s—Myspace (2003), Facebook (2004), and Twitter (2006). There are, I suppose, two ways of looking at the cultural and scientific lags and SNSs. On the one hand, it took a few years SNSs to grow to significant numbers. For example, it took Facebook four years (2004 – 2008) to grow to 100 million users. Within four years of that development there were multiple presentations at AAPOR on the subject. That’s certainly much faster than the development of CATI technology/adoption. On the other hand, social researchers took nearly a decade from the birth of widely popular SNSs to begin formally recognizing their research utility.

Now, we may be at the cusp of another such tsunami of consumer technology adoption. Groups disagree on the exact timing (e.g., Forbes says 2014 and MIT Technology Review says 2013), but the evidence points to the start of rapid growth in the use of internet connected sensors and devices for a multitude of purposes. I’ve recently written about how and why I think the devices and the IoT will affect social science data collection.

My question is whether the research community can be more proactive, and therefore decrease the scientific lag between adoption and research implementation. My hope is we will and that it will have a positive effect on survey data collection.

I’ll be presenting more thoughts on this topic at AAPOR and look forward to the discussion we have about big data in the session. Between now and then I’d welcome the thoughts others have on or experiences others have had with using wearable tech, sensors, or the IoT for research.

This was first posted on Survey Post  on 24/02/14

Brian Head is a research methodologist at RTI International with 5 years of experience in the government and not-for-profit research sectors.  Training in sociology and research methods and statistics led him to a career in research where his work has included questionnaire design and evaluation,  managing data collection efforts, and qualitative and quantitative data analysis.

Share