Why Data Tracking Provides a Much Less Accurate Picture of Media Exposure than Often Assumed
Most of what we know about politics, we know from the media. We get most of our information about political issues, actors, and opinions there, which makes the media an important influencing factor on public opinion. Nowadays, this applies also to social media to an increasing degree. Social media have become a very important news source all over the world, as shown by the Reuters Digital News Report. Therefore, exposure to news media (and meanwhile also to social media) is a very important variable in public opinion research. If we want to investigate processes of public opinion formation, we need to know which media a person uses when, how often, and how long. Conventionally, media exposure has been (and still is) measured in surveys, asking the users for example for the information sources they use on a regular basis or for the information sources they used for how long on the previous day. It is obvious that the reliability of such self-reports is limited. People often don’t remember their whole media exposure, and it is highly probable that they are even not aware of parts of it. Worse, the parts they remember and those that are forgotten might differ systematically, resulting in a wrong overall picture. In the digital news environment, a new – and at first glance better – opportunity of measuring media exposure has emerged, namely data tracking. Data tracking enables researchers to record the online activities of users, allowing for a more precise and differentiated picture of media exposure. Thus, many consider data tracking by far superior to surveys. It looks like data tracking provides us with the opportunity of getting the whole picture of media exposure. However, upon closer examination, it turns out that tracking brings along its very own methodological problems, particularly when investigating social media exposure.
Tracking measures potential, not actual media exposure
Let’s start with some general limitations of measuring media exposure by use of data tracking. First, data tracking can only cover media exposure that happens online, leading to an underestimation of overall media exposure. Many users are, for example, still used to watch news sitting in front of the TV set in the living room or to read news in a printed newspaper at the breakfast table – media exposure that will be excluded from the tracking results. This problem is particularly important in the many geographical regions where by far not everyone has internet access (see Internet penetration rate worldwide). Second, neither having turned the computer or the smartphone on nor having opened a website does necessarily mean that a person actively receives any content. For example, it often happens that a website is running in the background, not being recognized by the user, due to manifold reasons. Nevertheless, this website will be tracked, leading to an overestimation of media exposure. In other words: tracking measures potential, not actual reception.
Tracking results in biased data
Third, people access the Internet by use of many different devices, for example a laptop at home, a computer at work, a smartphone or a tablet en route and so on. Ideally, in order to not underestimate media exposure, we would need to get a full picture of a single persons’ online consumption by getting and merging tracking data from all different devices this person uses. However, this is a technically enormously difficult undertaking. Fourth, data tracking brings along ethical considerations because it penetrates deep into the users’ privacy, which is why the users’ informed content should be a necessary precondition of it. However, the users who will agree with tracking their data are probably a very specific sample differing from the rest of the population concerning characteristics like an over-average time spent online and a below-average online privacy concern, resulting in sampling bias. In addition, the knowledge that online consumption is tracked can change a person’s online consumption behavior, potentially further increasing the bias of the data.
We can’t track what people actually do on Facebook
In addition to these general problems of tracking, there is one more specific problem when it comes to tracking social media consumption. Most tracking tools allow for recording time spent on a certain domain, for example facebook.com – but it is not possible to track the users’ activities within these platforms. In other words, we can track that someone uses Facebook in general, but we are unable to find out which specific Facebook pages, groups, or even posts and comments this person visits. Thus, we are unable to find out if this person receives, for example, news, entertainment, or more private information on Facebook. And if we don’t know which content users receive on Facebook, it is quite difficult to investigate Facebook’s role for public opinion formation. In this special case, it is highly probable that survey data are by far superior to tracking data.
Tracking is by far less reliable than it seems
These methodological problems make data tracking by far less reliable for investigating media exposure as it seems at first glance. Thus, we should be very careful with considering tracked data an unbiased source of “true” media exposure by default. Rather, we should view them as a useful complement to survey data which remain important sources for public opinion research, even in the age of social media. Ideally, we should work on merging tracking and survey data which would provide us with outstanding possibilities of mutually validating both types of data. However, such attempts will bring along new methodological pitfalls. At the end, we still aren’t (and probably never will be) able to draw a one-to-one picture of media exposure – but at the same time, for most of our questions, the data we have is always good enough.
Read more about the methodological pitfalls of data tracking in our article.
Melanie Magin
Melanie Magin (M.A., Dr. phil.) is an Associate Professor in Media Sociology at the Department of Sociology and Political Science at NTNU in Trondheim. She is a member of various research groups, such as Elections, Values and Political Communication (EVPOC) and Digitalization and Social Life.
