Epistemic Foundations: Why Methods Matter

When a newspaper reports that youth unemployment in a region rose by six points last year, and a sociologist reports the same fact in a peer-reviewed article, the two statements are, at first reading, indistinguishable. The numerical claim is identical, the region is the same, the year is the same. What is different is the epistemic scaffolding behind the claim — how the number was produced, what it is taken to represent, how much confidence it can bear, and what it can be used to support in further argument. The sociologist's claim is embedded in a disciplined process of measurement, comparison, and inference; the journalist's claim typically is not, and is not intended to be. The difference is not that one is true and the other false. The difference is that one is warranted in a specific sense that the other is not (Weber 1904, pp. 50-58).

A first module in any introduction to sociology needs to address this difference directly, because the value of everything the discipline produces depends on it. Sociology is not primarily a discipline of striking observations or well-told stories, though it sometimes generates both. It is a discipline organized around the problem of going from particular observations to warranted claims about populations, mechanisms, or processes that extend beyond the particular case. How it does this — the methods it uses and the forms of reasoning it authorizes — is what distinguishes sociological knowledge from journalism, from commonsense observation, and from anecdote.

Reliability, validity, generalizability. Three technical concepts organize the sociologist's toolkit for thinking about whether a claim is warranted. Reliability asks whether the same measurement procedure, applied again, would produce the same result — whether the observation is stable rather than an artifact of a particular moment, researcher, or instrument. Validity asks whether the measurement is actually measuring what it is taken to measure — whether a self-reported survey item about political trust, for example, tracks the underlying construct rather than some adjacent attitude (Carmines and Zeller 1979, pp. 11-15). Generalizability, sometimes called external validity, asks whether a finding from a particular sample, site, or time extends to other cases of interest — whether a pattern observed in one city's schools is likely to hold in schools elsewhere (Shadish, Cook, and Campbell 2002, pp. 83-86).

Journalism, commonsense observation, and anecdote can all be insightful. They are, for that reason, often the starting point of sociological work. What they do not provide, on their own, is the scaffolding that turns insight into a warranted claim. A journalist interviewing three unemployed workers in a single town may produce vivid, accurate, and morally important reporting. The sociologist's distinct contribution is to ask: is this pattern present in the population, how widely, and by what mechanism, and with what confidence (Mills 1959, pp. 19-25)?

From particular to population. The characteristic move of sociological inference is from particular observations, whether quantitative or qualitative, to claims about populations or mechanisms. This move requires discipline. A single case, however vividly described, does not warrant a population-level claim; a population-level statistic, however large, does not warrant a claim about what produces it. The methods that occupy the rest of this module — survey research, experimental and quasi-experimental designs, ethnography, comparative-historical analysis, computational methods — are best understood as different disciplined routes for making this move, each with its own characteristic strengths, characteristic blind spots, and characteristic standards for what counts as a warranted conclusion (Lieberson 1985, pp. 3-9).