Observation selection theory and cosmological fine–tuning

Abstract

When our measurement instruments sample from only a subspace of the domain that we are seeking to understand, or when they sample with uneven sampling density from the target domain, the resulting data will be affected by a selection effect. If we ignore such selection effects, our conclusions may suffer from selection biases. A classic example of this kind of bias is the election poll taken by the Literary Digest in 1936. On the basis of a large survey, the Digest predicted that Alf Langdon, the Republican presidential candidate, would win by a large margin. But the actual election resulted in a landslide for the incumbent, Franklin D. Roosevelt. How could such a large sample size produce such a wayward prediction? The Digest, it turned out, had harvested the addresses for its survey mainly from telephone books and motor vehicle registries. This introduced a strong selection bias. The poor of the depression era — a group that disproportionally supported Roosevelt — often did not have phones or cars. The Literary Digest suffered a major reputation loss and soon went out of business. It was superseded by a new generation of pollsters, including George Gallup, who not only got the 1936 election right, but also managed to predict what the Digest’s prediction would be to within 1%, using a sample size that was only one-thousandth as large. The key to his success lay in his accounting for known selection effects. Statistical techniques are now routinely used to correct for many kinds of selection bias.