Statistician Brian Blais has attempted to criticize Tim's and my use of a full partition between R and ~R (the resurrection and no resurrection) in evaluating the evidence for the resurrection. He has called this "inflating the Bayes factor with nothing." I intend to do a video series on my Youtube channel on various skeptical attempts to avoid using a partition, which is the actually confusing approach to modeling the impact of evidence. That series will talk about other skeptics who have tried something similar and what's epistemologically wrong with doing so.
Meanwhile, I thought I'd rescue from obscurity in a comments thread some comments I made in immediate response when Blais posted his criticisms. These are a little unorganized, but here they are for now.
There actually is no problem with this analysis (using a full partition), significant or otherwise. You simply prefer, for unknown reasons, to analyze an empirical inference in what to me seems like a confusingly roundabout way, in which one evaluates the specific evidence set E for some salient hypothesis H (e.g., testimony to some event, alleged video of that event, or what-not) by comparing the prior probability of H to the prior probability of some highly gerrymandered theory, ~H1, which is a subhypothesis of ~H specifically generated to give probability to the entirety of the set E that is equal to its conditional probability on H.
It is not even clear that there always exists such a ~H1 (unless one simply considers a non-explanatory declaration "~H and E, somehow" to be a hypothesis), and such a procedure is absolutely riddled with opportunities for epistemic confusion. In contrast, the odds form of Bayes' Theorem, so far from being misleading, yields the correct answer even in your "exercise for the reader" and does so with ease.
There you create a distribution in which the prior odds are 99/1
against H and the Bayes factor is 39/1 in favor of H from some evidence or set
of evidence E. The use of the odds form shows immediately that of course E
increases the probability of H but is not sufficient to swamp the prior odds.
We have there posterior odds of 39/99 (H over ~H), and an extremely simple
calculation (39/(99 + 39)) yields the posterior probability of H as approximately
.28.
I have actually seen a professional probability theorist confused by not using a partition in evaluating the evidence for miracles, to
the point that he switched between a naturalistic hypothesis
which was the conjunction of a set of theories specifically evolved to account
for the evidence for the resurrection (and frankly, even so it didn't do a good job!) and the individual conjuncts, or naturalism more broadly. In other
words, he treated the statement that a miracle has a low prior probability as
equivalent to saying that any naturalistic hypothesis, however convoluted and
consisting of however many independent parts conjoined together, evolved to
account for the evidence for the miracle, will always have a higher prior
probability than that of the miracle.
Needless to say, "Naturalism" is not the same thing as "this incredibly specific conjunctive naturalistic hypothesis that we dreamed up in a blatantly ad hoc manner so as to try to account for the evidence for this specific miracle." I believe that the failure to think in terms of partitions is at least partly to blame for this sort of error, which is rife.
Finally, I stress that I deem the full partition Bayes factor analysis to be the most natural way to conceive of evidence for any specific event for which there is specific evidence. When we say, "The evidence for a moon landing is very strong," a very natural way to think of that is as a reference to the specific evidence (videos, interviews, etc.) that man actually landed on the moon, not to more general evidence such as a knowledge of the state of technology at the time and whether it would have been good enough for a moon landing. The latter is relevant to the prior. The former is the specific evidence for the event.
I submit that considering a Bayes factor using a partition makes a lot more sense than instead saying, "How does the prior probability of a moon landing compare to the prior probability of no moon landing plus a conspiracy hypothesis specifically gerrymandered to account for this whole set of evidence that we have?" If there actually were a subhypothesis of the negation that gave identical probability to the whole of the specific set E to that given by H (and I question that), and if one were able to think about it with perfect clarity (also unlikely) and deduct from the prior of the subhypothesis all that needed to be deducted for the various ad hoc embellishments one would need to make, the final result would be the same. But I can't imagine why anyone would try to go about it in such a way, and I submit that if we were considering a variety of mundane events for which we have specific evidence, no one would think of preferring such a method.
One more point: There is a longstanding and lively debate on the best way to measure evidential force. Various methods are suggested, such as measuring the difference between the prior and the posterior of H--P(H|E) - P(H). Another proposed measure, sometimes called the r measure, is P(H|E)/P(H), and there are more candidates. Tim and I favor the likelihood ratio, sometimes called the L measure, which is P(E|H)/P(E|~H). Sometimes probability theorists use the log of the L measure.
In any event, I don't know of anyone who suggests that we should measure the force of evidence in the round-the-barn fashion that Brian is suggesting: "Take a highly ad hoc version of ~H, assuming that you can find one, that gives precisely the same conditional probability to the set of evidence E that H gives to E, while leaving the probability of E given all the rest of the partition as 0, then somehow or other calculate the prior probability of this subhypothesis ~H1, then calculate the posteriors of H and ~H as constituting a new partition in the same ratio as the priors of H and ~H1. Good luck with that."
The whole point of finding a good measure of the force of specific evidence E is to figure out the most intuitive way to measure what E actually does for or against H qua evidence, not to try as hard as possible to make it look like E isn't doing anything. Part of the reason we favor the L measure (an extremely well-known measure, favored by plenty of people who wouldn't be caught dead being Christians) is because it separates out the power of the specific evidence so cleanly from separate considerations that feed into the priors. In any event, our use of the L measure has precisely nothing to do with weird religious apologists engaging in some sort of probabilistic skulduggery.
Posts
