The odds form of Bayes's Theorem [Updated]

It's been brought to my attention that an atheist styling himself some sort of probability expert has been going about implying that Tim and I are deceptive or slippery in our presentation of our argument for the resurrection, that I was misleading in my interview with Luke Muehlhauser on Common Sense Atheism, that our argument for the resurrection in our Blackwell anthology paper is worthless, and heaven knows what else.

The problem seems to stem from the fact that our article focuses on the likelihood ratio (known as the Bayes factor) for the evidence we adduce rather than on the prior probability of the resurrection.

First of all, for the record: No, we were not remotely deceptive or misleading about this in the article. We were painfully explicit about it. (The entire preprint text of the article is available on-line here.) Viz.:

Even as we focus on the resurrection of Jesus, our aim is limited. To show that the probability of R given all evidence relevant to it is high would require us to examine other evidence bearing on the existence of God, since such other evidence – both positive and negative – is indirectly relevant to the occurrence of the resurrection. Examining every piece of data relevant to R more directly – including, for example, the many issues in textual scholarship and archeology which we shall discuss only briefly – would require many volumes. Our intent, rather, is to examine a small set of salient public facts that strongly support R. The historical facts in question are, we believe, those most pertinent to the argument. Our aim is to show that this evidence, taken cumulatively, provides a strong argument of the sort Richard Swinburne calls “C-inductive” – that is, whether or not P(R) is greater than some specified value such as 0.5 or 0.9 given all evidence, this evidence itself heavily favors R over ~R.

and

But our estimated Bayes factors for these pieces of evidence were, respectively, 10^2, 10^39, and 10^3. Sheer multiplication through gives a Bayes factor of 10^44, a weight of evidence that would be sufficient to overcome a prior probability (or rather improbability) of 10^–40 for R and leave us with a posterior probability in excess of 0.9999.

In my interview with Luke M., I said this (transcribed from the podcast, available here, at approximately 16:10 to 19:30):

In Bayesian terms, what we do in the article is that we try to separate what...one might call...the indirect evidence, which would be relevant to that prior probability, from the direct evidence. So the things that would be relevant to the prior probability would be things like evidence for and against theism, for example, evidence for and against the existence specifically of the God of Israel, the God of the Jews, or other evidence prior to Jesus' purported resurrection regarding who Jesus was, and so forth. That would all be relevant to the prior. And what we focus on in the article instead is what we might call the direct evidence, the evidence that supposedly tells you what happened, what you might call reports...You might call it evidence after the fact. So what we focus on are the testimony of the disciples and of certain women that said that they saw and spoke with Jesus, the evidence of the disciples' willingness to die for that testimony, and the evidence of the conversion of the Apostle Paul. And what we try to do is we use a modeling device known as a Bayes factor. Roughly speaking, a Bayes factor tries to model, number one, which way the evidence is pointing and, number two, how strongly the evidence is pointing that way. And what you're trying to do at that point is you're trying to look at explanatory resources of the hypothesis, in this case, the resurrection, and the negation of the hypothesis. How well does each of these explain the evidence, and is there a big difference between how well each of these explains the evidence? I should clarify that when I say a difference, too, it's actually a ratio...it's very important that you measure it by the ratio, not by the difference. But you need to look at those two hypotheses and see which one gives you a better expectation of that evidence and how much better is that expectation. So we estimate Bayes factors for these various separate pieces of evidence, then we argue for the legitimacy of multiplying these Bayes factors, because that gives you a lot of kick, and you have to discuss that issue, and we do, of independence, and whether it's legitimate to multiply them in order to combine those Bayes factors, and that ends up with this very high, high combined Bayes factor in our estimate...And so what we estimate is that you could have this overwhelmingly low prior probability (and I don't actually think that the prior probability is this low. I think it's low, but I don't think it's this low) of 10^-40 and still give a probability to the resurrection in excess of .9999. And we don't get to that by saying in fact the evidence gives us a posterior probability in excess of .9999. We just say, well this is the power of the...combined Bayes factor, and a combined power that great could overcome this great of a prior improbability and would give you this high of a posterior probability. So that's the basic method.

This is all exceedingly clear: We were arguing for a certain magnitude of confirmation of the resurrection by the evidence we adduce.

I understand that the current atheist meme on this, which shows a rather striking lack of understanding of probability, is to say that if one does not argue for a particular prior probability for some proposition, one literally can say nothing meaningful about the confirmation provided by evidence beyond the statement that there is some confirmation or other.

This is flatly false, as both the second of the quotations above from the paper and my rather detailed explanation to Luke M. show.

Let me try to lay this out, step by step, for those who are interested:

The odds form of Bayes's Theorem works like multiplying a fraction by a fraction--a fairly simple mathematical operation we all learned to do in grammar school (hopefully).

The first fraction is the ratio of the prior probabilities. So, let's take an example. Suppose that, to begin with (that is, before you get some specific evidence) some proposition H is ten times less probable than its negation. The odds are ten to one against it. Then the ratio of the prior probabilities is

1/10.

Now, the second fraction we're going to multiply is the ratio of the likelihoods. So, for our simple example, suppose that the evidence is ten times more probable if H is true than if H is false. The evidence favors H by odds of 10/1. Then the ratio of the likelihoods (which is also called a Bayes factor) is

10/1.

If you multiply

1/10 x 10/1

you get

10/10.

The odds form of Bayes's Theorem says that the ratio of the posterior probabilities equals the ratio of the priors times the ratio of the likelihoods. What this means is that in this imaginary case, after taking that evidence into account, the probability that the event happened is equal to the probability that it didn't: what we would call colloquially 50/50. (You'll notice that the ratio 50/50 has the same value as the ratio 10/10. In this case, that's no accident.)

Okay, now, suppose, on the other hand, that the second fraction, the ratio of the likelihoods, is

1000/1. That is, the evidence is 1000 times more probable if H is true than if H is false. So the evidence favors H by odds of 1000 to 1.

Then, the ratio of the posteriors is

1/10 x 1000/1 = 1000/10 = 100/1,

which means that after taking that evidence into account (evidence that is a thousand times more probable if H is true than if it is false), we should think of the event itself as a hundred times more probable than its negation.

See how this works?

What this amounts to is that if we can argue for a high Bayes factor (that second fraction), even if we don't say what the prior odds are, we can say something very significant--namely, how low of a prior probability this evidence can overcome. That is exactly what we say in the second quotation from our paper that I gave above. It is exactly what I explain to Luke M. We say that we have argued for "a weight of evidence that would be sufficient to overcome a prior probability (or rather improbability) of 10^–40 for R and leave us with a posterior probability in excess of 0.9999."

In our paper, we concentrate on the Bayes factor. The Bayes factor shows the direction of the evidence and measures its force. We argue that it is staggeringly high in favor of R for the evidence we adduce. Naturally, the skeptics will not be likely to agree with us on that. My point here and now, however, is that neither in the paper nor in my interview was there a mistake about probability, any insignificance or triviality in our intended conclusion, nor any deception. We are clear that we are not specifying a prior probability (to do so and to argue for it in any detail would require us to evaluate all the other evidence for and against the existence of God, since that is highly relevant to the prior probability of the resurrection, which obviously would lie beyond the scope of a single paper). Nonetheless, what we do argue is, if we are successful, of great epistemic significance concerning the resurrection, because it means that this evidence is so good that it can overcome even an incredibly low prior probability.

I trust that this is now cleared up.

Update: See also this discussion of Bayesian probability and Richard Carrier at Victor Reppert's blog, here.

Update 2: See the comments thread. Luke and Richard have both apologized for their comments in the interview, and I do accept those apologies.