Conditional Probability Fallacy

From SkepticWiki

The Conditional Probability Fallacy is the assumption that the probability of B, given A is roughly equal to (or directly related to) the probability of A, given B.

The fallacy is notoriously found in jury-trial arguments, political argument, sales pitches, and pseudoscientific tests, where it may (intentionally or not) confuse non-statisticians into reaching conclusions not necessarily supported by the facts.

[edit] Illustration

For example, suppose that when a dowser identifies a likely location, that underground water is found 80% of the time. The Conditional Probability Fallacy would lead to the unfounded conclusion that 80% of underground water is detectable by the dowser.

As another example, suppose that a fuel additive manufacturer claims (truthfully) that 95% of cars will pass an emissions certification when using the product. It would be fallacious to assume, without further information, that the use of the additive increases, decreases, or has any effect at all on the test.

[edit] Conditional Probability

Consider the case of a hypothetical dowser, who claims 80% accuracy, meaning that 80% of “hits” indicate underground water. Letting A represent a “hit” event at a particular location, and B represent the occurrence of water at a location, the notation P(B|A)=80% represents this fact.

Further, suppose that only 20% of locations with no water will cause a “hit”: P(B|~A) = 20%.

What we may wish to know is the probability P(A|B): Given that there is water, how likely is the dowser to find it? The Conditional Probability Fallacy may lead one to conclude that this is at or near 80%. Yet, the following possible scenarios show that P(A|B) may be higher, lower, or equal to P(B|A):

• Scenario 1: P(A|B) = 95%:

Water, Hit: 66.1%

Water, Miss: 3.5%

Dry, Hit: 16.5%

Dry, Miss: 13.9%

• Scenario 2: P(A|B) = 16%

Water, Hit: 3.6%

Water, Miss: 19.1%

Dry, Hit: 0.9%

Dry, Miss: 76.4%

• Scenario 3: P(A|B) = 80%

Water, Hit: 40%

Water, Miss: 10%

Dry, Hit: 10%

Dry, Miss: 40%

So even if a dowser claims an 80% success rate, and a 20% “false positive” rate, there is still no way to gauge their true water-detecting ability, as all these varied scenarios are consistent with the claimed rate. To determine which scenario we are looking at we would need to know P(B), the probability that water is in any given location, and P(A), the probability that a given location is hit. In fact, the Conditional Probabilities are related by the following formula:

P(B|A) = P(A|B) * P(B) / P(A).

[edit] Real Examples

• "Women are much more likely than men to be killed by an intimate partner. In 2000, intimate partner homicides accounted for 33.5 percent of the murders of women and less than four percent of the murders of men."[1]
• “According to Justice Department statistics and the analysis of immigration experts, the [claim that illegal-immigrants are law-abiding] often isn't true. …Some estimates show illegals now make up half of California's prison population...”[2]