Focused Correlation, Confirmation, and the Jigsaw Puzzle of Variable Evidence Philosophy of Science, 78 (July 2011) pp. 376–392. 0031-8248/2011/7803-0003$10.00 Copyright 2011 by the Philosophy of Science Association. All rights reserved. 376 Focused Correlation, Confirmation, and the Jigsaw Puzzle of Variable Evidence* Maximilian Schlosshauer and Gregory Wheeler†‡ Focused correlation compares the degree of association within an evidence set to the degree of association in that evidence set given that some hypothesis is true. Wheeler and Scheines have shown that a difference in incremental confirmation of two evidence sets is robustly tracked by a difference in their focus correlation. In this essay, we generalize that tracking result by allowing for evidence having unequal relevance to the hypothesis. Our result is robust as well, and we retain conditions for bidirectional tracking between incremental confirmation measures and focused correlation. 1. Introduction. Auditors have found an irregularity in the books for Acme, Inc. Bungled paperwork explains most anomalies of this kind, but fraud cannot be ruled out, so teams are assembled to investigate. Team 1 learns that Carson, a longtime Acme bookkeeper, is divorcing his wife, which is sad news for the Carson family but of little interest to Team 1. Most people manage a divorce without defrauding their employer, after all. Team 2 learns that Carson has a Brazilian lover, but they merely pause to think, lucky Carson. Team 3 finds out that Carson took a flight to Rio de Janeiro seated beside a stunning but dodgy Brazilian financier. Considered in isolation, each team’s evidence signals little more than middle-life boredom. Yet, when taken together, the evidence suggests fraud. It is not the facts alone that warrant hauling Carson in for ques- tioning but rather how those pieces of evidence fit together into a com- pelling case against him. But what does it mean for evidence to ‘fit together’, and what is it *Received April 2010; revised August 2010. †To contact the authors, please write to: Gregory Wheeler, Center for Artificial In- telligence, New University of Lisbon, Portugal; e-mail: grw@fct.unl.pt. ‡This work was supported by the European Science Foundation, the Portuguese Science and Technology Foundation, and the Danish Natural Science Research Council. mailto:grw@fct.unl.pt CORRELATION, CONFIRMATION, AND VARIABLE EVIDENCE 377 about combined evidence that lends more support to a hypothesis than each piece of evidence on its own? One answer is to compare the likelihood of evidence occurring together to its co-occurrence given a cause or some unifying explanation. The events of the story are unusual on their own, but a conspiracy to defraud Acme involving Carson provides a unifying explanation for those events, one that would be alarming enough for Acme investigators to want a word with Mr. Carson. The core idea animating this example is encapsulated in a measure called focused correlation (Myrvold 1996; Wheeler 2009), which is defined for n binary evidence variables and a single binary hypothesis variable as follows: n!1. . .P(HFE ! ! E )P(H )1 nFor (E , . . . , E ) pH 1 n . . .P(HFE ) # # P(HFE )1 n . . . . . .P(E ! ! E FH )/P(E FH ) # # P(E FH )1 n 1 np . (1). . . . . .P(E ! ! E )/P(E ) # # P(E )1 n 1 n Focused correlation has been given a variety of interpretations: to capture the effect that diversity of evidence has on confirmation of some hy- pothesis (Myrvold 1996); to provide a partial explication of the relation- ship between epistemic coherence and incremental confirmation of a hy- pothesis (Wheeler 2009); and, in normalized form, to provide an account of unified evidence for a hypothesis (Myrvold 2003). But, the basic idea is that by comparing the degree of association within an evidence set to the degree of association in that evidence set given some hypothesis, in certain circumstances several questions about confirmation relationships between evidence and that hypothesis can be answered. The conditions under which focused correlation tracks confirmation, pace Myrvold (2003), depend on what type of logical or causal structure governs the relationships between evidence and hypothesis in a problem (Wheeler and Scheines 2011). For example, Erik Olsson (2005) and Luc Bovens and Stephan Hartmann (2006) have drawn attention to problem- atic features of models in which all binary evidence variables Ei of a problem are conditionally independent of the binary hypothesis variable H; that is, . The correlation between any distinct pairGi ( j, E !! E FHi j of evidence variables Ei and Ej in this class of models is the product of the correlations between each E and H (Danks and Glymour 2001). But this fact yields counterintuitive results if one interprets association as epistemic coherence since then association among the evidence is strictly less than the (nondeterministic) confirmation that each piece of evidence lends to the hypothesis. In the case of focused correlation, it is easy to see from equation (1) that the focused correlation of an evidence set with respect to some hy- 378 MAXIMILIAN SCHLOSSHAUER AND GREGORY WHEELER pothesis under this conditional independence assumption is strictly less than one, and thus is negative, whereas the confirmation of the hypothesis by that evidence may be positive. This conditional independence condition is central to the witness testimony models of Bovens and Hartmann (2003) and Olsson (2005), and it is the condition that drives their noted negative results concerning the construction of a probabilistic measure of coherence (Wheeler 2009). But, contra Olsson (2005, 135), there are interesting cases outside of this class of models where a measure like focused correlation can be useful, and, pace Luc Bovens and Stephan Hartmann (2006), there are conditions under which a difference in confirmation lent to some hypothesis by two evidence sets is tracked by a difference in their focused correlation, even within this obstinate class of models.1 To illustrate, consider again L’affaire de Carson. Suppose that a performance review of the Acme Investigation Department involves comparing the evidence set E, consisting of a di- vorcing employee who traveled with a dubious associate, to the evidence set E*, consisting of a divorcing employee who is having an affair. Acme is a big company with many cases on file, and their models suggest that E is a better indicator of fraud than E* because fraud is a slightly ‘better’ explanation for the co-occurrence of divorce and suspicious travel than it is for divorce and an adulterous affair. Under what conditions can Acme management draw this inference? In Wheeler and Scheines (2011), sufficient conditions for a wide variety of incremental confirmation measures are given, under which a difference in focused correlation entails a difference in confirmation lent to that hypothesis and vice versa. But their results are only trivially true in this problematic class of models because one of the conditions for the com- parison result is that all individual items of evidence have equal relevance to the hypothesis. In effect, this restrictive condition, when combined with the conditional independence assumption mentioned above, deals with the problematic cases driving Bovens and Hartmann’s counterexamples (2003, 2006), by ensuring that at least one of the antecedent conditions is false. The motivation for this strong assumption is clear in their project, for they address whether a difference in association of two evidence sets induced by some hypothesis can track confirmation of that hypothesis. 1. Observe that this problematic class of models may be characterized by those in which this conditional independence condition holds. Hence, that class of models— provided that the assumptions needed to activate the theory of graphical causal models are satisfied—can be identified nonparametrically and handled by other means. So, while it is true that no probabilistic measure of coherence can operate correctly within this class, that is only one way to look at it; another is to observe that there is no need for a coherence measure in this class of models. CORRELATION, CONFIRMATION, AND VARIABLE EVIDENCE 379 But including equal relevance as a ceteris paribus condition will not do for practical comparisons of evidence sets; it is much too restrictive. In this article, we show that the equal evidence assumption can indeed be relaxed to identifiable cases involving unequal relevance. Our result is robust as well, and we retain conditions for bidirectional tracking between the class of incremental confirmation measures and focused correlation. The cases in which tracking fails are instructive, suggesting that focused correlation is a stronger measure of evidential support than classical in- cremental confirmation measures, and we illustrate this point through some examples. The structure of the article is as follows. In section 2, we set up the machinery by making explicit our notation and the confirmation measures we consider. Section 3 presents our results, and in section 4, we explain why two common nonincremental confirmation measures are not tracked by focused correlation. This analysis offers further evidence for distin- guishing between classical, incremental confirmation measures and other notions of confirmation. Proofs are set in the appendix. 2. Setup. In this section we specify notation and list the confirmation functions we will consider. Define a probability space such that is a j-algebra over a(Q, F, P) F set Q, and is a probability measure defined on the spaceP:F r [0,1] satisfying(Q, F, P) P1. .P(Q) p 1 P2. , when are countable, pairwise disjoint ""P(" X ) p ! P(X ) Xip1 i i iipi elements of .F A probability structure is a tuple , where is aM p (Q, F, P, V ) (Q, F, P) probability space and V is an interpretation function associating each element with a truth assignment on the primitive propositions inq ! Q such that for each and for everyA, B, . . . ! F V(q)(A) ! {1, 0} q ! Q proposition in F. For each primitive proposition in F, we define if and onlyM, q X A if (iff ) and proceed by induction on the structure of prop-V(q)(A) p 1 ositional formulas. Since P is defined on events in rather than prop-F ositions, let denote the set of outcomes within Q in M where A is[[A]] M true, which will correspond to a subset of . The following equationsF make explicit the relationship between propositions and events for arbi- trary propositional formulas A and B: i) ,[[A " B]] p [[A]] ! [[B]]M M M ii) ,[[A # B]] p [[A]] " [[B]]M M M iii) .[[¬A]] p [[A]]M M 380 MAXIMILIAN SCHLOSSHAUER AND GREGORY WHEELER An evidence set is a set of propositions, written . We definedE p {E , E }1 2 focused correlation in equation (1) with respect to random variables Ei and H each taking values 0 or 1. We use the abbreviation E for E p 1 and for and likewise will use the abbreviation¬E E p 0 P(E " ¬1 for . We will relax notation by express-E FH ) P(E p 1 ! E p 0FH p 1)2 1 2 ing focused correlation of propositions as well as of variables, using the conventions here for distinguishing between propositions and variables to signal which is which. 2.1. Confirmation Measures. A statement E is confirmation to hy- pothesis H with respect to a classical probability model specifying the measure P just when E and H are positively correlated under P; that is, when . A confirmation function measures the degree toP(HFE ) 1 P(H ) which evidence confirms a hypothesis, and there are several proposals:2 inc (H, E , E ) :p P(HFE " E ) ! P(HFE ).1 1 2 1 2 1 P(HFE " E ) ! P(HFE )1 2 1inc (H, E , E ) :p .2 1 2 1 ! P(HFE )1 P(EFH ) ! P(EF¬H ) ko(H, E) :p . P(EFH ) # P(EF¬H ) P(HFE) r(H, E) :p log . P(H ) P(EFH ) l(H, E) :p log . P(EF¬H ) 2.2. Comparing Equal-Relevance Evidence Sets. In our discussion through- out, we assume that , a probability distribution defined over a domainP(D) of propositions , is positive. In addition, we may appeal to twoAH, ES conditions. 2. Measures and are both variants of L. Jonathan Cohen’s notion of incre-inc inc1 2 mental convergence (Cohen 1977): reports the contribution that simpliciter makesinc E1 2 to H, whereas reports the contribution makes with respect to the possible availableinc E2 2 evidence; and are the two cases that comprise measure Z (Crupi, Tentori, andinc inc1 2 Gonzalez 2007), where is used if , and is used oth-inc (H, E) P(HFE) 1 P(H) inc (H, E)2 1 erwise. The Kemeny and Oppenheim fitness measure is ko (1952); r is a generic relevance measure, versions of which have been endorsed from Keynes (1921) to Milne (1997), among others; l is ordinally equivalent to ko; and ko, r, and l are typically discussed for evidence E representing an evidence set of arbitrary size by a single conjunctionE of the propositions in . We will discuss cases where . See Kyburg (1983) forE FEF p 2 an overview of confirmation measures and Eells and Fitelson (2002) for a recent dis- cussion. CORRELATION, CONFIRMATION, AND VARIABLE EVIDENCE 381 (A1) Positive Relevance: All propositions in E are positively relevant to H, just in case .GE ! E, P(HFE ) 1 P(H ) 1 P(HF¬E )i i i (A2) Equal Relevance: All propositions in a set of evidence E are equally confirmatory, just in case , andGE , E ! E, P(HFE ) p P(HFE )i j i j .P(HF¬E ) p P(HF¬E )i j First, evidence and hypothesis may be correlated or anticorrelated, unless independent. Thus, evidence may be either positively relevant or negatively relevant to a hypothesis, if it is relevant at all. Condition A1 restricts attention to cases where evidence is positively relevant. Second, strength of relevance need not be the same, but condition A2 restricts attention to just those cases where all pieces of evidence are equally relevant to the hypothesis.3 This is the condition we shall explore how to weaken. But first, let’s see the benefit from a positive distribution over D that satisfies both A1 and A2. Proposition 1 (Wheeler and Scheines 2011). If andE p {E , E }1 2 , and satisfies A1 and A2 with respect to H,E* p {E , E } E " E*1 3 then all of the following inequalities are equivalent:4 • For (E) 1 For (E*)H H • r(H, E) 1 r(H, E*) • l(H, E) 1 l(H, E*) • ko(H, E) 1 ko(H, E*) • inc (H, E) 1 inc (H, E*)1 1 • inc (H, E) 1 inc (H, E*)2 2 Proposition 1 tells us that focused correlation tracks incremental con- firmation and vice versa, but the proof leans on A2 to isolate the effect of focused correlation on incremental confirmation. This condition is too restrictive to exploit in an application, and, as observed in the introduc- tion, it deals with a problematic class of models by effectively excluding them. In the next section, we consider how to preserve these tracking properties of focused correlation without condition A2. 3. Unequal Relevance. We consider, as before, two evidence sets E p and , but now we shall relax assumption A2: we{E , E } E* p {E , E }1 2 1 3 no longer require that all propositions , , are equally confir-E i p 1, 2, 3i 3. See lemma 2 and remarks in the appendix about the strength of these two as- sumptions when combined. 4. For the proposition to hold for , we stipulate that standsinc inc (H, E) 1 inc (H, E*)1 1 1 for . A similar remark applies for interpreting .inc (H, E , E ) 1 inc (H, E , E ) inc1 1 2 1 1 3 2 382 MAXIMILIAN SCHLOSSHAUER AND GREGORY WHEELER matory. That is, we allow for the degree of positive relevance ofP(HFE )i the individual pieces of evidence to vary. What we would like to check now is whether, in this generalized sit- uation, focused correlation continues to track confirmation, and vice versa. That is, we would like to explore under what conditions proposition 1 still holds. ‘Confirmation’ is here understood as quantified by the mea- sures defined in section 2.1. We shall denote these measures collectively by in what follows; that is, jointly stands for any of thec(H, E) c(H, E) functions , , , , and .inc (H, E) inc (H, E) ko(H, E) r(H, E) l(H, E)1 2 A quick look at the definitions of these confirmation measures (sec. 2.1) shows that they are based only on the probabilities of H conditional on the full evidence set and on the shared evidence piece but not onE1 the probabilities of H conditional on the distinct individual pieces andE 2 , that is, not on and . The appearance ofE P(HFE ) P(HFE ) P(HFE )3 2 3 2 and in equation (1) is therefore a particular feature of focusedP(HFE )3 correlation. This feature exemplifies the additional input to the measure, which takes into account the conditional probabilities for allP(HFE )i .E ! Ei As an aside, we note that the relevance of the evidence willP(HFE ) E1 1 not enter into the following considerations, not withstanding its role in and . This is so because is shared among both evidence setsinc inc E1 2 1 and , and we are here concerned only with a comparison of theE E* degrees of focused correlation and confirmatory power of these two sets. 3.1. From Focused Correlation to Confirmation. Let us first describe the condition under which a larger degree of focused correlation for evi- dence set than for implies larger confirmation of the{E , E } {E , E }1 2 1 3 hypothesis by than by .{E , E } {E , E }1 2 1 3 Proposition 2. If andFor (E , E ) 1 For (E , E )H 1 2 H 1 3 P(HFE ) For (E , E )2 H 1 3 1 , (2) P(HFE ) For (E , E )3 H 1 2 then also .c(H, {E , E }) 1 c(H, {E , E })1 2 1 3 Thus, the confirmation measures track focused correlationc(H, {E , E })1 2 if the bound (2) is fulfilled. In particular, this will be the case whenever , but the implicationP(HFE ) $ P(HFE ) For (E , E ) 1 For (E , E ) %2 3 H 1 2 H 1 3 will also hold if by anc(H, {E , E }) 1 c(H, {E , E }) P(HFE ) ! P(HFE )1 2 1 3 2 3 amount that is a function of the difference in the focused correlations of the two sets and , as specified by (2). However, if dropsE E* P(HFE )2 below this bound, then the tracking fails, and the confirmation measures will indicate less confirmatory power for than , despitec(H, {E , E }) E E*1 2 the fact that has larger focused correlation than .E E* CORRELATION, CONFIRMATION, AND VARIABLE EVIDENCE 383 This observation has an intuitive explanation, as c(H, {E , E }) !1 2 is equivalent to when both A1c(H, {E , E }) P(HFE " E ) ! P(HFE " E )1 3 1 2 1 3 and A2 hold. Now, as mentioned above, focused correlation goes a step further than the measures because it weighs the joint rele-c(H, {E , E })1 2 vance by how much each individual piece of evidence confirmsP(HFE) H. If and individually confirm H very little compared to the con-E E1 2 firmation of H provided by the conjunction , while the probabilityE " E1 2 of H given does not exceed by much the product of the proba-E " E1 3 bilities of H given and alone, then the set may have largerE E {E , E }1 3 1 2 focused correlation than the set , even if{E , E } P(HFE " E ) !1 3 1 2 . In other words, may ariseP(HFE " E ) For (E , E ) $ For (E , E )1 3 H 1 2 H 1 3 from a situation in which is so much smaller than asP(HFE ) P(HFE )2 3 to be able to counteract the greater confirmatory power of over asE* E quantified by the measures .c(H, {E , E })1 2 We will illustrate our results by presenting alternative versions of Mr. Carson’s scandal. In each of our examples, we shall let represent theE1 fact that Carson is divorcing his wife, p Carson’s trip abroad seatedE 2 next to a shady financier, p Carson’s affair with a Brazilian lover,E 3 and H p Carson swindled Acme. We begin by considering the case in which the confirmation measures fail to track focused correlation.c(H, E) Example 1. Suppose that the evidence set {E p divorce, E p travel}1 2 has larger focused correlation than , with re-{E p divorce, E p affair}1 3 spect to the hypothesis H. We interpret this relation as saying that evidence of both Carson’s divorce proceedings and his travel cohere better toward supporting the hypothesis of fraud than evidence of Carson’s divorce and affair. But this fact does not necessarily mean that the coincidence of divorce and travel is also more confirmatory of the hypothesis of fraud than the divorce-plus-affair scenario. Why not? Imagine a backstory such that Carson’s affair, considered in isolation, supports fraud much more strongly than a trip to Brazil, considered in isolation. Perhaps poor Carson is in over his head: his affair is a surprise to all and an expensive one, at that. His trip to Rio, although unusual, pales by comparison, and on its own there is no reason to think that his having a shady seat companion is anything but a coincidence. Rel- ative to this background knowledge, A, . Now addP (HFE ) 1 P (HFE )A 3 A 2 the news of Carson’s divorce proceedings. This would not dent the con- firmatory power of the affair scenario since adding a divorce to that affair only piles on more expenses for Carson. But learning of the divorce would also do little to alter the confirmatory force of the travel scenario on judging whether Carson is complicit in defrauding Acme. Thus, .P (HF{E , E }) 1 P (HF{E , E })A 1 3 A 1 2 384 MAXIMILIAN SCHLOSSHAUER AND GREGORY WHEELER But then there is little reason to believe that we need the coincidence of a divorce and an affair to send up a red flag in Acme’s fraud department, as opposed to just knowing of Carson’s liaison. The coherence of divorce and affair may therefore be less than the coherence of divorce and travel with respect to confirming the hypothesis of fraud, and still the conjunc- tion of divorce and affair may be more confirmatory of fraud than the conjunction of divorce and travel. As we have seen, this happens when the confirmatory power of the affair (considered by itself ) becomes suf- ficiently large in comparison with the confirmatory power of the travel (considered by itself ). When coherence is measured by focused correlation, what qualifies as ‘sufficiently large’ is then precisely quantified by the bound (2). 3.2. From Confirmation to Focused Correlation. Let us now tackle the converse relation, namely, the implication from a difference in confir- mation to a difference in focused correlation. Proposition 3. If andc(H, {E , E }) 1 c(H, {E , E })1 2 1 3 P(HFE ) & P(HFE ), (3)2 3 then also .For (E , E ) 1 For (E , E )H 1 2 H 1 3 The bound (3) is a catchall condition that holds for all confirmation measures considered in section 2.1. It may be tightened by focusingc(H, E) on a particular measure. For example, for and , theinc (H, E) inc (H, E)1 2 bound is P(HFE ) inc (H, E , E )2 k 1 2 ! , k p 1, 2, (4) P(HFE ) inc (H, E , E )3 k 1 3 which allows for within certain limits.P(HFE ) 1 P(HFE )2 3 The existence of a bound on the ratio of to has aP(HFE ) P(HFE )2 3 similar explanation as the bound (2). If by a suffi-P(HFE ) 1 P(HFE )2 3 ciently large amount, then the larger confirmatory power of overE E* may be overshadowed by the fact that the confirmation of H by alone,E 3 with respect to the confirmation by the conjunction , is much lessE " E1 3 than the confirmation of H by alone, with respect to the confirmationE 2 by the conjunction . Or, put more plainly, the beliefs in the setE " E1 2 cohere better toward supporting H than do the beliefs in the set .E* E Example 2. In the spirit of the introduction, suppose that Carson’s affair is viewed as an ordinary disaster, and his travel plans are more suspicious—it is known that he booked both his seat and his com- panion’s, let’s say. So, suppose that the conjunction of divorce and travel is more confirmatory of the hypothesis of fraud than the conjunction CORRELATION, CONFIRMATION, AND VARIABLE EVIDENCE 385 of divorce and an affair on this background knowledge, labeled B: , and .c (H, {E , E }) 1 c (H, {E , E }) P (HFE " E ) 1 P (HFE " E )B 1 2 B 1 3 B 1 2 B 1 3 Now, what happens when we consider focused correlation instead of confirmation? Equation (1) weighs and byP (HFE " E ) P (HFE " E )B 1 2 B 1 3 how much each of the individual pieces of evidence differing between the two evidence sets—that is, Carson’s travel ( ) and affair ( )—confirmsE E2 3 the suspicion of fraud. If condition (3) is violated, that is, if P (HFE ) 1B 2 , this would mean that isolated knowledge of Carson’s travelP (HFE )B 3 makes us more inclined to think of the possibility that he may be engaged in fraudulent business than if we simply heard that Carson is embroiled in an affair. It is the merit of focused correlation to now put up a cautionary flag when it comes to considering the implications for the relative degrees of coherence of the two evidence sets and{E p divorce, E p travel}1 2 . How so?{E p divorce, E p affair}1 3 The argument is similar to the discussion of the previous example in section 3.2. Since according to background B the evidence of travel, con- sidered in isolation, is already more supportive of the hypothesis of fraud than the evidence of an affair, also considered in isolation, and since the only other piece of evidence—namely, the fact that Carson is in the midst of divorce proceedings—is common to both evidence sets, then one should not automatically conclude that it is the coincidence of divorce and travel that boosts our confidence in fraud. Rather, it may be the case that the evidence of travel alone is what makes the hypothesis so eminently plau- sible, not the fact that it coincides with Carson’s divorce. Conversely, while Carson’s affair alone may barely raise suspicions of fraud, its co- incidence with Carson’s being in the midst of a messy—and potentially financially threatening—divorce may boost the likelihood of fraud by a large margin. In other words, although the coincidence of Carson’s divorce and travel is more confirmatory of fraud than the coincidence of his divorce and affair, the latter evidence set may well be judged to have a larger degree of coherence. Focused correlation captures this subtlety. Conversely, if condition (3) is obeyed, that is, if ,P (HFE ) & P (HFE )B 2 B 3 focused correlation and confirmation will gravitate toward favoring the same evidence set. Together with the fact that P (HFE " E ) 1B 1 2 , we can now safely conclude that divorce and travel cohereP (HFE " E )B 1 3 better toward supporting the hypothesis of fraud than the conjunction of divorce and affair: although evidence of travel alone is less (or equally) indicative of fraud compared with the evidence of an affair, when taken together with the evidence of an ongoing divorce, Carson’s travel will ring Acme’s alarm bells more readily than evidence of Carson’s involve- ment with his Brazilian sweetheart. 3.3. Bidirectional Tracking. Propositions 2 and 3 show that when the 386 MAXIMILIAN SCHLOSSHAUER AND GREGORY WHEELER ratio of to is within a certain range, focused correlationP(HFE ) P(HFE )2 3 will be tracked by confirmation, and when it is within another range, focused correlation will track confirmation. It is important to note that these ranges overlap. Propositions 2 and 3 readily quantify this overlap, that is, the range of values of , for which bidirectionalP(HFE )/P(HFE )2 3 tracking holds. Combining propositions 2 and 3 yields: Corollary 1. If For (E , E ) P(HFE )H 1 3 2 ! & 1, (5) For (E , E ) P(HFE )H 1 2 3 then .For (E , E ) 1 For (E , E ) # c(H, {E , E }) 1 c(H, {E , E })H 1 2 H 1 3 1 2 1 3 That is, ensuring that the tracking goes both ways does not require us to impose the rather strong assumption A2 of equal relevance of all bits of evidence. Choosing a particular confirmation measure allows us to extend even further the range over which the bidirectional tracking holds. For example, for or , we can use the bound (4) instead of (3), whichinc (H, E) inc (H, E)1 2 gives a ‘relaxed’ version of corollary 1: Corollary 2. If For (E , E ) P(HFE ) inc (H, E , E )H 1 3 2 k 1 2 ! ! , k p 1, 2, (6) For (E , E ) P(HFE ) inc (H, E , E )H 1 2 3 k 1 3 then .For (E ,E ) 1 For (E , E ) # inc (H, {E , E }) 1inc (H, {E , E })H 1 2 H 1 3 k 1 2 k 1 3 Figure 1 illustrates the different ranges of for whichP(HFE )/P(HFE )2 3 uni- and bidirectional tracking between focused correlation and confir- mation obtains. 3.4. Equality of Focused Correlation and Confirmation. For the sake of completeness, let us also explicitly consider the connection between equal values of focused correlation and equal values of confirmation: Proposition 4. For (E , E ) p For (E , E ) # c(H, {E , E }) pH 1 2 H 1 3 1 2 iffc(H, {E , E })1 3 P(HFE ) p P(HFE ). (7)2 3 In other words, equal values of focused correlation for the two sets E and translate into equal degrees and ofE* c(H, {E , E }) c(H, {E , E })1 2 1 3 confirmation—and vice versa—if and only if the equal-relevance as- sumption A2 is made. The reason for this result should now be clear from the above discussion of propositions 2 and 3. 4. Discussion. In this article, we have demonstrated that the equal-rele- CORRELATION, CONFIRMATION, AND VARIABLE EVIDENCE 387 Figure 1. Regimes of for which focused correlation tracks orP(HFE )/P(HFE )2 3 is tracked by confirmation measures. The relevance of a piece of evidence forEi a given hypothesis H is , and ‘ ’ is shorthand for the implicationP(HFE ) For % ci , which represents aFor (E , E ) 1 For (E , E ) % c(H, {E , E }) 1 c(H, {E , E })H 1 2 H 1 3 1 2 1 3 tracking of focused correlation by the confirmation measures . Similarlyc(H, E) for ‘ ’. Over a range of values for , bidirectional track-c % For P(HFE )/P(HFE )2 3 ing obtains (bold box); that is, For (E , E ) 1 For (E , E ) # c(H, {E , E }) 1H 1 2 H 1 3 1 2 . For specific choices of , for example, for the measuresc(H, {E , E }) c(H, E)1 3 or shown here, the tracking range is extended further.inc (H, E) inc (H, E)1 2 vance assumption can be relaxed within certain limits without upsetting the bidirectional tracking between focused correlation and several incre- mental confirmation measures; if only unidirectional tracking is required, these limits become even less stringent. However, if the degrees of relevance diverge too strongly, the tracking fails. We have shown that the ratio of to is crucial in assessing, both qualitatively and quan-P(HFE ) P(HFE )2 3 titatively, when and where the tracking fails. However, this observation should not be viewed as reflecting a failure or deficiency of the measure of focused correlation. Rather, it may be considered as illuminating a shortcoming of incremental confirmation measures since they fail to take into account the additional information provided by the relevance of each individual bit of evidence. As we have seen, this information is important in flagging a situation in which a large boost in confirmation caused by adding a piece of evidence has nothing 388 MAXIMILIAN SCHLOSSHAUER AND GREGORY WHEELER or only little to do with the coincidence of this evidence with the existing body of evidence. One merit of focused correlation is to detect such cases. In one class of cases that has received a lot of attention, namely, the single-factor common-cause model mentioned in the introduction, any measure linking positive association to positive confirmation breaks down, and comparisons (assuming A2) are never false because they cannot be different. But these facts take nothing away from focused correlation, for two reasons. First, because this class of models is determined nonpara- metrically, we can learn whether one hypothesis is a common cause (Silva et al. 2006) and then attack the problem by other methods—namely, by looking at the relative strengths of the evidence. We do not need the sign of focused correlation in single-factor common-cause models to indicate the sign of confirmation if we understand clearly when the two do not align. In the case of single-factor common-cause models, the correlation of the evidence is completely determined by the relevant strengths of the evidence (Danks and Glymour 2001). Faulting focused correlation for failing to indicate confirmation in this class of models is akin to faulting screwdrivers for failing to drive nails: it is a true claim that is entirely beside the point. Second, it should be clear that the failure of focused correlation to track differences in confirmation of evidence in common- cause models in proposition 1 is driven by A2 rather than a deficiency in focused correlation. But, again, although our results here allow for some tracking in common-cause models, at bottom a difference in confirmation in this class of models boils down to a difference in the individual strengths, .P(HFE )i Focused correlation is defined generally for n evidence variables, but we only discuss evidence sets of size two, and difficulties loom for attempts to make comparisons of larger evidence sets (Bovens and Hartmann 2006). Even though focused correlation is defined for arbitrary-sized information sets, the relationships of confirmation, covariance, and correlation are binary—or three-place in conditional form. Thus, the expansion of evi- dence sets beyond size two will require a partition of the evidence set since there are many incremental confirmation questions that are com- patible with one focused correlation problem involving an evidence set of a size greater than two. To expect otherwise is a category mistake, and negative results should be no surprise. That said, if there is not an interest in linking focused correlation for large evidence sets—which, technically, is not a measure of correlation but is instead a distance-from-independence measure—to confirmation, then one may explore extending the results presented here to facilitate direct comparisons of large evidence sets. Finally, two commonly discussed confirmation measures have been omitted from discussion, Carnap’s (1962) relevance measure t and the old CORRELATION, CONFIRMATION, AND VARIABLE EVIDENCE 389 evidence measure oe (Christensen 1999; Joyce 1999), which are defined as t(H, E )P(H " E ) ! P(H )P(E ) and oe(H, E )P(HFE ) ! P(HF¬E ). Neither measure is tracked by focused correlation. The reason is that both measures are less constrained than incremental confirmation measures. In the case of Carnap’s measure of relevance, observe that is thet(H, E ) covariance of the binary variables H and E: Cov (H, E ) p P(H " E ) ! P(H )P(E ) p P(H )(P(EFH ) ! P(E )). Although positive covariance for H and a single evidence variable E is ensured by the positive relevance condition A1, conditions A1 and A2 and equation (9a) do not constrain the sign of for evidence setst(H, E) of size two. In the case of oe, there is some recognition already of theE difference between incremental confirmation measures and measures for novelty of evidence like oe (Joyce 1999). The failure to compare evidence sets by their degree of surprise, particularly under the restrictive conditions of positivity, A1, A2, and equation (9a), is an additional reason to sharply distinguish between these two types of measures. Countermodels for oe are discussed in the appendix. Appendix The following proofs of propositions 2–4 are similar in spirit and rely heavily on lemma 1, which follows from the definition of inFor (E , E )H 1 2 equation (1). Lemma 1. If with , thenFor (E , E ) p lFor (E , E ) l 1 0H 1 2 H 1 3 P(HFE " E ) P(HFE )1 2 2p l . (8) P(HFE " E ) P(HFE )1 3 3 Proof of Lemma 1. P(HFE " E )P(H ) P(HFE " E )P(H )1 2 1 3p l P(HFE )P(HFE ) P(HFE )P(HFE )1 2 1 3 P(HFE " E ) P(H ) P(HFE " E ) P(H )1 2 1 3# p l # P(HFE ) P(HFE ) P(HFE ) P(HFE )2 1 3 1 P(HFE " E ) P(HFE )1 2 2p l . P(HFE " E ) P(HFE )1 3 3 QED 390 MAXIMILIAN SCHLOSSHAUER AND GREGORY WHEELER Proof of Proposition 2. We begin with two observations, assuming A1 and A2: P(HFE " E ) 1 P(HFE " E ) # c(H, {E , E }) 1 c(H, {E , E }), (9a)1 2 1 3 1 2 1 3 P(HFE " E ) p P(HFE " E ) # c(H, {E , E }) p c(H, {E , E }). (9b)1 2 1 3 1 2 1 3 Turning to , by A2, . So, by lemma 1, theFor (E) P(HFE )/P(HFE ) p 1H 2 3 left-hand side of equations (9a) and (9b) may be expressed by equation (10) when and , respectively.l 1 1 l p 1 P(HFE " E )1 2 p l. (10) P(HFE " E )1 3 Without A2, observe that P(HFE " E ) P(HFE )1 2 2p l , P(HFE " E ) P(HFE )1 3 3 P(HFE " E ) P(HFE )1 2 3# p l, P(HFE " E ) P(HFE )1 3 2 P(HFE " E )P(H ) P(HFE )P(HFE )1 2 1 3# p l, (11) P(HFE )P(HFE ) P(HFE " E )P(H )1 2 1 3 For (E , E )H 1 2 p l. For (E , E )H 1 3 So , given (11), and , by hypothesis.l 1 1 For (E , E ) 1 For (E , E )H 1 2 H 1 3 Therefore, , and thusP(HFE " E ) 1 P(HFE " E ) c(H, {E , E }) 11 2 1 3 1 2 iffc(H, {E , E })1 3 P(HFE ) 1 For (E , E )2 H 1 3 1 p . (12) P(HFE ) l For (E , E )3 H 1 2 QED Proof of Proposition 3. By assumption, , andc(H, {E , E }) 1 c(H, {E , E })1 2 1 3 thus from equations (9a) and (9b). Then byP(HFE " E ) 1 P(HFE " E )1 2 1 3 lemma 1 P(HFE " E ) P(HFE )1 2 2 1 , P(HFE " E ) P(HFE )1 3 3 P(HFE " E ) P(HFE " E )1 2 1 3 1 . P(HFE ) P(HFE )2 3 We can tighten the bound if we choose a particular confirmation measure. For example, CORRELATION, CONFIRMATION, AND VARIABLE EVIDENCE 391 inc (H, E , E ) p g inc (H, E , E )k 1 2 k 1 3 # P(HFE " E ) p gP(HFE " E ), k p 1, 2,1 2 1 3 and thus the bound is P(HFE ) inc (H, E , E )2 k 1 2 ! g p , (13) P(HFE ) inc (H, E , E )3 k 1 3 which proves equation (4). QED Proof of Proposition 4. If , then definition (1)For (E , E ) p For (E , E )H 1 2 H 1 3 implies that iff . Con-P(HFE " E ) p P(HFE " E ) P(HFE ) p P(HFE )1 2 1 3 2 3 versely, if , then equation (9b) implies thatc(H, {E , E }) p c(H, {E , E })1 2 1 3 , and definition (1) then shows thatP(HFE " E ) p P(HFE " E )1 2 1 3 iff . QEDFor (E , E ) p For (E , E ) P(HFE ) p P(HFE )H 1 2 H 1 3 2 3 Countermodels for Measure Tracking oe. To show that P(HFE " E ) 11 2 does not entail under pos-P(HFE " E ) oe(H, {E , E }) 1 oe(H, {E , E })1 3 1 2 1 3 itivity, A1, and A2, it suffices to provide a model in which P(HFE " E ) 1 P(HFE " E ) (14)1 2 1 3 but P(HFE " E ) ! P(HF¬E # ¬E )1 2 1 2 (15)- P(HFE " E ) ! P(HF¬E # ¬E ).1 3 1 3 The constraints A1 and A2 are strong by design, and we can simplify equation (15) by observing the following lemma. Lemma 2. If is a positive probability distribution overP(D) AH, ES satisfying A1 and A2, then i) , andP(E ) p P(E ) p P(E )1 2 3 ii) .P(HF¬E ) p P(HF¬E ) p P(HF¬E ) p a1 2 3 Proof of Lemma 2. From A2, it follows that the covariance of each piece of evidence with the hypothesis is identical, for ,P(E FH )/P(E ) i p 1, 2, 3i i which can be rewritten as P(E )[P(HFE ) ! P(H )].i i Assumption A2 guarantees lemma 2i, and A1 ensures that the covariance is positive. From total probability, A2, and lemma 2i, P(H ) p P(HFE )P(E ) # P(HF¬E )1 ! P(E ).i i i i QED Lemma 2 allows for the reduction of (15) to 392 MAXIMILIAN SCHLOSSHAUER AND GREGORY WHEELER P(HFE " E ) ! [2a ! P(HF¬E " ¬E )]1 2 1 2 - P(HFE " E ) ! [2a ! P(HF¬E " ¬E )]. (16)1 3 1 3 But while by hypothesis, the remaining termsP(HFE " E ) 1 P(HFE " E )1 2 1 3 are unconstrained. Countermodels exist that violate the inequality. Anal- ogously, there are countermodels in which P(HFE " E ) p P(HFE " E )1 2 1 3 but . In sum, more incremental confirmationoe(H, E " E ) ( oe(E " E )1 2 1 3 does not ensure more evidential surprise, and it should be clear from the discussion that the converse does not hold either. REFERENCES Bovens, L. and S. Hartmann. 2003. Bayesian Epistemology. Oxford: Oxford University Press. ———. 2006. “An Impossibility Result for Coherence Rankings.” Philosophical Studies 128: 77–91. Carnap, R. 1962. The Logical Foundations of Probability. 2nd ed. Chicago: University of Chicago Press. Christensen, D. 1999. “Measuring Confirmation.” Journal of Philosophy 96:437–61. Cohen, L. J. 1977. The Probable and the Provable. Oxford: Clarendon. Crupi, V., K. Tentori, and M. Gonzalez. 2007. “On Bayesian Measures of Evidential Support: Theoretical and Empirical Issues.” Philosophy of Science 74 (2): 229–52. Danks, D., and C. Glymour. 2001. “Linearity Properties of Bayes Nets with Binary Vari- ables.” In Uncertainty in Artificial Intelligence: Proceedings of the 17th Conference, ed. J. Breese and D. Koller, 88–104. San Francisco: Kaufmann. Eells, E., and B. Fitelson. 2002. “Symmetries and Asymmetries in Evidential Support.” Philosophical Studies 107 (2): 129–42. Joyce, J. 1999. The Foundations of Causal Decision Theory. Cambridge: Cambridge Uni- versity Press. Kemeny, J., and P. Oppenheim. 1952. “Degrees of Factual Support.” Philosophy of Science 19:307–24. Keynes, J. M. 1921. A Treatise on Probability. London: Macmillan. Kyburg, H. E., Jr. 1983. “Recent Work in Inductive Logic.” In Recent Work in Philosophy, ed. T. Machan and K. Lucey, 87–150. Totowa, NJ: Rowman & Allanheld. Milne, P. 1997. “ Is the One True Measure of Confirmation.” Philosophylog [P(h/eb)/P(h/b)] of Science 63:21–26. Myrvold, W. 1996. “Bayesianism and Diverse Evidence: A Reply to Andrew Wayne.” Phi- losophy of Science 63:661–65. ———. 2003. “A Bayesian Account of the Virtue of Unification.” Philosophy of Science 70:399–423. Olsson, E. 2005. Against Coherence: Truth, Probability and Justification. Oxford: Oxford University Press. Silva, R., C. Glymour, R. Scheines, and P. Spirtes. 2006. “Learning the Structure of Latent Linear Structure Models.” Journal of Machine Learning Research 7:191–246. Wheeler, G. 2009. “Focused Correlation and Confirmation.” British Journal for the Philos- ophy of Science 60 (1): 79–100. Wheeler, G., and R. Scheines. 2011. “Causation, Association, and Confirmation.” In Ex- planation, Prediction, and Confirmation: New Trends and Old Ones Reconsidered. The Philosophy of Science in a European Perspective. Dordrecht: Springer. Cit p_2:1: Cit p_4:1: Cit p_6:1: Cit p_8:1: Cit p_10:1: Cit p_14:1: Cit p_15:1: Cit p_18:1: