key: cord-0862990-9bx0sedy authors: Maru, Yiheyis Taddele; Sparrow, Ashley; Butler, James R.A.; Banerjee, Onil; Ison, Ray; Hall, Andy; Carberry, Peter title: Towards appropriate mainstreaming of “Theory of Change” approaches into agricultural research for development: Challenges and opportunities date: 2018-05-16 journal: Agric Syst DOI: 10.1016/j.agsy.2018.04.010 sha: 37690abc411192607d8e7a2ee414ec16c4002c9b doc_id: 862990 cord_uid: 9bx0sedy Food insecurity persists in many parts of Africa and Asia, despite ongoing agricultural research for development (AR4D) interventions. This is resulting in a growing demand for alternative approaches to designing and evaluating interventions in complex systems. Theory of Change (ToC) is an approach which may be useful because it enables stakeholders to present and test their theories and assumptions about why and how impact may occur, ideally within an environment conducive to iterative reflection and learning. However, ToC is yet to be appropriately mainstreamed into development by donors, researchers and practitioners. We carried out a literature review, triangulated by interviews with 26 experts in African and Asian food security, consisting of researchers, advisors to programs, and donors. Although 17 (65%) of the experts had adopted ToC, their responses and the literature revealed four challenges to mainstreaming: (i) different interpretations of ToC; (ii) incoherence in relationships among the constituent concepts of ToC; (iii) confused relationships between ToC and project “logframes”; and (iv) limitations in necessary skills and commitment for enacting ToC. A case study of the evolution of a ToC in a West African AR4D project over 4 years which exemplified these challenges is presented. Five recommendations arise to assist the mainstreaming of ToC: (i) select a type of ToC suited to the relative complexity of the problem and focal system of interest; (ii) state a theory or hypotheses to be tested as the intervention progresses; (iii) articulate the relationship between the ToC and parallel approaches (e.g. logframe); (iv) accept that a ToC is a process, and (v) allow time and resources for implementers and researchers to develop ToC thinking within projects. Finally, we suggest that communities of practice should be established among AR4D and donor organisations to test, evaluate and improve the contribution that ToCs can make to sustainable food security and agricultural development. Finding ways to improve the effectiveness and impact of food security interventions is one of the key challenges facing the development assistance community (Foran et al., 2014; Ozor et al., 2013) . Interventions have an uneven record of success and worryingly high rates of food insecurity remain in many parts of Africa and Asia (E.g. Banerjee et al., 2014; Deaton and Lipka, 2015) . One of the major responses to limited success has been an increasing demand for demonstrating achievement of results and value for money from food security interventions (Buntaine et al., 2013) . Under this growing results-orientated culture there has been more reflection on the conceptual and theoretical foundations of project design, and how and why success or failure occurs. From this reflection, a number of concepts and approaches have gained prominence, including developing a Theory of Change (ToC) to underpin intervention design (Davies, 2004; Vogel, 2012) . ToC refers to a process where stakeholders develop, monitor and evaluate theories that underpin the design of an intervention and explain how and why impact will be achieved through the implementation of the intervention (Blamey and Mackenzie, 2007; James, 2011) . The literature traces the dominant ToC lineage to the field of theorybased evaluation approaches (Vogel, 2012) . These evaluation approaches were introduced over four decades ago to explain how and why an intervention achieved or contributed to impact (Weiss, 1972) , rather than focusing only on measuring whether or not an intervention had achieved stated outputs and outcomes (Chen and Rossi, 1983; Connell and Kubisch, 1998; Pawson and Tilley, 1997; Rogers et al., 2000) . The call for ToC-informed design of interventions was thus triggered by the needs of evaluation practitioners. James (2011) identifies a second contribution to the evolution of ToCthe community development domain's work on participatory approaches (such as Participatory Action Research, action learning and empowerment) that have long advocated for conscious and continuous joint reflection as a catalyst for learning and informed action to bring about positive changes. This strand is also important because it connects ToC to proactive change through single-, double-and triple-loop learning. Single-loop learning refers to modification or incremental improvement of action strategies without questioning the underlying assumptions and goals. Double-loop learning is the revisiting and reframing of assumptions and goals (Argyris and Schön, 1999) . In tripleloop learning, one starts to reconsider underlying values, beliefs and paradigms, because the initial world-view no longer seems to hold (Flood and Romm, 1996; Pahl-Wostl, 2009) . Proponents argue that theory-based design and evaluation enhances learning from programs (Funnell and Rogers, 2011; Vogel, 2012) through its explanation of mechanisms of how, why and in what context an intervention achieves or contributes to impact (Mayne, 2012) . In other words, it provides information beyond answering whether or not the intervention simply achieved or contributed to impact (Shaffer, 2013) , particularly in relation to complicated, dynamic and complex issues (Funnell and Rogers, 2011; Rogers, 2008) . At the turn of the 21st century, ToC and impact pathways thinking were introduced to the agricultural research for development (AR4D) sector. Thornton et al. (2017) define AR4D as a set of applied research approaches that aim to contribute directly to the achievement of international development targets, usually involving demand-led prioritization of research, participatory and action research, and stakeholder involvement and capacity development. Most AR4D interventions have lofty food security and/or agricultural development goals, but often the theories and pathways for how and why the particular intervention would contribute to or achieve impact were not well articulated, encapsulated in design, or tested (Douthwaite et al., 2003) . Kuby (1999) refers to this as the "missing middle" or "output-impact gap." Douthwaite et al. (2003) developed Impact Pathways Analysis (IPA) as a version of program theory or ToC (Rogers et al., 2000) that incorporated recent conceptual advances and articulations of the "missing middle" and "attribution gap" in AR4D. They used the terms "ToC" and "IPA" interchangeably, but preferred the latter because of the familiarity and pragmatic nature of the term to practitioners working in agricultural research and development interventions (Douthwaite et al., 2003; Douthwaite et al., 2007; Kuby, 1999; Mackay and Horton, 2003; Secretariat, 2000; Springer-Heinze et al., 2003) . More recently, key developers of IPA have made distinctions between IPA and ToC, where the former "maps out causalitynormally using boxes and arrows", and the latter "explains the assumptions behind the arrows" (Douthwaite et al., 2013) . These differences between ToC and IPA echo Weiss' (1997) distinction between "implementation theory" and "program theory", which she noted are often confused or lumped together. Implementation theory focuses on the necessary steps through which an intervention will be carried out, thereby mirroring IPA. In contrast, program theory focuses on the responses an intervention generates, or the mechanisms of change triggered by the intervention (Pawson and Tilley, 1997; Blamey and Mackenzie, 2007) . These distinctions are important, since most current work on Impact Pathways, and indeed the application of ToC in AR4D, is largely about implementation logic rather than deep reflection on underlying worldviews, assumptions and theories that explain the mechanics that generate the desired changein the manner of triple-loop learning. More than a decade on, ToC is becoming a more common requirement in the design and funding of AR4D interventions (Thornton et al., 2017; Vogel, 2012) . This evolution of development thinking is important and likely to continue, but there are concerns that ToC could simply become another burdensome administrative requirement that brings no substantive change beyond simplistic compliance or "boxticking" (e.g. Green, 2012; Valters, 2014) . This paper assesses the challenges and potential solutions to appropriately mainstreaming ToC into the design and evaluation of AR4D interventions. By "mainstreaming" we refer to the process of embedding a new concept, principles or an approach into a routine practice of individuals and organisations of relevant domains (McCarthy, 2010) , while recognising that there is no guarantee that the new approach will be institutionalised as originally intended (Squires, 2005) . First, we conducted a literature review, triangulated via interviews with experienced practitioners and donors in the agricultural and development field to ascertain the current understanding and application of ToC (Section 2). Four major challenges to mainstreaming emerging as themes from the analysis of literature and interviews are described in Section 3. We then present a case study of the evolution of ToC practice in an AR4D project in West Africa, which exemplifies several of these challenges (Section 4). We conclude with some recommendations about how ToC can be mainstreamed into AR4D, and its practice refined and improved through ongoing testing, reflection and learning. The literature review included recent books, journal publications and grey literature about ToC practice generally and also within the A4RD and food security domain. Based on their networks and knowledge, the authors developed an initial list of 70 potential interviewees considered to be at the forefront of the AR4D domain and who were focused on Africa and Asia, the global hotspots of chronic food insecurity and poverty. The potential interviewees worked in different national, regional and international research, academic, non-government, donor, private, and public organisations, and included equal representation of women and men. Of the 70 in the original list, 44 individuals were prioritized and invited to an interview; 28 individuals accepted the invitation and ultimately 26 (8 women and 18 men) made themselves available for interview. Twelve interviewees were researchers, nine were managers or advisors in development programs, and five were from governmental or philanthropic donor organisations. The interviews involved a set of semi-structured questions about the expert's understanding of ToC and impact pathways, and their experience of applying ToC in intervention design, implementation and impact assessment. Interviews were transcribed and analysis of transcripts was assisted by use of NVivo qualitative analytical software (QSR International Pty Ltd, 2012) . Analysis of both the literature and interview transcripts employed a constant comparative technique from a grounded theory approach (Glaser, 2017; Strauss and Corbin, 1997) in order to develop an understanding of the state of, and constraints to mainstreaming ToC in AR4D. Four thematic challenges emerged from the research: (i) different interpretations of ToC; (ii) incoherence in relationships among the constituent concepts of ToC; (iii) confused relationships between ToC and the "logframe" which is still a dominant design tool in AR4D interventions (Prinsen and Nijhof, 2015) ; and (iv) necessary skills and commitment for enacting ToC. Over the years, there has been a growing diversity of ToC interpretations, and a broadening of its domains of application (James, 2011) . In its application, ToC's original and predominant use has been as a guide for theory-based evaluations. Recently ToC has been increasingly used as an essential part of the process of designing interventions that guides implementation and also informs ex-post impact analysis (Douthwaite et al., 2013) . Stein and Valters (2012) identified a continuum of interpretations of ToC and we found that usage in the literature can be grouped into three broad categories along this continuumdescriptive, explanatory, and reflectiveeach with different implications for ToC practice (Fig. 1) . At the descriptive end of the continuum, ToC is understood as a precise planning tool that extends the "assumptions" box in a logframe (Green, 2012) . In the centre of the continuum the explanatory category provides a response to the causal questions of how and why an intervention works, although such explanations often tend to remain technical. The reflective end of the continuum goes deeper and considers that in addition to having technical dimensions, the enabling environment is also critical as development problems have social, institutional and political dimensions which make problems complex. This ToC interpretation promotes complexity-aware and deliberate reflection to identify and challenge paradigms, theories and assumptions on how change will happen through the interplay between an intervention's design and its context. Of the interviewees, 17 (65%) stated that they or their organisations had adopted ToC and were encouraging or requiring programs and projects to develop ToC as part of their design. We also found a similar diversity of interpretations of ToC among them, which could be mapped into the three categories along the continuum (Table 1) . Only four (24%) of the 17 interviewees defined ToC according to the reflective category. Three were researchers and one a donor; none was a manager or practitioner. The four noted the growing importance of a reflective process that provides opportunity to explicitly state assumptions and discuss complex causal relationships. Two of them (both female) also noted that a ToC approach should be participatory, gendersensitive and address issues of power relations among stakeholders in development interventions, as well as identifying key actors and partnerships that may be required to achieve impact. This parallels a view emerging in the literature that sees ToC as a deep reflective process which makes explicit and challenges worldviews, theories, assumptions and power relations, and to develop a shared vision on how and why an intervention could lead to impact (e.g. James, 2011) . Diversity in interpretation of an evolving ToC approach is expected. However, lack of clarity in how this range of interpretations, which includes rehashing of conventional approaches, can significantly diminish ToC's potential to assist the design, implementation and evaluation of AR4D interventions (Green, 2012; Vogel, 2012) . Given the complexity of food security and agricultural development challenges (Foran et al., 2014) , in most instances descriptive or explanatory interpretations of ToC may not be adequate to address established approaches, assumptions and paradigms that often create barriers for innovation and change (Klerkx et al., 2012) . Instead, reflective ToC that also facilitates institutional and social changes through explicit doubleand triple-loop learning may be required to achieve impact. Theories, assumptions and evidence are three key components in developing any ToC. However, we found ambiguities and incoherence in the literature and among interviewees about the relationships between these concepts and their role in ToC development. While seminal work on ToC emphasises the need to consider theories of social change (e.g. Chen and Rossi, 1989; Weiss, 1995; Weiss, 1997) , many ToC practice guides and reports of applications of ToC do not articulate the need for a foundation in theories that inform change processes (e.g. Annie, 2004; Jost et al., 2014; Mackinnon et al., 2006) . Chen (2006) notes that any ToC is underpinned explicitly or implicitly by theory with normative and causative components. The normative component provides guidance on what goals or outcomes should be pursued or examined and how the intervention should be designed and implemented. The causative component specifies how the program works by identifying the conditions under which certain processes will arise and what their likely consequences will be. Most current ToCs underpinning AR4D and other development interventions are a-theoretical in that they don't engage with social, behavioural, economic, institutional and/or biophysical theories that explain the mechanisms by which an intervention may bring or contribute to impact. Most ToCs are simply developed around empirical observation or assumptions based on experience. A recent evaluation of a CGIAR research program found the ToC to be inadequately theorised: There are numerous theories that address technology adoption (e.g. sociological theories of diffusion of innovations, the economic theory of induced innovation, the systems of innovation theories, etc.). Compared with these, the theory of change from the Bangladesh hub is very simplistic. The evaluation team … suggests that theories of change that are grounded in the relevant bodies of theory would be both more compelling and more effective in facilitating Participatory Action Research and contributing to global knowledge (Birner et al., 2015, p. 54). Most interviewees did not mention the role of theory in ToC. Only two of the 17 (12%), both manager-practitioners, expressed the limited role of theory in their practice: We don't have enough time to dwell so much on the theoretical development … we are more concerned about getting people out of poverty (R# 1). So I think that cumulative experience of trial and error, and lots of failings and knowing what works and what doesn't, so I think I do it automatically but I don't have a flow chart on what you would call a theory (R# 9). Only one interviewee, a researcher, entertained the importance of theoretical foundations for a theory of change: We built an elaborate theory why an uninsured risk leads to poverty traps; low level of living standard, where they are unable to maintain and build herd, leads to land degradation, localized crime and a host of other problems that are mutually reinforcing (R# 26). There is a no explicit linkage of most ToC exercises in the AR4D domain to even current popular theoriesincluding "innovation diffusion" (Rogers, 2010) and "innovation systems" (Hall, 2007) that inform widely-used approaches such as Transfer of Technology and Agricultural Innovation Systems, respectively. It seems that theories such as innovation diffusion or innovation systems are often selected as a matter of organizational and/or individual preference, rather than the result of deliberation of their ability to address problems within their contexts. ToC exercises ought to bring these theories under critical scrutiny in which they are assessed for 'fitness of purpose' for an intervention. Maru et al. (2016, this special issue) identify four classes of theories implicit in ToCs across AR4D interventions: institutional change, market linkage, building social capital, and innovation capacity. They suggest that making these theories explicit enables testing their relevance, and the consideration of alternative and complementary theories. Recently, Mayne (Mayne, 2015 (Mayne, , 2016 suggested a generic ToC informed by a behavioural change theory, which proposes that change occurs as a result of interaction between three necessary conditions: capabilities, opportunities and motivation. While this is an important development in theory-informed ToC, a generic ToC underpinned by behavioural change theory is unlikely to be appropriate if the primary mechanism for impact is structural change (e.g. institutions, networks and/or infrastructure) rather than individuals' behaviours and choices. There are many cases where structural change has to precede, or at least must be implemented in parallel with, behavioural change among beneficiaries to achieve desired sustainable development impacts (e.g. Butler et al., 2016; Hounkonnou et al., 2012) . In West Africa, Hounkonnou et al. (2016, this special issue) clearly demonstrate that institutional changes at levels higher than the farm were a necessary precondition for smallholder agricultural development. Furthermore, more than one theory might be entertained (Funnell and Rogers, 2011) . Stakeholders may have alternative or rival theories on how the intervention would trigger behavioural and/or structural change to achieve impact (Rogers and UNICEF, 2014) , and there may also be theories that address different aspects of a single intervention. In the absence of suitable established theories informing ToC, evidence can be collected to develop an integrated set of hypotheses or a grounded theory to underpin a ToC (Glaser and Strauss, 2009 ). Such evidence can be collected through a participatory scoping inquiry, and from work done on similar problems and contexts elsewhere, or transdisciplinary workshops. As one researcher interviewee noted, such evidence can be used to generate theories inductively: Conduct empirical study to understand context and relationships and core variables of a system, then inductively move to theory, deductively generate testable hypotheses, and design products or processesimplement, monitor and evaluate (R# 26). In contrast with the role of theory in ToC, identifying and articulating assumptions were widely considered as central to a ToC exercise, both in the literature and among interviewees. This centrality seems to have added to the blurring of meaning between theory and assumptions (Stein and Valters, 2012) . ToCs are often considered synonymous with beliefs or assumptions that stakeholders hold about how change occurs in an intervention (Vogel, 2012) . Nkwake (2013) notes that while there is an emphasis on examining underlying assumptions, there is little in the ToC literature about examining the nature of assumptions, methods of explicating them, and how they relate to theory. An assumption is any statement about something that is taken for granted or believed to be true (Argyris, 1976) in the context of a defined situation. Assumptions are also presuppositions or approximations we make about something when we know little about it (Kriström, 1990) . In contrast, theories are systematic explanations of observations Table 1 Examples of interviewees' interpretations of ToC or impact pathways, categorised according to the continuum in Fig. 1 , and numbers of the 17 interviewees who stated that they or their organisations had adopted ToC. A general ToC is to understand the problem you are trying to solve, come forward with a solution through research, workout on how to deliver the solution in an affordable price to the people who can benefit from it (R# 2). Descriptive 8 6 2 0 Impact pathway -start from what is the endpoint we want to get to, study where are we now, and what are the key things that will need to happen to get us through that change process, then the specific design of the research is derived from that understanding (R# 19). ToC involve quantifiable outputs, outcomes and impacts (R# 25). A causal model is one form of ToC (R# 23). Explanatory 5 1 1 3 ToC should be rational, sensible and able to explain what a program does and tries to achieve (R# 10). ToC helps to clarify and test the logic, assumptions and risk and to identify the multiple steps and players along the way to impact (R# 10). ToC, though challenging, needs continual reflection as different people come with different terminology, understanding and perspectives that need to come together to form a coherent pathway to impact (R# 11). Reflective 4 0 1 3 Respondent interview codes are shown by R#. Y.T. Maru et al. Agricultural Systems 165 (2018) 344-353 that relate to a particular aspect of nature (Babbie, 2015) . Theories possess explanatory and predictive power and use few assumptions as premises or preconditions to generate an integrated set of testable hypotheses. Theories do not generally make assumptions in their conventional meaning (i.e. as statements accepted as true); instead, key assumptions are either supported by evidence (such as from testing of previously existing theories in certain contexts), or the evidence will be produced in the course of testing the theory in other contexts (Nkwake, 2013) . The set of assumptions in a theory must have internal logical consistency, and to be consistent they must be explicit. The process of identifying and questioning key assumptions is crucial for achieving common understanding among stakeholders about how an intervention will generate impact. If conflicting assumptions are not discussed upfront and resolved, they can lead to conflicting expectations from an intervention (Weiss, 2000) . There is considerable subjectivity and a degree of arbitrariness in the identification and selection of assumptions underpinning a ToC. Where assumptions have been stated, they have almost always been external and of an extreme nature (e.g. a civil war does not recur). However, there is another type, known as causal assumptions (Nkwake, 2013) , that are internal and integral to cause and effect relationships involved in bringing impact. For example, in an intervention aimed at improving the nutrition of children, Mayne (2015) identified an assumption that mothers make dietary decisions for their children, and husbands and mothers-in-law are supportive of those decisions. Such a causal assumption is deeply cultural and underlines the potential social complexity of that project's context. The role of evidence is not clear in the ToC literature (Stein and Valters, 2012) . However, two interviewees, both researchers, spoke of the role of evidence in developing and adapting a robust ToC: Experience and research-based evidence can play a role in clarifying the nature of a problem and opportunity, and the context for the intervention providing empirical basis for a sound ToC (R# 26). A theory is only as good as the evidence used to test it. We have evidence that we haven't really tested the theories of change we operate by (R# 16). Evidence also has an ongoing role during implementation and evaluation of the intervention. It is collected to test the intervention's theory, validate, refine and change assumptions, adapt the intervention accordingly, as well as finally evaluate whether intervention has achieved its stated impact (Patton, 2011) . The process of developing a ToC provides opportunities to open up space and create appropriate incentives for questioning dominant paradigms, theories, assumptions and approaches. This questioning can draw on empirical and experiential evidence or new theories, with collective reflection and dialogue playing important roles in resolving differences. Again, this speaks to the importance of reflective learning when applying ToC in any intervention. However, double-and tripleloop learning also requires explicit stakeholder willingness and adaptability, as this may demand personal and organizational change in the routine practices, underlying institutions and paradigms. Furthermore, in contrast to usual practice, a ToC should not be a one-off activity in the design phase of an intervention. Rather, it is a commitment to an iterative process of review and revision as new understanding and opportunities emerge. As one donor interviewee noted: I think the challenge will be the continual reflection on the ToC. So we've had a number of workshops and it was a bit painful to be quite honest, and different people had different levels of buy-in and terminologies and people coming from different perspectives … from a policy perspective or from a science perspective, an in-the-fields perspective or a bureaucratic perspective… which is beneficial in a way because you bring different perspectives into the mix … So it's not just a design team or implementation team exercise. It's potentially an innovation platform that you're working with (R# 11). This requires attention to the sort of incentives that project staff and stakeholders have to regularly collect evidence and reflect as both expected and unexpected changes unfold (Barnett and Gregorowski, 2013) . An important question about ToC is its relationship with other established project planning approaches in the AR4D domain. The most widespread results-based management approach is the Logical Framework Approach, or "logframe" (Bakewell and Garbutt, 2005) . Many interviewees raised the comparison; some considered the logframe to be interchangeable with ToC, and some said that both logframes and ToCs are required by donors. The logframe, as currently widely applied, has the causal chain standardized as a pipeline model with four components: impact (a higher-level goal to which an intervention and others contributed), inputs (activities), outputs, and purpose (which includes the rationale for producing the outputs). These are subtended by a matrix with an articulated narrative description of objectives, objectively verifiable quantitative and qualitative indicators, means of verification, and assumptions and factors outside the control of the intervention on which the success of that component depends (Gasper, 1997) . Several reviews of the logframe have shown its strengths and weaknesses. Strengths include that it a forces project stakeholders to think carefully and systematically through what they are planning to do (Cracknell, 1989) ; it provides a hierarchical relationship between essential elements of a project, as well as a framework for monitoring and evaluation which can compare planned with actual results; and it provides a simple summary of the key elements of a project proposal in a consistent and coherent way (Bakewell and Garbutt, 2005) . The primary weakness of the logframe is that it oversimplifies the reality of intervention-outcome/impact relationships, particularly when dealing with complex development issues in which interacting feedback loops may generate unintended effects (Dale, 2003; Hummelbrunner, 2010) . Thus logframes can instil a mistaken belief in predictability and control over how events will unfold during a project (Reeler, 2007) , and fail to deal well with the slow or negative progress typical of the early stages of many types of projects (Bakewell and Garbutt, 2005; Hummelbrunner, 2010; Woolcock, 2009) . Completing a logframe matrix has now become a widespread mandatory funding requirement, with standardized templates that allow little flexibility to understand and deal with complicated and complex problems (Barnett and Gregorowski, 2013; Vogel, 2012) . Early in the development of the logframe, Cracknell (1989, p. 167) warned: "like every such 'formalised' system, it [the logframe] could only too easily degenerate into another piece of bureaucracy if not applied imaginatively and intelligently". Bakewell and Garbutt (2005) also noted that the problem with the approach is not with the framework itself, but the way that it is has been standardized and used. Vogel (2012) noted that it is still difficult for many to make a distinction between the recently introduced ToC approach and the more familiar and widely-applied logframe. Likewise, half of our interviewees, especially managers and practitioners whose interpretation of ToC was mainly descriptive, treated ToC as interchangeable with the logframe, and observed that both are often required for accountability: "I don't see any difference between the logframe and ToC or result frames. All are requirements from donors to articulate how we achieve outcomes and impacts." (R# 2). "It [ToC] and logframes simply tell us this is where I want to go, how I Y.T. Maru et al. Agricultural Systems 165 (2018) 344-353 will do it. There are possibilities for change and I am receptive of these changes that take me where I want to go (R# 1). "The key for ToC or logframing or result-framing is to understand the key concepts and applying them flexibly" (R# 3). Conflation of logframes and ToC could risk having a negative influence on the mainstreaming of ToC, because of perceived inheritance of the flaws of logframes as practised, and a lack of emphasis on reflective integration with project implementation. The treatment of both logframes and ToC as only donor accountability tools may stifle the potential of the ToC to overcome the poor usage of logframes, and thus to help close the "output-impact gap" (Vogel, 2012) . Furthermore, a ToC for a complex development problem and context may not need a logframe or results chain, because having such a blueprint means that the problem is treated as simple, and solutions as projectable (Barnett and Gregorowski, 2013; Reeler, 2007) . Rather, what is needed in dealing with complex problems is an adaptive guide. For example, a theory of complex adaptive systems can lead probing experimentation and principles of adaptive governance to guide intervention (Ison et al., 2014) , linking to double-and triple-loop learning and adaptation (Patton, 2011; Snowden, 2000) . Mainstreaming ToC appropriately requires new sets of skills to facilitate building a ToC appropriate to a problem and its context, by ensuring effective communication, participation and partnership among stakeholders. Proper engagement of all stakeholders with the different aspects of ToC requires very significant, but often underappreciated and underestimated commitment in terms of attention, effort and time (Jost et al., 2014a) . Green (2012) notes that uses of ToC are currently very top-down, usually drawn up by "experts" in the country office, rather than through a participatory, reflective processpresumably to maintain investments of time and other resources within budget. For example, a review by Barrett et al. (2009) noted that many donors expect the CGIAR to perform an impossible task, generating research that delivers large-scale, sustained poverty reduction in a very short time. However, creating these top-down accounts of prospective change that speak more to donor interests than to on-ground realities of the people affected by these interventions is problematic (Douthwaite et al., 2009; Valters, 2014; Thornton et al., 2017) . The perverse incentive to seek to "dumb down" and simplify what are often irreducibly complex problems is pervasive. Venturing out to employ reflective and complexity-aware ToCs with flexibility to learn and adapt interventions still meet resistance because they take time, and do not provide the mistakenly-perceived certainty provided by simplified results-based planning approaches (Douthwaite and Hoffecker, 2017) . Incentive and reward structures based on visible and immediate outputs and outcomes will discourage, even disadvantage, those employing complexity-aware ways of designing and implementing ToC for complex problems. There is a critical need for donors and agencies to provide incentives for the development of the new skill sets in ToC fit for specific combinations of problems, opportunities and contexts among staff who facilitate the design, implementation and on-going evaluation of interventions. Such skills are likely to include inter-or trans-disciplinary approaches, partnership brokering, and multi-loop learning tools (Butler et al., 2017) . Given that the mainstreaming of ToC is a process, here we illustrate the evolution of the design, implementation and impact assessment of an AR4D project in West Africa. This demonstrates how the application of a bottom-up, reflective approach by the project team fostered a nascent community of practice which focussed on the analysis of context, widening the scope of consideration, and questioning of assumptions and paradigms. During 2011-2014, with funding from the Australian Department of Foreign Affairs and Trade, and in partnership with Australia's Commonwealth Scientific and Industrial Research Organization, the West and Central African Council for Agricultural Research and Development established six multi-country projects that agreed to apply participatory research-for-development methodological tools to address improvements in aspects of crop and livestock production (Hall et al., 2016) . One project was led by the Association for the Promotion of Livestock in the Sahel and Savannas (APESS), an internationally-funded association that works towards environmentally and economically sustainable animal husbandry and forage crop practices by traditional herders, and the greater involvement of animal producers in the economic, political and social development of West African countries. The project focused on development of opportunities for enhanced meat and milk output and profitability by animal producers in five countries in the Sahel (Sparrow and Traoré, 2017, this special issue) , and the team consisted of agronomy and animal science researchers from universities and government agricultural research organisations in five countries. During the initial phases of implementation, a logframe was drafted by the project team to meet donor requirements for the design, monitoring and evaluation of all development projects. At project initiation in 2011, the logframe was a daunting document with five result areas subtended by 46 sub-results, 53 assumptions, and 36 quantitative indicators with targets for project success. Assumptions were as general as "social and political stability is maintained at national and regional levels for smooth flow of resources, knowledge and technology" and targets were as specific as "at least 50% of producers have a pit manure heap by 2013". The mismatch between the specificity of assumptions and impact expectations was significant. It was clear that the project team had created the logframe as a top-down process because it was a donor requirement; saw the logframe as a checklist of activities, rather than as a tool to reflect upon and encapsulate a pathway to impact or ToC; and had little conception of a ToC other than as a purely additive structure (i.e. if the listed activities had been conducted and all the assumptions met, then impact would follow). At best, the logframe was a simple descriptive ToC, as per Fig. 1 . In late 2012, the project's first forage-crop trials became the subject of reflective questioning because they were primarily small-plot experiments with little involvement of local producers, and no consideration was being given to the whole-of-farm context that the technology would ultimately interact with. In response, a hypothetical programmatic model linking crop trials and in-field demonstrations to whole-of-farm economic models was developed to assess whether the project could enable adoption by producers. In mid-2013, under donor pressure to demonstrate how impact would truly arise, the logic linking crop trials to farm models was taken a step further to explore the potential influences of institutions, opportunities and barriers on a producer's ability to move from increased knowledge and understanding, to changed farm management and increased productivity. The team agreed that innovation and adoption of new technologies and practices by producers could only come about through participatory engagement of marketplace actors with producers. As an example, the team developed a causal model for the production and supply of fresh milk (rather than imported powdered milk) to local tourist hotelsincluding the identification of intermediate outcomes across different actors/sectors which lead to the ultimate outcome of increased profitability and well-being for households (Fig. 2) . This analysis was a major step forward from the initial logframe-as-checklist, and represented the beginnings of an explanatory project ToC, in the sense of Fig. 1 . It recognised the intermeshing of two complex systems, with complex dynamics through feedback loops and hence potentially non-linear outcomes. While not a complete, theoretically-underpinned ToC, the APESSled case study illustrates how a logframe approach was insufficient to explain a complicated situation, and forced the evolution of an explanatory ToC which began to question causal assumptions and paradigms. Had this exercise been developed further before the project closed in 2014, the team could have progressed further along the continuum, using the causal-model thinking as a foundation. However, several factors impeded this evolution. First, the agronomists and animal scientists who dominated the team were unwilling to use their combined years of field experience with producers and marketplace actors to consider social and institutional factors driving human behaviour, and instead determined that only social scientists were capable of doing so. Nonetheless, their learning did begin the establishment of a community of practice within APESS. The second factor was the amount of time required to allow the learning of the team to evolve from single-to double-loop learning, and with it a deeper explanatory ToC approach. In the event, four years were insufficient to enable this evolution to occur. The third factor was the lack of commitment and resources provided by the donor to encourage such an evolution. In fact, the development of the causal model by the project team was the result of pressure from the donor to demonstrate how impact would arise, rather than any interest in using a ToC to augment the prescribed logframe. ToC as an approach to designing and evaluating development interventions has undergone considerable conceptual refinement since it was first applied to AR4D more than a decade ago, and covers a range of interpretations from descriptive to reflective (Fig. 1) . The fact that 65% of our interviewees were applying ToC confirmed that it is an important concept that is being used widely, but with insufficient regard to divergent meanings, lack of coherent interpretation of its constituent concepts (theory, assumptions and evidence), confusion about its relationship with established planning tools (i.e. logframes), and potential for bureaucratic camouflaging of previously-applied approaches. If ToC is to contribute to theory-informed, effective and integrated design, implementation and evaluation of AR4D interventions, we recommend that practitioners and stakeholders should avoid using ToC casually as a "buzzword" (Cornwall, 2010) especially if there is a danger of camouflaging current practice. Instead, they should: • recognise the different potential meanings that can be attributed to ToC and be clear whether they are using it as a descriptive, explanatory or reflective tool; • state a theory or a set of hypotheses which are explicitly and iteratively tested through reflective learning as the intervention progresses; • articulate clearly the relationship of ToC to other approaches (e.g. the logframe) if they are to be used as alternatives or in parallel; • accept that ToC is a process rather than a single exercise, because theories and assumptions require testing through ongoing evidence collection and reflective analysis; and • allow adequate time, resources and reflection space for project team members and stakeholders to move towards systemic ToC thinking if starting from a logframe approach, especially for interventions on complex problems. A possible starting point for selecting a ToC appropriate to a specific problem or opportunity may be the Cynefin Framework (Snowden and Boone, 2007) , which provides guidance on behaviours of systems with different degrees of order and complexity, and the kinds of knowledge that are necessary to understand the system (Table 2) . These categories also seem to correspond to levels of learning required to adequately understand the system and to achieve change (Table 2) . Fig. 2 . Causal model of how higher-level well-being-related outcomes and impacts for APESS milk producers depend on the interactions between improved productivity and value chains, as well as a series of intermediate outcomes. Y.T. Maru et al. Agricultural Systems 165 (2018) 344-353 For Cynefin's "simple" systems, where cause-and-effect relationships are direct, clear, and more or less agreed by stakeholders, a descriptive ToC primarily demands clear articulation of causal links and external assumptions. In this case, a logframe with sufficient detail of the causal relationships between intervention and expected outcomes could be sufficient, and the ToC may only need to generate single-loop learning among technical domain experts to correct and improve delivery of interventions. An explanatory ToC is more appropriate if the system is "complicated", whereby there are many interactions among causal factors, and double-loop learning among multi-disciplinary experts and other stakeholders is necessary to understand and revisit underlying external and internal assumptions. "Complex" systems involve numerous interacting factors, interactions are nonlinear and involve feedback loops which result in small changes having potentially disproportionately large consequences. Cause and effect relationships are only understood retrospectively. Impact tends to be an emergent property, and interventions must be considered as experimental probing of the system (Rogers, 2008) . This class of system requires reflective monitoring (Van Mierlo et al., 2010) or developmental evaluation (Patton, 2011) to track dynamics of the intervention-problem-context interaction, and continuously adapt interventions to achieve desired outcomes. Participatory triple-loop learning and a transdisciplinary approach among stakeholders who bring multiple forms of knowledge is essential to permit transformative understanding of cause and effect relationships, solutions, and processes of institutional change (Pahl-Wostl, 2009 ). Chaotic systems are characterised by turbulence that rapidly becomes highly unordered and unmanageable (Lazaroff and Snowden, 2006) . Cause and effect are discernible only in retrospect, if at all. The future of these systems is not knowable or predictable. An agriculturerelated example was the confusion and panic in the early stages of the bovine spongiform encephalopathy (BSE or Mad Cow Disease) outbreak in the UK, which was assumed as a known problem that didn't affect humans, but only in retrospect was it found that the agents were prions which could be contracted by humans (French, 2013) . Other examples could be coup d'états, conflicts that lead to genocide, emergence of novel diseases such as Severe Acute Respiratory Syndrome (SARS), large-scale terror attack such as 9/11, or nuclear accidents such as Chornobyl (French, 2013; Lazaroff and Snowden, 2006; Snowden and Boone, 2007; Tadros and Allouche, 2017) . There is no a priori ToC that can be applied to chaotic situations, other than rapid top-down action which is necessary to create a central attractor for stability. Real time and rapid learning is necessary to sense any signal of pattern emerging from the action and to respond adaptively (French, 2013; Hummelbrunner and Jones, 2013; Lazaroff and Snowden, 2006) . A key challenge of matching a ToC to the level of system complexity is that stakeholders may differ in their views of the complexity of the problem. A complex food security problem may be treated by some stakeholders as complicated or even a simple farming productivity issue (Foran et al., 2014) . The reasons for varying stakeholder perceptions may include a desire to advance a particular pathway or a specific technology, or a donor's desire to see tangible results in a hurry. Several authors have reported the unsatisfactory outcome of problem oversimplification (Hummelbrunner, 2010; Ramalingam, 2013) . Resultsbased approaches that instil unrealistic certainty of outcomes encourage aid recipients to neglect hard and complex problems that may not show immediate and visible outcomes (Reeler, 2007) . Even in organisations with extensive experience of AR4D, attempts to apply reflective ToC to complex problems with long time horizons have been met with misunderstanding and apprehension (Douthwaite and Hoffecker, 2017) . In chaotic emergency situations decisive and rapid top down responses may be imperative in the short term to stabilise an unmanageable situation. However, there is a danger that responses in such circumstances often persist, even when the challenge has changed to a complex or complicated system which requires a different approach (Adams, 2017; Snowden and Boone, 2007) . These challenges have implications for the resourcing of skills and capacity-building for ToC approaches by donors, researchers and implementing organisations. A significant commitment is also required to create an environment that encourages critical reflection and different levels of individual and social learning. Such commitment is crucial when developing ToC as part of the design of interventions as probes into complex problems, given that solutions are not likely to be obvious from the outset, and flexible adaptation will be required throughout the intervention. Well-executed and effective mainstreaming of ToC cannot be expected to happen at once. As shown in the case study, it may require a gradual learning process for practitioners, researchers and donors alike. While the growing interest in ToC is an opportunity, it also presents a risk to ToC becoming another burdensome requirement, a buzzword, or a crude donor accountability instrument. To counter this, we concur with Valters (2014) that donors, implementers and researchers who understand ToC and its potential should build communities of practice (see Wenger, 1998 ) that take the complexity of social-ecological change seriously, and promote the responsible mainstreaming of ToC. While ToC is not a panacea to all challenges of design and evaluation for impact in AR4D, mainstreaming it appropriately can make a significant contribution. Our evidence suggests that much could be done towards this goal by providing conceptual clarity, awareness and training among AR4D researchers, practitioners and donors. It is also essential that systemic learning from practice occurs to build up a repertoire of ToC experience and capability. Part of this task is to find practical ways of evaluating the relative effectiveness of ToC to achieve donor and other international goals, such as the Sustainable Development Goals, compared with other result-based approaches such as logframes. (2007) Y.T. Maru et al. Agricultural Systems 165 (2018) 344-353 How Chronic Violence Affects Human Development, Social Relations, and the Practice of Citizenship: A Systemic Framework for Action Theory of Change: A Practical Tool for Action, Results and Learning Single-loop and double-loop models in research on decision making On Organizational Learning The Practice of Social Research The Use and Abuse of the Logical Framework Approach Historical divergence in public management of foodgrain systems in India and Bangladesh: opportunities to enhance food security Stripe Review of Social Sciences in the CGIAR. Available at SSRN 1844803 Learning about Theories of Change for the Monitoring and Evaluation of Research Uptake Evaluation of the CGIAR Research Program on Aquatic Agricultural Systems (AAS) Theories of change and realistic evaluation peas in a pod or apples and oranges? Produces Few Results: An Evaluation of the Long-Run Institutional Development Impacts of World Bank Environmental Projects Priming adaptation pathways through adaptive co-management: design and evaluation for developing countries A hierarchy of needs for achieving impact in international research for development A theory-driven evaluation perspective on mixed methods research Evaluating with sense the theory-driven approach Issues in the theory-driven perspective Applying a theory of change approach to the evaluation of comprehensive community initiatives: progress, prospects, and problems Introductory Overview-Buzzwords and Fuzzwords: Deconstructing Development Discourse Evaluating the effectiveness of the logical framework system in practice The logical framework: an easy escape, a straitjacket, or a useful planning tool? Scale, complexity and the representation of theories of change Political instability and food security Participatory Impact Pathways Analysis (PIPA) and Research Priority Assessment Towards a complexity-aware theory of change for participatory research programs working within agricultural innovation systems Impact pathway evaluation: an approach for achieving and attributing impact in complex systems Impact pathway evaluation of an integrated < i > Striga hermonthica < /i > control project in Northern Nigeria Using Theory of Change to Achieve Chaos, Complexity, and Sociology: Myths, Models, and Theories Diversity Management: Triple Loop Learning Taking complexity in food systems seriously: an interdisciplinary analysis Cynefin, statistics and decision analysis Purposeful Program Theory: Effective Use of Theories of Change and Logic Models Logical Frameworks': A Critical Assessment: Managerial Theory, Pluralistic Practice Discovery of Grounded Theory: Strategies for Qualitative Research The Discovery of Grounded Theory: Strategies for Qualitative Research Can Theories of Change Help Researchers (or their Funders) Have More Impact? Impact of Social Sciences Blog The Origins and Implications of Using Innovation Systems Perspectives in the Design and Implementation of Agricultural Research Projects: Some Personal Observations The Journey to R4D: An Institutional History of an Australian Initiative on Food Security in Africa. Innovation Systems: Towards Effective Strategies in Support of Smallholder Farmers An innovation systems approach to institutional change: smallholder development in West Africa Triggering regime change: a comparative analysis of the performance of innovation platforms that attempted to change the institutional context for nine agricultural domains in West Africa Beyond Logframe: Critique, Variations and Alternatives. Beyond Logframe; Using Systems Concepts in Evaluation A Guide to Managing in the Face of Complexity Programs, projects and learning inquiries: institutional mediation of innovation in research for development Theory of Change Review: A Report Commissioned by Comic Relief. Comic Relief CCAFS Theory of Change Facilitation Guide Evolution of Systems Approaches to Agricultural Innovation: Concepts, Analysis and Interventions. Farming Systems Research into the 21st Century: The New Dynamic A non-parametric approach to the estimation of welfare measures in discrete response valuation studies Innovation is a Social Process: What Does this Mean for Impact Assessment in Agricultural Research. Publication of the Section on Strategic Development of GTZ Expanding the use of impact assessment and evaluation in agricultural research and development Mapping Change: Using a Theory of Change to Guide Planning and Evaluation. Ford Foundation Integrated agricultural research for development (IAR4D) from a theory of change perspective Contribution analysis: coming of age? Useful theory of change models The COM-B Theory of Change Model The Practice of Institutionalizing Ideas: Institutionalizing'Popular Power Designing complex development programs. In: Working with Assumptions in International Development Program Evaluation Challenges of food security in Africa and the way forward A conceptual framework for analysing adaptive capacity and multilevel learning processes in resource governance regimes Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use Realistic Evaluation Between logframes and theory of change: reviewing debates and a practical experience NVivo Qualitative Data Analysis Software Aid on the Edge of Chaos: Rethinking International Cooperation in a Complex World A Theory of Social Change and Implications for Practice: Planning, Monitoring and Evaluation Using programme theory to evaluate complicated and complex aspects of interventions Diffusion of Innovations Theory of Change: Methodological Briefs-Impact Evaluation No Program theory evaluation: practice, promise, and problems Impact Assessment of Agricultural Research: Context and State of the Art Q-Squared: Combining Qualitative and Quantitative Approaches in Poverty Analysis Cynefin: A Sense of Time and Space A leader's framework for decision making Limits to the applicability of the innovation platform approach for agricultural development in West Africa: socio-economic factors constrain stakeholder engagement and confidence Impact pathway analysis: an approach to strengthening the impact orientation of agricultural research Is mainstreaming transformative? Theorizing mainstreaming in the context of diversity and deliberation Understanding Theory of Change in International Development: A Review of Existing Knowledge Political settlements as a violent process: deconstructing the relationship between political settlements and intrinsic, instrumental and resultant forms of violence Responding to global change: a theory of change approach to making agricultural research for development outcome-based Theories of Change in International Development: Communication, Learning, or Accountability? Justice and Security Research Programme Learning towards system innovation: evaluating a systemic instrument Review of the Use of 'Theory of Change'in International Development Methods for Assessing Program Effectiveness Nothing as practical as good theory: exploring theory-based evaluation for comprehensive community initiatives for children and families Theory-based evaluation: past, present, and future Theory-Based Evaluation: Theories of Change for Poverty Reduction Programs. Evaluation and Povery Reduction Communities of practice: learning as a social system Toward a plurality of methods in project evaluation: a contextualised approach to understanding impact trajectories and efficacy The research was conducted as part of CSIRO-BecA-CORAF/ WECARD Africa Food Security Initiative funded by the Australian Government's Department of Foreign Affairs and Trade (Funding Agreement No. 57685), as part of its overseas development assistance program. We thank two anonymous reviewers for their valuable comments that helped to refine this paper.