key: cord-0286031-ov5dwwqi authors: Hall, J.; Bingham, D. D.; Seims, A.; Dogra, S. A.; Burkhardt, J.; Nobles, J.; McKenna, J.; Bryant, M.; Barber, S.; Daly-Smith, A. title: A whole system approach to increasing childrens physical activity in a multi-ethnic UK city: a process evaluation protocol date: 2021-05-28 journal: nan DOI: 10.1101/2021.05.26.21257853 sha: 4e996cc49d08ef4cb917a2b6cc49812eed869a2b doc_id: 286031 cord_uid: ov5dwwqi Background: Engaging in regular physical activity requires continued complex decision-making in varied and dynamic individual, social and structural contexts. Widespread shortfalls of physical activity interventions suggests the complex underlying mechanisms of change are not yet fully understood. More insightful process evaluations are needed to design and implement more effective approaches. This paper describes the protocol for a process evaluation of the JU:MP programme, a whole systems approach to increasing physical activity in children and young people aged 5-14 years in North Bradford, UK. Methods: This process evaluation, underpinned by realist philosophy, aims to understand the development and implementation of the JU:MP programme and the mechanisms by which JU:MP influences physical activity in children and young people. It also aims to explore behaviour change across wider policy, strategy and neighbourhood systems. A mixed method data collection approach will include semi-structured interview, observation, documentary analysis, surveys, and participatory evaluation methods including reflections and ripple effect mapping. Discussion: Not only is this an innovative approach to process evaluation but it will also feed into iterative programme development to generate evidence-based practice and deliver practice-based evidence. This paper advances knowledge regarding the development of process evaluations for evaluating systems interventions, and emphasises the importance of process evaluation. 7 streams that are delivered across the whole LDP area, and nine that are developed and delivered at 152 a neighbourhood level (see Figure 3 ). Initially, the neighbourhood approach was operationalised within three 'Pioneer Neighbourhoods' 167 (pathfinder phase -2019-2021) to undertake a test and learn process. Subsequently, the programme 168 will be delivered in the five remaining neighbourhoods (2021) (2022) (2023) (2024) , to cover the whole LDP area. The accelerator phase neighbourhoods are further broken down into those that are directly 170 facilitated by the JU:MP team, as in the pathfinder phase (n = 3), and those whose delivery will be 171 externally commissioned (n = 2). The programme model is illustrated in but that knowledge is 'value-laden' and as such we can only understand reality from within a 176 particular discourse [37] . Realism holds that reality exists in an open-system, meaning that attention 177 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.26.21257853 doi: medRxiv preprint interact to influence outcomes [38] . Process evaluation is typically understood as "the evaluation of explain how outcomes are reached" [29] . Underpinned by realist principles, the role of context is prioritised in establishing intervention 182 (in)effectiveness [29] . Examining context implies focusing on social processes to establish an 183 understanding of how different notions of intervention feasibility, acceptability and effectiveness 184 can be framed. Another part of realist evaluation allows the development and / or refinement of 185 theories relating to mechanisms of change, focusing on context-mechanism-outcome configurations. This supports the iterative development of programme logic models and theories of change [39] . However, a realist approach acknowledges that people attach meaning to experiences, and 188 meanings are implicated within causal processes [37] ; behaviour therefore cannot be fully explained, 189 as people are conscious beings that act back on the structures and processes of social life [40] . Within the complex intervention evaluation field, recent calls to embed complexity science and 191 systems principles within process evaluation design reflect a move towards understanding how The overarching aim of the process evaluation is to understand the programme implementation and 206 the mechanisms through which JU:MP influences behaviour change across the neighbourhood, and 207 wider policy and strategy systems that it is seeking to influence. The evaluation also facilitates 208 dynamic system change via informing the refinement of the programme and associated theory of 209 change. To address these aims, and in accordance with the JU:MP delivery approach, the process 210 evaluation includes three distinct but interrelated packages of work: (1) a strategic-level evaluation, (2) a neighbourhood-level evaluation, and (3) an end-user evaluation. Table 1 provides an overview 212 of the scope and objectives of each process evaluation work package. Stakeholders involved in the development and delivery of JU:MP at a strategic-level, i.e. beyond 222 individual neighbourhoods, will be invited to participate. Data collection methods include surveys, 223 semi-structured interviews, participant observations, and reflections, which will all be implemented 224 at multiple time points throughout programme delivery; see section 2.6 for further detail. The 225 evaluation will iteratively refine as priorities surface; for example, we have recently incorporated a 226 sub-study to provide a 'deep dive' into the strategic-influencing work of JU:MP to examine the wider 227 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 28, 2021. intended and unintended impacts of city-wide policy and strategic working related to PA, following 230 the addition of policy and strategy as a theme within the theory of change. A minimum data-set will be collected from each neighbourhood, with additional data collection 238 occurring within selected 'deep dive' neighbourhoods. These neighbourhoods will include one from 239 the pioneer neighbourhood phase (with the primary aim of piloting and refining the data collection Amendments to the evaluation protocol will be made following piloting and prior to implementing The end-user process evaluation will examine the experiences and impact of JU:MP amongst 254 children and families. This will feature focus groups with children and parents/guardians from across 255 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.26.21257853 doi: medRxiv preprint the accelerator direct delivery neighbourhoods, approximately 12 months and 24 months following approximately four local families. Citizen science methods will be adopted, which involves members 258 of the public (non-scientists) collecting and analysing data, in collaboration with researchers [ 38], to 259 foster community engagement. Multiple and innovative methods of data collection will be employed, which could include written or 261 video diaries, or photo-elicitation techniques, walk-and-talk interviews, but crucially, the families will 262 be engaged in developing the research approach, collecting and analysing their own data, and (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. All potential participants across both the strategic and neighbourhood level studies will be engaged 310 in the design and delivery of JU:MP, and as such will already be known and identifiable to the 311 research team, via the implementation team. Potential participants will be given an information 312 sheet for the research, and informed consent will be obtained prior to data collection commencing. Data collection will take place at multiple time-points over a significant time-period (up to for years). At each data collection 'point', participants will be verbally reminded that they are taking part in the 315 study and what it involves, and will be given a verbal reminder to let the researcher know at any 316 time if they wish to withdraw their consent to participate. This section provides a rationale for and description of each data collection method that is being 319 utilised within the JU:MP process evaluation. Table 2 (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 28, 2021. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 28, 2021. Meeting and workshop notes and neighbourhood action plans will be included in qualitative 379 analyses (table 2) . Additionally, key documents such as service agreements and project plans will be 380 requested from stakeholders prior to interviews, to aid the interview process e.g. discussing how 381 and why plans were delivered as intended or amended. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Qualitative data including semi-structured interview data, reflections, key documents including 409 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Influences on behaviour survey: baseline, 6months, 12-months and 24 months (All participants, all neighbourhoods) Network mapping survey: baseline, 6-months, 12-months and 24 months (All recruited participants, all neighbourhoods) Process observations of meetings including: Implementation team meetings: one in every four (attended by core team members such as the programme director, community engagement managers and communications officer) Other key strategic meetings identified in collaboration with the implementation team Process observations of action group workshops: every workshop, approximately once every six weeks (deep-dive neighbourhoods only) Key documents for each work stream collated every 6 months (including service agreements, project plans and evaluations) Action group workshop notes (All neighbourhoods) Neighbourhood action plans (All neighbourhoods) Interviews with around 20 strategic stakeholders every 6 months (including members of the core team and one strategic lead for each workstream at each time point) Interviews with around six additional wider stakeholders every 12 months (three members of the executive board and three members of the strategic development) Interviews with around 20 neighbourhood stakeholders at 6 and 18 months (including key delivery stakeholders such as JU:MP connector, Islamic Religious Setting lead, school lead etc. from across deep-dive neighbourhoods only) Interviews with around two commissioned organisation stakeholders at 6 and 18 months (commissioned neighbourhoods) Group reflections embed into key meetings: Weekly implementation team meetings: one in every four (attended by core team members such as the programme director, community engagement managers and comms officer) Weekly research team meetings: one in every four (attended by core team members such as the research directors and research fellows) - REM workshops embedded into strategic development group meetings: every 6 months REM workshops embedded into action group meetings: every 6 months (all neighbourhoods) Table 2 . Data collection methods for the strategic and neighbourhood process evaluation. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.26.21257853 doi: medRxiv preprint meeting notes, and process observation summaries will be analysed using a framework approach 411 [61] . Framework analysis is a type of thematic analysis aimed at providing descriptive and/or 412 explanatory findings clustered around themes. Uniquely, framework analysis features using a matrix 413 to systematically reduce the data. The key steps involved include (1) familiarisation, (2) identifying a 414 thematic framework, (3) indexing (applying the thematic framework to the data set), (4) charting 415 (entering data into framework matrices), and (5) mapping and interpretation [56] . (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 28, 2021. Given the time often required to conduct robust qualitative work, a challenge here is ensuring that 479 the process evaluation findings remain 'relevant' as the JU:MP programme progresses and evolves in 480 an agile way It is, therefore, important to ensure that the findings are fed back in a timely manner 481 and in an appropriate format to allow the team to 'step back' and engage in systematic planning. Whilst the evaluation outlined in this paper is resource-intensive, it is set up to generate a deep and 483 rich understanding of the processes underpinning programme design, implementation and impact, 484 and thus will be invaluable in supporting other communities to apply a similar approach and / or to 485 learn from things that have not delivered expected successes. An embedded research team is critical for the development of research-practice partnerships, which 487 facilitates evidence-based practice, and the development of practice-based evidence through the 488 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.26.21257853 doi: medRxiv preprint worked alongside the programme team to develop a protocol that aligns with and meets the needs 493 of the programme, and involvement is also required to produce detailed and in-depth observational 494 records that reflect participant experiences. Detachment is also required throughout the research 495 process, for example when analysing data, to ensure that the analysis is reality-congruent and The process evaluation outlined within this paper forms part of a wider evaluation approach, which 498 includes an effectiveness evaluation (neighbourhood control trial, and a before and after evaluation (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The dataset that will be generated and analysed during the current study are not publicly available 532 to reserve the anonymity of research participants. The authors report no competing interests. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 28, 2021. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 28, 2021. ; https://doi.org/10.1101/2021.05.26.21257853 doi: medRxiv preprint An ecological approach to 648 creating active living communities Realism and Social Science Building realist program theory for large complex and 651 messy interventions Realist complex intervention 653 science: applying realist principles across all phases of the Medical Research Council 654 framework for developing and evaluating complex interventions Breaking the boundaries between nursing and sociology: a critical realist 656 ethnography of the theory-practice gap NIHR SPHR Guidance 658 on Systems Approaches to Local Public Health Evaluation NIHR SPHR Guidance on Systems Approaches to Local Public Health Evaluation Qualitative process evaluation from 663 a complex systems perspective: a systematic review and framework for public health 664 evaluators Leveraging citizen science 666 and information technology for population physical activity promotion Fostering 669 implementation of health services research findings into practice: a consolidated framework 670 for advancing implementation science CFIR simplified: Pragmatic 672 application of and adaptations to the Consolidated Framework for Implementation Research 673 (CFIR) for evaluation of a patient-centered care transformation within a learning health 674 system The behaviour change wheel: a new method for 676 characterising and designing behaviour change interventions Validation of the theoretical domains framework for use in 678 behaviour change and implementation research Acceptability, reliability, and 680 validity of a brief measure of capabilities, opportunities, and motivations Factors influencing primary health care professionals' physical activity promotion behaviors: 684 a systematic review Mapping networks: exploring the utility of social network 686 analysis in management research and practice Social Network Analysis Toolkit: Planning a Social Network Analysis Preparing and conducting interviews to collect data Learning through reflection: the critical role of reflection in work-based learning No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. 53. Watson TJ. Ethnography, reality, and truth: the vital need for studies of 'how things work' in 695 organizations and management A field guide to ripple effects mapping. 700 University of Minnesota Libraries Publishing Assessing a rural academic-community partnership using ripple 702 effect mapping Ripple 704 effects mapping: capturing the wider impacts of systems change efforts in public health Turning the tide on 707 poverty: documenting impacts through ripple effect mapping Qualitative data analysis for applied policy research Analyzing qualitative data Using the framework method for the 711 analysis of qualitative data in multi-disciplinary health research Framework analysis: a worked 714 example of a study exploring young people's experiences of depression The potential of reflective practice to develop individual orthopaedic nurse 717 practitioners and their practice Protocol for the process 719 evaluation of the promoting activity, independence and stability in early dementia (PrAISED), 720 following changes required by the COVID-19 pandemic GoActive: a protocol 722 for the mixed methods process evaluation of a school-based physical activity promotion 723 programme for 13-14year old adolescents Protocol for a mixed 725 methods process evaluation of a hybrid implementation-effectiveness trial of a scaled-up 726 whole-school physical activity program for adolescents Process evaluation of a whole-of-community 729 systems approach to address childhood obesity in western Victoria Evaluating the design and implementation of the 732 whole systems integrated care programme in North West London: why commissioning 733 proved (again) to be the weakest link Public health asks of systems science: to advance our evidence-based practice, 735 can you help us get more practice-based evidence? Embedded 737 research: a promising way to create evidence-informed impact in public health? Involved-detachment: A balance of passion and reason in feminisms and 740 gender-related research in sport, tourism and sports tourism Moving from collision to integration: Reflecting on the experience of mixed 743 methods No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity Coming at things differently: Future directions of possible 2025 Design Pathfinder Phase Accelerator Phase (2021-2024) 3 Pioneer Neighbourhoods Continue…