Using photographic interpretation to evaluate the safety of home environments Preventive Medicine Reports 4 (2016) 459–462 Contents lists available at ScienceDirect Preventive Medicine Reports journal homepage: http://ees.elsevier.com/pmedr Using photographic interpretation to evaluate the safety of home environments Carlee Lehna PhD, APRN-BC a,b,⁎, Stephanie Twyman BSN, BA, RN a, John Myers PhD b a University of Louisville School of Nursing, United States b School of Medicine, Department of Pediatrics, United States ⁎ Corresponding author at: 555 S. Floyd St., Louisville, K E-mail address: c0lehn01@louisville.edu (C. Lehna). http://dx.doi.org/10.1016/j.pmedr.2016.08.014 2211-3355/© 2016 Published by Elsevier Inc. This is an op a b s t r a c t a r t i c l e i n f o Article history: Received 27 June 2016 Received in revised form 18 August 2016 Accepted 24 August 2016 Available online 26 August 2016 In the US there were 400,000 home fires resulting in 2755 deaths, 12,450 injuries, and $6.9B lost. The purpose of this study was to evaluate the content-validity of photographs taken in the home for use as an educational instrument to teach about “safe” and “unsafe” fire safety practice for adults and older adults. A total of 73 home fire safety experts were provided 27 photographs to evaluate home fire safety practice. Initially, a Krippendorff's alpha was calculated for the first 24 questions to evaluate inter-rater reliability, and differences in demographics were evaluated. Unique codes and themes for the last three questions were identified and inter-rater reliability examined. A majority of re- spondents were female (n = 43, 60.6%), college educated (n = 61, 83.6%), nurses (n = 25, 33.8%), or worked for a fire department (n = 21, 29.6%). Their mean age was 45.5 years and they had 11.05 years of experience. The first 24 questions had high inter-rater reliability (Krippendorff α = 0.831). No significant differences existed between the strata of the demographic variables (all p-values N 0.05). Similarly, based on the codes and themes identified, the last three questions had moderate-to-good inter-rater reliability (Krippendorff α = 0.764). Providing photographs as a ‘seek-and-find’ or ‘What's wrong with this picture?’ tools and simplified visual images is an excellent way to aid recognition of unsafe home fire safety environments. Education through non-traditional visual methods increases the possibility of change for diverse low-literacy populations. © 2016 Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Keywords: Photographs Home fire safety Instrument development Diverse low-literacy populations 1. Introduction Nationally, 380,300 residential fires were reported in 2013 which in- cluded 2755 deaths, 12,450 injuries, and $6.9B lost (United State Fire Administration (USFA), 2015). Cooking (188,000) and heating (49,000) were the leading causes of fire (United State Fire Administration (USFA), 2015). Deaths were related to carelessness (465), electrical mal- function (335), smoking (320), and intentional reasons (320) (United State Fire Administration (USFA), 2015). Injuries were caused by cooking (4225), open flame (1150), and carelessness (1075) (United State Fire Administration (USFA), 2015). Carelessness ($1B) and electrical mal- function ($923M) accounted for the greatest fire dollar losses (United State Fire Administration (USFA), 2015). Though preventable, residential fires are significant causes of death, injury, and dollars lost in the United States. Assessment of home fire safety (HFS) practices most frequently in- volved checking for fire alarm location(s), and functionality (Gielen et al., 2013; Miller et al., 2014; Parker et al., 2013; Sidman et al., 2011). Less often was tap hot water temperature tested (Parker et al., 2013), carbon monoxide alarms presence assessed (McDonald et al., 2013), presence and practice of fire escape plans determined (Yang et al., 2006), or participant knowledge regarding fire safety obtained (Parker Y 40202, United States. en access article under the CC BY-NC et al., 2013). The United States Fire Administration's Home Fire Safety Checklist examines HFS from multiple practice perspectives (United States Fire Administration). The checklist has eight subscales (i.e., smoke alarms, cooking safety, electrical and appliance safety, carbon monoxide (CO) alarms, candle safety, smoking safety, heating safety, and home escape plans) with three to six items per subscale (United States Fire Administration). To date, no reported evidence of reliability or validity is available for this instrument. The practice of HFS is com- plex, needing a comprehensive approach using an instrument with ev- idence of reliability and validity. In two previous studies by the authors on HFS education, the team studied 103 parents of newborns (with and without special needs) (Lehna et al., 2015a) and 125 older adults (community active and homebound) (Lehna et al., 2015b). Participants were asked to watch a 5-minute HFS DVD; complete a pre, post, and 2-week follow-up phone call. In addition, in-home fire safety checks using the Home Fire Safety Checklist were conducted (Lehna et al., 2015a; Lehna et al., 2015b). At the time of the HFS check some participants of the parent study provided researchers permission to take photographs of “safe” and “un- safe” home fire hazards within their residence. Initially, to examine the content validity of the photographs, two fire inspectors independently rated the photographs (inter-rater reliability). The local fire inspectors rated all the photographs twice, two-weeks apart in time (test-retest re- liability) (Lehna et al., 2015c). Inter-rater reliability between the two -ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). http://creativecommons.org/licenses/by-nc-nd/4.0/ http://dx.doi.org/10.1016/j.pmedr.2016.08.014 mailto:c0lehn01@louisville.edu http://dx.doi.org/10.1016/j.pmedr.2016.08.014 http://creativecommons.org/licenses/by-nc-nd/4.0/ http://www.sciencedirect.com/science/journal/ http://ees.elsevier.com/pmedr 460 C. Lehna et al. / Preventive Medicine Reports 4 (2016) 459–462 fire inspectors was 0.92, and the intra-rater reliability was 0.91. When community participants and HFS team members rated the photographs the inter-rater reliability rating was 0.76 using Krippendorff's Alpha Coef- ficient (Lehna et al., 2015c). As community members rated the photographs for content validity, team members found that the photographs evoked discussion between older adult participants about similar situations from their past (Lehna et al., 2015c). Including photographs in a HFS education program could potentially increase knowledge retention adding to improvement in HFS practice, especially for those who were older, had lower literacy, or for whom English was a second language. With the exception of the two studies (Lehna et al., 2015a; Lehna et al., 2015b), assessment of HFS practices remains fragmented and dis- jointed. Researchers mainly focus on smoke alarm placement and func- tionality (Gielen et al., 2013; Miller et al., 2014; Parker et al., 2013; Sidman et al., 2011). The research team is attempting to shift the para- digm to prevention (eradicating unsafe practices) from a traditional re- actionary approach (fire alarms). There was a need to further evaluate the effectiveness of the photographs obtained in the two previous stud- ies (Lehna et al., 2015a; Lehna et al., 2015b) with both a local and a na- tional group of burn prevention experts. Photograph 7. Unsafe - Candle in shallow container; surrounding flammable materials potential fire hazard. 2. Methods 2.1. Measure For the current project focusing on photographic instrument develop- ment, 27 photographs were grouped according to the United States Fire Administration's Home Fire Safety Checklist (United States Fire Administration) and presented in two formats: as a PowerPoint presenta- tion and as a poster. Expert participants used either presentation format for their evaluation. Participants rated whether photographs were “safe”, “unsafe”, and “unable to determine”. While n = 18 (75%) photo- graphs were anticipated to be viewed as unsafe, n = 6 (25.0%) were an- ticipated to be viewed as safe. The authors purposively set the allocation rate at 3:1. The ordering of photographs was randomly determined (see Table 2). `For 24 of the photographs, the 73 home fire safety respondents sim- ply recorded if they felt the practice in the photograph was “safe” or “unsafe.” For the three additional photographs, the practice was clearly unsafe and the respondents answered an open-ended question to de- scribe the unsafe practice. Examples of some of the unique HFS photo- graphs include: unsafe cooking practices (see Photograph 5); unsafe candle safety (see Photograph 7); and safe fire escape practice (see Photograph 20). Photograph 5. Unsafe cooking practices. 2.2. Procedure This anonymous survey was institutional review board exempt. Re- turn of the completed forms considered consent. Initially, a Krippendorff's alpha was calculated for the first 24 questions to evaluate inter-rater reliability, and differences in demographics were evaluated. Unique codes and themes for the last three questions were identified and inter- rater reliability examined (Photographs 25 and 26). 2.3. Sample Participants were recruited at the 2015 American Burn Association Annual Conference and Meeting. At the Burn Prevention Committee Photograph 20. Safe – In-home posted fire escape plan. Photograph 25. Unsafe - Electrical strip without surge protector, cords under mat. Table 1 Demographics of expert panel. Categorical variable N = 73 (%) Female 43 (60.7%) Education HS/GED 12 (16.4%) College 42 (57.5%) Advance degree (Masters, PhD, MD) 19 (26.1%) Profession Fire Department Staff 21 (29.6%) Clinician (RN, MD) 31 (43.7%) OT/PT 6 (8.5%) Other (law, research, admin, etc.) 13 (18.3%) Burn Prevention 44 (60.3%) Continuous variable Mean (SD) Age 45.5 (11.8) Yrs. experience 11.1 (8.7) Table 2 Photograph alphas. Photo Krippendorff α 461C. Lehna et al. / Preventive Medicine Reports 4 (2016) 459–462 meeting, the PowerPoint presentation was presented to members and photographs were rated. The PowerPoint presentation also was emailed to all members to have experts rate at their home location. The poster was available in the exhibit room at the Burn Prevention booth for any participant to judge. Completed response sheets were either submitted in-person or by email to the principal investigator (PI). Locally, fire inspector participants were recruited from an urban cen- tral fire state headquarters during roll call. Recruitment also occurred at a monthly Safe Kids Coalition meeting where members completed the evaluation while either viewing the PowerPoint presentation or exam- ining the poster. 3. Results A majority of respondents were female (n = 43, 60.7%), college ed- ucated (n = 61, 83.6%), nurses (n = 25, 33.8%) or worked for a fire de- partment (n = 21, 29.6%). Their mean age was 45.5 years and they had 11.1 years of burn prevention experience (see Table 1). The first 24 questions had high inter-rater reliability (Krippendorff α = 0.831). Table 2 shows the individual alphas for the respective photographs. As shown in Table 3, a great deal of agreement in response/view (safe vs. unsafe) of the individual photographs occurred (average agree- ment of 89.6%). No significant differences existed between the strata of the demographic variables and responses of safe vs. unsafe (all p- values N 0.05). Photograph 26. Unsafe - Flammable materials and chemicals on dryer. Based on the codes and themes identified, the final open-ended questions had moderate-to-good inter-rater reliability (Krippendorff α = 0.764). Participants who responded (n = 69, 100%), correctly iden- tified in photograph 25 two themes, the power strip was without the surge protector (n = 59, 85.5%) and cords were running under the mat (n = 10, 14.5%). For photograph 26, participants described two themes, the fire hazard was flammable chemicals (n = 46, 67.6%) and potentially flammable materials on the dryer/washer (n = 16, 23.5%). For photograph 27, the respondents had one theme, a wiring/cord issue (overload, too many wires/cords, etc.; n = 59, 90.8%). 4. Discussion This study further extends previous mixed method predominantly qualitative (ethnographic focus) support of photographic analysis ob- tained during HFS checks (Lehna et al., 2015c) through further valida- tion with experts. There is a paucity research on fire safety practice within individual homes. Currently, HFS education programs lack com- prehensive approaches for teaching and assessing home practice, often focusing on one or two aspects of HFS (e.g., smoke alarms, carbon mon- oxide alarms, scald prevention) (Gielen et al., 2013; Miller et al., 2014; Parker et al., 2013; Sidman et al., 2011; McDonald et al., 2013; Yang et al., 2006). Findings from this study, moderate to strong evidence of 1 0.777 2 0.785 3 0.821 4 0.777 5 0.874 6 0.785 7 0.816 8 0.618 9 0.621 10 0.827 11 0.906 12 0.832 13 0.833 14 0.846 15 0.832 16 0.806 17 0.786 18 0.774 19 0.706 20 0.829 21 0.809 22 0.851 23 0.853 24 0.894 Table 3 Most popular responses and agreement for individual photographs. Photograph Most popular response Percent of same response among panel N = 73 (%) 1 Safe 68 (93.2%) 2 Unsafe 70 (95.9%) 3 Unsafe 72 (98.6%) 4 Unsafe 69 (94.5%) 5 Unsafe 72 (98.6%) 6 Safe 62 (85.0%) 7 Unsafe 72 (98.6%) 8 Safe 62 (84.9%) 9 Unsafe 70 (95.9%) 10 Unsafe 65 (89.0%) 11 Safe 62 (84.9%) 12 Unsafe 70 (95.9%) 13 Unsafe 49 (67.1%) 14 Unsafe 73 (100.0%) 15 Unsafe 67 (91.8%) 16 Safe 49 (67.1%) 17 Unsafe 66 (90.4%) 18 Unsafe 67 (91.8%) 19 Unsafe 69 (94.5%) 20 Unsafe 70 (95.9%) 21 Unsafe 41 (56.2%) 22 Unsafe 72 (98.6%) 23 Unsafe 70 (95.9%) 24 Safe 66 (90.4%) 462 C. Lehna et al. / Preventive Medicine Reports 4 (2016) 459–462 inter-rater reliability and validity, provide evidence to support use of “safe” or “unsafe” photographs for the basis of future HFS curriculum development (e.g., story book or PowerPoint formats). Use of multi- method education strategies has been found to be successful in teaching parents. Reimer and Kagan in a series of three consecutive studies developed and tested the effectiveness of a burn prevention curriculum for Amish children (Reimer and Kagan, 2012a; Reimer and Kagan, 2012b; Reimer and Kagan, 2013). By survey, mothers identified burn hazards unique Amish communities (Reimer and Kagan, 2012a). Using the survey infor- mation an education curriculum was developed which involved teachers telling stories and running through scenarios with an open- viewed doll house as a game to identify home burn hazards. Change in burn prevention knowledge was measured through pre- and post-test- ing (Reimer and Kagan, 2012b; Reimer and Kagan, 2013). Using a cur- riculum based on this unique community's burn prevention needs in different formats (storytelling and open-viewed doll house) was shown to be highly effective and culturally accepted (Reimer and Kagan, 2012b; Reimer and Kagan, 2013). Sinha and colleagues evaluated the effectiveness of “Tales of Burn Safety” comic book with two groups of school aged children, one from West Virginia (n = 74) and the other from West Bengal (n = 39). Change in student learning was evaluated by pre- and post-testing (Sinha et al., 2011). In implementing a multi-learning approach (words with supporting graphics) cues are both written and visual for learners which the authors found to be effective. 5. Conclusion Using ‘seek-and-find’ or ‘What's wrong with this picture?’ tools and simplified visual images aid in recognition of unsafe home environments. Fire and burn prevention education through multi- and non-traditional methods for older adults, migrant and refugee popula- tions, or those having low literacy increases the possibility of change. Transparency document The Transparency document associated with this article can be found, in online version. Conflict of interest statement All authors declare no conflict of interest. Acknowledgements Funding was provided by the Federal Emergency Management Agency Fire Prevention & Safety Grant # EMW-2012-FP-01181. To the 2015 American Burn Association Burn Prevention Committee; the Louisville Fire Department; and Safe Kids Louisville, led by Kosair Children's Hospital and Children's Hospital Foundation Office of Child Advocacy at Kosair Children's Hospital. References Gielen, A.C., Shields, W., Frattaroli, S., et al., 2013. Enhancing fire department home visit- ing programs: results of a community intervention trial. JBCR e250–e256. Lehna, C., Fahey, E., Janes, E., et al., 2015a. Home fire safety education for parents of new- borns. Burns 41 (6), 1199–1204. http://dx.doi.org/10.1016/j.burns.2015.02.009. Lehna, C., Coty, M.B., Fahey, E., et al., 2015b. Intervention study about the changes in home fire safety knowledge in urban older adults. Burns 41 (6), 1205–1211. http://dx.doi. org/10.1016/j.burns.2015.02.012. Lehna, C., Twyman, S., Fahey, E., et al., 2015c. “Worried about them when we left:” a mixed-methods essay. Qual. Rep. 20 (2), 49–62 Available at: http://www.nova.edu/ ssss/QR/QR20/2/lehna4.pdf. McDonald, E.M., Gielen, A.C., Shields, W.C., et al., 2013. Residential carbon monoxide (CO) poisoning risk: correlates of observed CO alarm use in urban households. J. Environ. Health 76 (3), 26–32. Miller, T.R., Berger, G., Ballesteros, M.F., Bhattacharya, S., Gielen, A.C., Sheppard, M.S., 2014. Increasing smoke alarm operability through theory-based health education: a ran- domized trial. J. Epidemiol. Community Health 68, 1168–1174. Parker, E.M., Gielen, A.C., McDonald, E.M., et al., 2013. Fire and scald burn risks in urban communities: who is at risk and what do they believe about home safety? Health Ed. Research 28 (4), 599–611. Reimer, M.T., Kagan, R.J., 2012a. Development of a burn prevention teaching tool for Amish children. JBCR 33 (2), 259–264. http://dx.doi.org/10.1097/BCR.ob013e318233591c. Reimer, M.T., Kagan, R.J., 2012b. Pilot testing of a burn prevention tool for Amish children. JBCR 33 (2), 265–271. http://dx.doi.org/10.1097/BCR.0b013e3182335903. Reimer, M.T., Kagan, R.J., 2013. Multicenter testing of a burn prevention teaching tool for Amish children. JBCR 33 (1), 58–64. http://dx.doi.org/10.1097/BCR.0b013e318278169a. Sidman, E.A., Grossman, D.C., Mueller, E.A., 2011. Comprehensive smoke alarm coverage in lower economic status homes: alarm presence, functionality, and placement. J. Community Health 36, 525–533. Sinha, I., Patel, A., Sun Kim, F., MacCorkle, M.L., Watkins, J.F., 2011. Comic books can edu- cate children about burn safety in developing countries. JBCR 32 (4), e112–e117. http://dx.doi.org/10.1097/BCR.0b013e3182223c6f. United State Fire Administration (USFA), 2015. Residential Building Fire Trends (2004– 2013). USFA Fire Estimates Summary. FEMA Retrieved January 30, 2016 from: https://www.usfa.fema.gov/downloads/pdf/statistics/res_bldg_fire_estimates.pdf. United States Fire Administration, d. Home fire safety checklist Retrieved on 29 January 2016 from http://www.usfa.fema.gov/downloads/pdf/home_safety_checklist.pdf. Yang, J., Peek-Asa, C., Allareddy, V., Zwerling, C., Lundell, J., 2006. Perceived risk of home fire and escape plans in rural households. Am. J. Prev. Med. 30 (1), 7–12. http://dx.doi.org/10.1016/j.pmedr.2016.08.014 http://refhub.elsevier.com/S2211-3355(16)30104-8/rf0005 http://refhub.elsevier.com/S2211-3355(16)30104-8/rf0005 http://dx.doi.org/10.1016/j.burns.2015.02.009 http://dx.doi.org/10.1016/j.burns.2015.02.012 http://www.nova.edu/ssss/QR/QR20/2/lehna4.pdf http://www.nova.edu/ssss/QR/QR20/2/lehna4.pdf http://refhub.elsevier.com/S2211-3355(16)30104-8/rf0025 http://refhub.elsevier.com/S2211-3355(16)30104-8/rf0025 http://refhub.elsevier.com/S2211-3355(16)30104-8/rf0025 http://refhub.elsevier.com/S2211-3355(16)30104-8/rf0030 http://refhub.elsevier.com/S2211-3355(16)30104-8/rf0030 http://refhub.elsevier.com/S2211-3355(16)30104-8/rf0035 http://refhub.elsevier.com/S2211-3355(16)30104-8/rf0035 http://refhub.elsevier.com/S2211-3355(16)30104-8/rf0035 http://dx.doi.org/10.1097/BCR.ob013e318233591c http://dx.doi.org/10.1097/BCR.0b013e3182335903 http://dx.doi.org/10.1097/BCR.0b013e318278169a http://refhub.elsevier.com/S2211-3355(16)30104-8/rf0055 http://refhub.elsevier.com/S2211-3355(16)30104-8/rf0055 http://refhub.elsevier.com/S2211-3355(16)30104-8/rf0055 http://dx.doi.org/10.1097/BCR.0b013e3182223c6f https://www.usfa.fema.gov/downloads/pdf/statistics/res_bldg_fire_estimates.pdf http://www.usfa.fema.gov/downloads/pdf/home_safety_checklist.pdf http://refhub.elsevier.com/S2211-3355(16)30104-8/rf0075 http://refhub.elsevier.com/S2211-3355(16)30104-8/rf0075 Using photographic interpretation to evaluate the safety of home environments 1. Introduction 2. Methods 2.1. Measure 2.2. Procedure 2.3. Sample 3. Results 4. Discussion 5. Conclusion Transparency document Conflict of interest statement Acknowledgements References