1 volume 7 issue 14/2018 F I N G A L ’ S C A V E : T H E I N T E G R AT I O N O F R E A L - T I M E A U R A L I S AT I O N A N D 3 D M O D E L S Shona Kirsty Noble The Glasgow School of Art 167 Renfrew St Glasgow G3 6RQ United Kingdom s.noble@gsa.ac.uk Abstract: Fingal’s Cave: an Audiovisual Experience is an immersive virtual reality application that combines 3D models, a narrative soundscape and interactive auralisation in a recreation of a visit to Fingal’s Cave. This research explores the importance of audio in heritage visualisations and its practical implementation. Fingal’s Cave is a sea cave on the Isle of Staffa off the west coast of Scotland revered for its extraordinary acoustics. Audio is extremely important in the history and culture of Fingal’s Cave and it has long been romanticised, inspiring countless folklore, art, poetry and music. The visualisation is designed to encourage viewers to become a part of the cultural narrative and explore the cave for themselves, move around and speak to hear their voice auralised as it would be inside the cave. This is the first time the acoustic characteristics of a heritage site have been included in a visualisation in this interactive manner. This paper reviews whether auralisation is effective and meaningful and supports a creative response to heritage sites. The impact of the visualisation in terms of engaging with communities of interest and in the field of audio in heritage visualisation is discussed. The research suggests it is necessary that audio be included in heritage visualisations to give a full and complete understanding of how people experience it. Keywords: Real-time Auralisation, Virtual Reality, Heritage Visualisation, Acoustic Response, Fingal’s Cave, Staffa, Intangible Heritage, Audiovisual Data I n t r o d u c t i o n Fingal’s Cave: An Audiovisual Experience is a practical demonstration of the integration of auralised audio with a 3D model. In this research, the 3D model forms a representation of the visual elements of the example heritage site, Fingal’s cave, while the auralised audio forms a representation of the acoustic properties. While there is an extensive body of research into the importance of aurality and its inclusion in studies of cultural heritage mailto:s.noble@gsa.ac.uk S. K. Noble, Fingal’s Cave: The Integration of Real-Time Auralisation and 3D Models 2 (especially works by Foka & Arvidsson1, Mattern2 and Sterne3), and similarly in aural reproduction using digital technologies (see works by Brereton4, Guthrie5 and Laird6), there is further research needed that fully addresses the intersection between them. This means that key issues relating to engagement and how dissemination modes ultimately influence our perception of heritage sites could benefit from applying the principles of the prototypes in this research. The very ephemeral nature of sound makes it one of the most intangible forms of heritage, but through full acoustic analysis, it can be further understood and replicated. This research aligns with the work of Betts7 and Veitch8, in that it advocates moving away from a focus on vision as the sense that best describes an experience of a place, and instead invoking full embodiment. The application developed is a multisensory exploration featuring real-time auralisation to create an immersive experience that recreates the effects on sound of Fingal’s Cave. There is an interactive aspect which allows a participant to speak into a microphone and hear their own voice as if inside the cave. This draws upon research into virtual stage acoustics and involves real-time auralisation generated from an impulse response captured inside Fingal’s Cave. As well as application of real-time auralisations to the users’ voice, the virtual experience also has a narrative soundscape featuring readings of poetry written about the cave, quotes from its historical visitors and ambient wave recordings. The output is implemented in Unity, a free to use game development software, and viewed through a virtual reality Head Mounted Display (HMD). This work supports the ongoing research of the Historical Archaeological Research Project on Staffa (HARPS)9 which is a collaboration between the Glasgow School of Art’s School of Simulation and Visualisation and the National Trust for Scotland. HARPS is adopting a new multi-disciplinary approach to recording and exploring the historical archaeological potential of the island. Work includes excavation, Reflectance Transformation Imaging (RTI) of tourist graffiti in Fingal’s Cave, and laser scan, audio and photogrammetric surveys.10 HARPS is the source of the base data sets used in this project, specifically the laser scan and audio sound sweep data of Fingal’s Cave, as well as photogrammetric data of Staffa. Now a national nature reserve, the current owners of Staffa, The National Trust for 1 Anna Foka and Viktor Arvidsson, ‘Experiential Analogies: A Sonic Digital Ekphrasis as a Digital Humanities Project’, DHQ: Digital Humanities Quarterly 10, no. 2, 2016, http://www.diva-portal.org/smash/get/diva2:938475/FULLTEXT01.pdf. 2 Shannon Mattern, ‘Ear to the Wire: Listening to Historic Urban Infrastructures’, Amodern 2, no. October, 2013, 1–24, http://amodern.net/article/ ear-to-the-wire/. 3 Jonathan Sterne, The Audible Past, Duke University Press, 2003, https://culturetechnologypolitics.files.wordpress.com/2015/11/jonathan- sterne-the-audible-past-intro.pdf. 4 Jude S Brereton, Damian T Murphy and David M Howard, ‘The Virtual Singing Studio: A Loudspeaker-Based Room Acoustics Simulation for Real-Time Musical Performance’, Joint Baltic-Nordic Acoustic Meeting, Odense, Denmark, 2012, 1–8. 5 A Guthrie et al., ‘Using Ambisonics for Stage Acoustics Research’, International Symposium on Room Acoustics, 2013, 1–10, ftp:// s00275-s00279.cisti.nrc.ca/outgoing/ISRA2013/CD_ISRA2013/Papers/P117.pdf; Anne Guthrie, ‘Stage Acoustics for Musicians: A Multidimensional Approach Using 3D Ambisonic Technology’, Rensselaer Polytechnic Institute, 2014, http://www.fraufraulein.com/anne/ AnneGuthrie_2014_05_07_Dissertation_FinalSmall.pdf. 6 Iain Laird, Damian Murphy and Paul Chapman, ‘Comparison of Spatial Audio Techniques for Use in Stage Acoustics Laboratory Experiments’, EAA Joint Symposium on Auralization and Ambisonics, Berlin, Germany, 2014, 140–46, http://eprints.whiterose.ac.uk/85334/; Iain Laird, Damian Murphy, and Paul Chapman, ‘Energy-Based Calibration of Virtual Performance Systems’, C15th International Conference on Digital Audio Effects, York, 2012, 15–18, http://eprints.whiterose.ac.uk/75125/; Iain Laird, Damian Murphy, and Paul Chapman, ‘Spatialisation Accuracy of a Virtual Performance System’, Joint Baltic-Nordic Acoustics Meeting, Odense, Denmark, 2012; Iain Laird et al., ‘Development of a Virtual Performance Studio with Application of Virtual Acoustic Recording Methods’, Audio Engineering Society 130th Convention, London, 2011, 1–12. 7 Eleanor Betts, Towards a Multisensory Experience of Movement in the City of Rome’, Rome, Ostia, Pompeii: Movement and Space, Ray Laurence and David J Newsome, eds, Oxford University Press, 2011, 118–32, https://doi.org/10.1093/acprof:osobl/9780199583126.001.0001; Eleanor Betts, ‘The Multivalency of Sensory Artifacts in the City of Rome’, Senses of the Empire Multisensory Approaches to Roman Culture, Eleanor Betts, ed, 1st ed., Routledge, 2017, 23–38; Eleanor Betts, ‘The Sacred Landscape of Picenum (900-100 BC): Towards a Phenomenology of Cult Places’, Inhabiting Symbols: Symbol and Image in the Ancient Mediterranean, Edward Wilkins, John B. and Herring, Accordia S, eds, Accordia Research Institute, University of London, 2003, 101–20. 8 Jeffrey Veitch, ‘Soundscape of the Street: Architectural Acoustics at Ostia’, Senses of Empire: Multisensory Approaches to Roman Culture, Eleanor Betts, ed, 1st ed., Routledge, 2017, 54–70. 9 HARPS, ‘The Historical Archaeology Research Project on Staffa (HARPS) | Society of Antiquaries of Scotland’, 2017, http://www.socantscot. org/research-project/the-historical-archaeology-research-project-on-staffa/. 10 Ibid. http://www.diva-portal.org/smash/get/diva2 http://amodern.net/article/ear-to-the-wire/ http://amodern.net/article/ear-to-the-wire/ https://culturetechnologypolitics.files.wordpress.com/2015/11/jonathan-sterne-the-audible-past-intro.pdf https://culturetechnologypolitics.files.wordpress.com/2015/11/jonathan-sterne-the-audible-past-intro.pdf ftp://s00275-s00279.cisti.nrc.ca/outgoing/ISRA2013/CD_ISRA2013/Papers/P117.pdf ftp://s00275-s00279.cisti.nrc.ca/outgoing/ISRA2013/CD_ISRA2013/Papers/P117.pdf http://www.fraufraulein.com/anne/AnneGuthrie_2014_05_07_Dissertation_FinalSmall.pdf http://www.fraufraulein.com/anne/AnneGuthrie_2014_05_07_Dissertation_FinalSmall.pdf http://eprints.whiterose.ac.uk/85334/ http://eprints.whiterose.ac.uk/75125/ https://doi.org/10.1093/acprof http://www.socantscot.org/research-project/the-historical-archaeology-research-project-on-staffa/ http://www.socantscot.org/research-project/the-historical-archaeology-research-project-on-staffa/ S. K. Noble, Fingal’s Cave: The Integration of Real-Time Auralisation and 3D Models 3 Scotland, want to attract people to visit the site11. However, there are several barriers to experiencing the island such as physical access issues (including cost), weather, and the conservation needs of the reserve. Using HARPS datasets, this paper discusses the benefits and problems of integrating auralised audio and 3D models in heritage visualisations. This aligns directly with the HARPS project’s objective, which is not to attempt to somehow replicate the experience of the cave in VR but to create a parallel, but affective immersive experience of Fingal’s Cave for those that are unable to visit the site in person. 1 F i n g a l ’ s C a v e : C u l t u r a l a n d H i s t o r i c a l C o n t e x t Fingal’s cave is a sea cave on the Isle of Staffa, one of the Inner Hebrides off the west coast of Scotland. It is to the west of Mull, between Iona and Gometra, as shown in Figure 1. Figure 1. Staffa is located to the west of the Isle of Mull, Scotland. Courtesy Google Maps.12 The name, Staffa, comes from the Norse ‘stafr,’ meaning ‘staff’ and is so named because of its distinctive columnar basalt structure13. The prismatic basalt columns and quasi-hexagonal pattern were formed upon the rapid cooling and hardening of volcanic material, at which time an enormous amount of tension builds up. The most economical way to release this tension is thought to be through hexagonal cracks in the rock14. The island is about one mile long by half a mile across, but features at least twelve prominent sea caves, the most famous of which is Fingal’s cave, shown to the right of Figure 2.15 11 National Trust for Scotland, ‘Staffa - Visit’, 2017, https://www.nts.org.uk/Visit/Staffa/. 12 Google Maps, ‘Staffa - Google Maps’, 2017, https://www.google.co.uk/maps/place/Staffa/@56.2504164,-6.0093908,8z/ data=!4m5!3m4!1s0x488b9652da67fd3f:0x13ff4b51d9ae3d31!8m2!3d56.4355675!4d-6.341756. 13 Donald B. MacCulloch, Staffa, 4th ed., David & Charles, 1975). 14 Philip Ball, ‘Pattern Formation in Nature: Physical Constraints and Self-Organising Characteristics.’, Architectural Design 82, no. 2, 2012, 22–27, http://www.philipball.co.uk/images/stories/docs/pdf/ADMC_Ball_final2.pdf. 15 John Patterson MacLean, An Historical, Archaeological and Geological Examination of Fingal’s Cave in the Island of Staffa, Rewritten, ULAN Press, 1887. https://www.nts.org.uk/Visit/Staffa/ https://www.google.co.uk/maps/place/Staffa/ http://www.philipball.co.uk/images/stories/docs/pdf/ADMC_Ball_final2.pdf S. K. Noble, Fingal’s Cave: The Integration of Real-Time Auralisation and 3D Models 4 Figure 2. Fingal’s Cave on the Isle of Staffa. Image © the author, 2017. The route from the landing place to the cave mouth is treacherous with its sea spray soaked stone, as can be seen in Figure 3, but the effort is rewarded, not only through its striking visuals, but through its extraordinary acoustics. The many surfaces of the cave create a uniquely resonant space: when a sound wave is emitted into a space, it reflects off surfaces in the space and these reflections attenuate over a certain amount of time. This reduction is a result of being absorbed by the walls of the space and the speed at which it reduces depends on the material and size of the space. This attenuation is known as the reverberance of the space16. The hard basalt rock columns are reflective and angular creating a diffuse reverberant field meaning sound is reflected in multiple different directions. Depending on the weather this may be pleasing to the ear or unsettling; under certain conditions, the caves of the island make a loud ‘booming’ noise, which was said to terrify inhabitants of the Island. 16 Springer, Springer Handbook of Acoustics, Thomas D. Rossing, ed, 2nd ed., Springer-Verlag New York, 2014, https://doi.org/10.1007/978-1- 4939-0755-7; Laird, Murphy, and Chapman, ‘Spatialisation Accuracy of a Virtual Performance System’. https://doi.org/10.1007/978-1-4939-0755-7 https://doi.org/10.1007/978-1-4939-0755-7 S. K. Noble, Fingal’s Cave: The Integration of Real-Time Auralisation and 3D Models 5 Figure 3. The access causeway for Fingal’s Cave. Image © the author, 2017. Sound is particularly culturally important to the identity of Scotland, and there is a long tradition of music, poetry and oral history being inspired by the landscape. Fingal’s cave and its acoustics is a major contributor to this, with many historical references noting its evocative acoustic qualities. The influence of this natural geological feature is widespread and can be seen through the depth of artistic responses to Staffa and Fingal’s Cave throughout the last two and a half centuries. Its many layers of engagement and creative responses began with the botanist, Joseph Banks’ visit in 177217. Banks made the connection between Fingal’s Cave and James Macpherson’s translation of Ossian’s poem, “Fingal” from the 18th century18. Although its authenticity is widely doubted, the ‘translation’ nonetheless brought Staffa and Fingal’s Cave further into the public view19. Since then, it has inspired countless poets, including Wordsworth, Hogg, Scott and Keats; musicians, most famously Mendelssohn, and more recently Pink Floyd; 17 Ralph Crane and Lisa Fletcher, ‘Inspiration and Spectacle: The Case of Fingal’s Cave in Nineteenth-Century Art and Literature’, Interdisciplinary Studies in Literature and Environment 22, no. 4, 2015, 778–800, https://doi.org/10.1093/isle/isv052. 18 Jennifer Davis Michael, ‘Ocean Meets Ossian: Staffa as Romantic Symbol’, Romanticism 13, no. 1, 2007, 1–14, https://muse.jhu.edu/ article/214802; Hugh Blair, ‘The Poetical Works of Ossian by James Macpherson with a Critical Dissertation by Hugh Blair’, 2009, http://www. exclassics.com/. 19 Nigel Leask, ‘Fingalian Topographies: Ossian and the Highland Tour, 1760 - 1805’, Journal for Eighteenth-Century Studies, 39, no. 2, 2016. https://doi.org/10.1093/isle/isv052 https://muse.jhu.edu/article/214802 https://muse.jhu.edu/article/214802 http://www.exclassics.com/ http://www.exclassics.com/ S. K. Noble, Fingal’s Cave: The Integration of Real-Time Auralisation and 3D Models 6 artists, including Turner20; and literature by the likes of Verne, among many others. The cave also features in multiple Gaelic and Irish legends, most famously in the story of Fionn MacCoul (Fingal), who built a causeway of stepping stones from Northern Ireland to Scotland to confront his rival Benandonner. The Giants Causeway in Northern Ireland and Fingal’s Cave on Staffa are said to be the last remaining pillars of this causeway21. These romanticized interpretations marked a change in worldview of Scotland from a place to be feared to somewhere much revered and sparked a surge in ‘geotourism’22. In modern times, these romanticized views are largely forgotten: Fingal’s Cave is seen primarily as a geological feature. Gordon23 proposes that reconnecting with the ‘voices’ of the stones of Scotland’s natural geodiversity will: “link people today with their cultural roots and sense of place”24. Dynamic engagement with landscape and its past romanticism may open further opportunities to connect with the past, present and future of our natural heritage. 2 A u d i o a n d H e r i t a g e Archaeoacoustics, the study of acoustics in relation to archaeological and heritage sites, is critical if we are to fully understand the impact and importance of a heritage site. This is particularly true in the case of Fingal’s cave as a lot of the engagement with this site surrounds its acoustic as much as its visual properties, which is attested to by the numerous references to sound in the creative works inspired by it (see examples below). Archaeological research in the last couple of decades has taken a much stronger interest in the specific relationships between artificial archaeological structures; stone circles, standing stones, chambered tombs, and churches, and the role that acoustics may have played in their design and even on their psychological impact. For examples of this see works by Diaz-Andreu25, Mattern26, Murphy27 and Watson and Keating28. Bradley’s work on the archaeological significance of natural places is also relevant29, and Green30 and Murphy and Shelley31 have curated useful repositories of impulse responses recorded at heritage sites across Scotland and beyond. Despite this, there remains much less focus on the acoustics of natural places and the impact that this may have had on earlier people, although Till32, who studied the sound archaeology of the natural caves of Altamira, Spain, suggests acoustics play a strong role in their contextualisation and their construction as significant places in the past. Something which the HARPS project argues also occurs at Fingal’s Cave. 20 Yale Center For British Art, ‘Staffa, Fingal’s Cave’, 2017, http://collections.britishart.yale.edu/vufind/Record/1669251. 21 Visit Scotland, ‘Scottish Folklore - Ghosts, Myths and Legends | VisitScotland — Phantom Piper, Fingals Cave, Corryvreckan Whirlpool’, 2016, http://ebooks.visitscotland.com/ghosts-myths-legends/phantom-piper-corryvreckan/. 22 S Schama, Landscape and Memory, Fontana Press, 1996. 23 John E Gordon, ‘Engaging with Geodiversity: ‘Stone Voices’, Creativity and Cultural Landscapes in Scotland’, Scottish Geographical Journal 128, no. 3–4, 2012, 240–265. 24 Ibid., 1. 25 Margarita Díaz-Andreu, ‘Archaeoacoustics of Rock Art: Quantitative Approaches to the Acoustics and Soundscape of Rock Art’, CAA2015. Keep The Revolution Going: Proceedings of the 43rd Annual Conference on Computer Applications and Quantitative Methods in Archaeology, Stefano Campana et al., eds, Archaeopress, 2015, 1049–58, https://www.researchgate.net/profile/Julio_Del_Hoyo-Melendez/publication/304793402_ Colour_and_Space_in_Cultural_Heritage_in_6Ds_the_Interdisciplinary_Connections/links/577b916808ae355e74f153de.pdf - page=1065. 26 Shannon Mattern, ‘Sonic Archaeologies’, The Routledge Companion to Sound Studies, Michael Bull, ed, 1st ed., Routledge, 2018, 222–30, http://wordsinspace.net/shannon/wp-content/uploads/2018/11/mattern_sonicarchaeologies_routledge_uneditedproofs.pdf. 27 Damian Murphy et al., ‘Acoustic Heritage and Audio Creativity: The Creative Application of Sound in the Representation, Understanding and Experience of Past Environments’, Internet Archaeology, no. 44, March 30, 2017, https://doi.org/10.11141/ia.44.12. 28 Aaron Watson and David Keating, ‘Architecture and Sound: An Acoustic Analysis of Megalithic Monuments in Prehistoric Britain’, Antiquity 73, no. 280, 1999, 325–336, https://doi.org/10.1017/S0003598X00088281. 29 Richard Bradley, An Archaeology of Natural Places, Routledge, 2000. 30 ‘Archaeoacoustics Scotland - Home’, 2017, http://www.archaeoacousticsscotland.com/. 31 Damian Murphy and Simon Shelley, ‘OpenAIR | The Open Acoustic Impulse Response Library’, 2017, http://www.openairlib.net/. 32 Rupert Till, ‘Sound Archaeology: Terminology, Palaeolithic Cave Art and the Soundscape’, World Archaeology 46, no. 3, 2014, 292–304, https:// doi.org/10.1080/00438243.2014.909106. http://collections.britishart.yale.edu/vufind/Record/1669251 http://ebooks.visitscotland.com/ghosts-myths-legends/phantom-piper-corryvreckan/ https://www.researchgate.net/profile/Julio_Del_Hoyo-Melendez/publication/304793402_Colour_and_Space_in_Cultural_Heritage_in_6Ds_the_Interdisciplinary_Connections/links/577b916808ae355e74f153de.pdf https://www.researchgate.net/profile/Julio_Del_Hoyo-Melendez/publication/304793402_Colour_and_Space_in_Cultural_Heritage_in_6Ds_the_Interdisciplinary_Connections/links/577b916808ae355e74f153de.pdf http://wordsinspace.net/shannon/wp-content/uploads/2018/11/mattern_sonicarchaeologies_routledge_uneditedproofs.pdf https://doi.org/10.11141/ia.44.12 https://doi.org/10.1017/S0003598X00088281 http://www.archaeoacousticsscotland.com/ http://www.openairlib.net/ https://doi.org/10.1080/00438243.2014.909106 https://doi.org/10.1080/00438243.2014.909106 S. K. Noble, Fingal’s Cave: The Integration of Real-Time Auralisation and 3D Models 7 Many historic cathedrals are noted for their acoustic qualities and this is important as Fingal’s Cave has often been referred to as a ‘cathedral’ and a ‘temple’. For example, the poem ‘Staffa, the Island / Fingal’s Cave’ by John Keats likens it to a church organ and cathedral: “…This was architectured thus By the great Oceanus! — Here his mighty waters play Hollow organs all the day; Here, by turns, his dolphins all, Finny palmers, great and small, Come to pay devotion due, — Each a mouth of pearls must strew! Many a mortal of these days Dares to pass our sacred ways; Dares to touch, audaciously, This cathedral of the sea! ...”33 Pentcheva’s34 study into the aural architecture of Hagia Sofia draws connections between the effect of both poetry, and the polymorphic materials used in the construction of the church have on the human brain. It may be supposed that the similar effects of the natural structure of Fingal’s cave sparked poetic imagination. Poetry moves beyond dryly descriptive language to conjure a dynamic and emotive vision of a subject, as in the following verse by James Hogg: “Dark Staffa! in thy grotto wild, How my wrapt soul is tought to feel! Oh! well becomes it Nature’s child Now in her stateliest shrine to kneel! Thou art no fiends’ nor giants’ home - Thy piles of dark and dismal grain, Bespeak thee, dread and sacred dome, Great temple of the Western Main! ...”35 Pentcheva’s research suggests that such ekphratic descriptions of a space can integrate “a direct response to sensual materiality of the space and uncovers in it a metaphysical dimension”36. This is reflected in its descriptive language; for example, sound is often described in a physical way as ‘piercing’ or ‘thumping’ as it has a direct effect on the body. This notion allows understanding of the importance of a space to past visitors. They were so taken by a space, visually, aurally and corporally, that they were compelled to write poetry or in the case of Mendelssohn and Fingal’s Cave, compose an overture. Though the name, Fingal’s Cave, comes from the mythical Celtic hero; the Gaelic name is ‘An Uamh Bhinn’37 which means the melodious cave. One early visitor, Pancoucke, noted the effects on his wife’s voice when singing inside the cave: “… Her voice vibrated throughout the columns, becoming fuller and more powerful, the tones seemed to take on a new life, the held notes became stronger; the religious majesty of the location infused these harmonies with something beautiful and grandiose. Everyone applauded and it seemed that even the gods of this enchanted place echoed this applause in the air.”38. 33 Bartleby, ‘Fingal’s Cave. John Keats (1795-1821). Staffa, the Island. Henry Wadsworth Longfellow, Ed. 1876-79. Poems of Places: An Anthology in 31 Volumes. Scotland: Vols. VI-VIII’, 2017, http://www.bartleby.com/270/3/341.html. 34 Bisserra V. Pentcheva, ‘Hagia Sophia and Multisensory Aesthetics’, Gesta International Centre of Medieval Art, 50/2, 2011, 93–111, http:// iconsofsound.stanford.edu/Pentcheva.Gesta 2011.Hagia Sophia.pdf. 35 Donald B. MacCulloch quoting Hogg (entry in visitor book in Ulva), Staffa, 4th ed., David & Charles, 1975, 148. 36 Pentcheva, ‘Hagia Sophia and Multisensory Aesthetics’, 96. 37 MacCulloch, Staffa, 110. 38 Translated extracts from Ibid. http://www.bartleby.com/270/3/341.html http://iconsofsound.stanford.edu/Pentcheva.Gesta 2011.Hagia Sophia.pdf http://iconsofsound.stanford.edu/Pentcheva.Gesta 2011.Hagia Sophia.pdf S. K. Noble, Fingal’s Cave: The Integration of Real-Time Auralisation and 3D Models 8 Pentcheva39 notes the ‘performative’ characteristics of the space when the acoustics work together with the visuals, an insight this research draws upon. One visitor to Fingal’s Cave, Queen Victoria, noticed that “the rocks under water were all colours, pink, blue, green, which had the most wonderful effect”40. The sea-soaked basalt pillars and shifting seas bounce light creating extraordinary effects. This coupled with the colours of the algae on the rocks and misty sea spray work to create an opalescent effect, shown in Figure 4. Figure 4. The effects of light in Fingal’s Cave. Image © the author, 2017. The presence of light on the natural rock formations can be compared to the importance of light in churches. Pentcheva41, observes that the built materials of Hagia Sofia “become animate in the shifting natural light, and these transient manifestations trigger the spectator’s memory and imagination to conjure up images”42. Perhaps these visual qualities, together with the dominance of wave sounds within the cave, are why Fingal’s Cave leaves such an impression on its visitors. 3 A u d i o a n d V i s u a l i s a t i o n Real-time auralisation is used frequently in stage acoustics research which explores the relation that spatial distribution of sound has on a musician’s experience in varying performance spaces such as concert halls. It is 39 Pentcheva, ‘Hagia Sophia and Multisensory Aesthetics’. 40 Eve Eckstein quoting Queen Victoria (Diary Entry 1847), Historic Visitors to Mull, Iona and Staffa, Excalibur Press of London, 1992, 156. 41 Pentcheva, ‘Hagia Sophia and Multisensory Aesthetics’. 42 Ibid., 93. S. K. Noble, Fingal’s Cave: The Integration of Real-Time Auralisation and 3D Models 9 pertinent to clarify some terms here, most importantly an impulse response, which is thought of as the ‘acoustic fingerprint’ of a space. They can be created by deconvolving a sine sweep recorded in that space to extract a known signal to discern the effects the space had on that signal43. An auralisation is the resultant audio after an impulse response has been applied, representative of how a voice or other audio would sound within the physical space in which the impulse response was recorded. Laird et al.44 developed a Virtual Performance Studio in which a musician can “practice in a virtual version of a real performance space in order to acclimatise to the acoustic feedback received on stage before physically performing there”45. Brereton implemented a Virtual Acoustic Environment, in which participants can physically move around the virtual space and when they speak they hear their acoustics change depending on their position within the space46. The acoustically rendered environments uses real-time convolution with ambisonic impulse responses. Ambisonic audio gives an impression of sound within the full sphere: an array of speakers (minimum of 4 for first-order ambisonics) are placed around a sweet-spot from which a spatial reproduction of a sound field can be experienced47. Real-time auralisation, derived from impulse responses captured within the cave, allows participants to hear themselves as if immersed in the reverb of the cave. This is achieved by first capturing an impulse response, recorded inside Fingal’s Cave, which is a short, sharp, clean sound representative of the acoustic ‘fingerprint’ of the space. It is then applied to, or convolved with, live speech to simulate the reverberation of the space where the impulse response was recorded. Where this project takes things one step further is to include a 3D recreation of Fingal’s Cave, tracked in 3D space and rendered in real-time depending on the position of the viewer’s head. In this way the 3D model is married with the live auralisations; what the user hears matches what they see. The visuals of the VR environment and the 3D sound work together in unison, creating a highly immersive, multisensory experience. As discussed, there is a strong case for including sound in digital reconstructions of heritage sites, but further to this, VR can be considered a platform for including kinaesthesia. As Slaney, Foka and Bocksberger make the case for in their forthcoming article, kinaesthesia can help with fully understanding physicalities of the space and “contribute to formulating conceptions of the ancient past”48. With virtual reality, the user can explore the virtual environment and interact with their surroundings and, as Forte notes, “what really changes our capacities of digital/virtual perception is the experience, a cultural presence in a situated environment”49. The use of VR is particularly appropriate in the case of Fingal’s cave as much engagement with this site surrounds its experiential as much as its visual properties. However, Jeffrey argues that a significant barrier to interaction with digital objects and spaces in VR is their ‘immateriality’50, and that digital ‘recreations’ struggle to carry the same signs of use that a physical artefact would51. The digital object is impervious to its visitors or users’ mark, therefore making it difficult to imagine it has a past steeped in use and reuse. 43 Francis Stevens, ‘Creswell Crags - Recording Report’, York, 2016, https://drive.google.com/file/d/0B7uX0JAxlCzKUWx3N3RLa2ZaYm8/view. 44 Laird et al., ‘Development of a Virtual Performance Studio with Application of Virtual Acoustic Recording Methods’. 45 Ibid. 46 Brereton, Murphy, and Howard, ‘The Virtual Singing Studio: A Loudspeaker-Based Room Acoustics Simulation for Real-Time Musical Performance’. 47 Springer, Springer Handbook of Acoustics, 814. 48 Helen Slaney, Anna Foka, and Sophie Bocksberger, ‘‘Ghosts in the Machine: Experiencing Animation,’’ Forthcoming 2019, 1. 49 Maurizio Forte, ‘Virtual Reality, Cyberarchaeology, Teleimmersive Archaeology’, Fabio Remondino and Stefano Campana, eds, 3D Recording and Modelling in Archaeology and Cultural Heritage: Theory and Best Practices, 2598, 2014, 113, https://vle.gsa.ac.uk/bbcswebdav/pid-230927- dt-content-rid-634603_1/courses/ACPG_IHV_1617/Virtual_Reality_Cyberaerchaeology_Teleim.pdf. 50 Stuart Jeffrey, ‘Digital Heritage Objects, Authorship, Ownership and Engagement’, Authenticity and Cultural Heritage in the Age of 3D Digital Reproductions, Paola Di Giuseppantonio DI Franco, Fabrizio Galeazzi, and Valentina Vassallo, eds, McDonald Institute for Archaeological Research, 2018, 51, https://www.repository.cam.ac.uk/bitstream/handle/1810/279673/Authenticity_chapter4.pdf?sequence=1. 51 Stuart Jeffrey, ‘Challenging Heritage Visualisation: Beauty, Aura and Democratisation’, Open Archaeology, 1, 2015, 144–52, https://doi. org/10.1515/opar-2015-0008. https://drive.google.com/file/d/0B7uX0JAxlCzKUWx3N3RLa2ZaYm8/view https://vle.gsa.ac.uk/bbcswebdav/pid-230927-dt-content-rid-634603_1/courses/ACPG_IHV_1617/Virtual_Reality_Cyberaerchaeology_Teleim.pdf https://vle.gsa.ac.uk/bbcswebdav/pid-230927-dt-content-rid-634603_1/courses/ACPG_IHV_1617/Virtual_Reality_Cyberaerchaeology_Teleim.pdf https://www.repository.cam.ac.uk/bitstream/handle/1810/279673/Authenticity_chapter4.pdf?sequence=1 https://doi.org/10.1515/opar-2015-0008 https://doi.org/10.1515/opar-2015-0008 S. K. Noble, Fingal’s Cave: The Integration of Real-Time Auralisation and 3D Models 10 4 T e c h n i c a l I m p l e m e n t a t i o n 4 . 1 C a p t u r i n g t h e S o u n d Staffa and Fingal’s cave as heritage sites present a multitude of potential problems when it comes to data capture and processing. The methodological approach becomes an assault course in dealing with issues with field recording and ways to overcome them. For auralisation, I used an ambisonic impulse response that was recorded in 2014 inside Fingal’s Cave as part of the HARPS project. Since the cave has an extremely high noise floor due to waves and wind, the acoustic characteristics were captured using a swept sine wave with the receiver positioned mid-way through the cave. An exponential sine sweep is a computer-generated signal composed of sine waves increasing in frequency exponentially from 20Hz to 20kHz which are emitted into a space, reverberate around the space and are captured by a microphone and storage medium52. Laird et al.53 have shown this method of auralisation can create a ‘reasonably accurate simulation’ of the reverberation of a space. The high noise level experienced when recording the impulse responses is masked by including the ambient noise of the cave in the soundscape. For the purposes of this project, allowances must be made as to the quality of the auralisation, since, as this is designed for use in tourism, the auralisation will be directly affected by the acoustic response of the space in which it is exhibited. The sine sweep was deconvolved using MatLab54 via a bespoke procedure written by Iain Laird. This procedure extracts the original emitted signal in order to discern the effects the space had on that signal55. Ronan Breslin, lecturer at the School of Simulation and Visualisation at the Glasgow School of Art, reduced audio distortions on the impulse response and applied it to the voice recordings in Reaper56 to simulate the cave’s reverberation. Audioclip 1 and Audioclip 2 show the effect the auralisation has on the recorded speech: Audioclip 1. An example voice recording before the impulse response is applied. Courtesy the author, 2017. 52 Angela Farina, ‘Simultaneous Measurement of Impulse Response and Distortion with a Swept-Sine Technique’, Proc. AES 108th Conv, Paris, France, no. I, 2000, 1–15, https://doi.org/10.1109/ASPAA.1999.810884; Angelo Farina, ‘Advancements in Impulse Response Measurements by Sine Sweeps’, The 122nd Convention of the Audio Engineering Society, Audio Engineering Society, 2007, http://pcfarina.eng.unipr.it/Public/ Papers/226-AES122.pdf. 53 Laird, Murphy, and Chapman, ‘Spatialisation Accuracy of a Virtual Performance System’. 54 MathWorks, ‘MATLAB - MathWorks’, 2017, https://uk.mathworks.com/products/matlab.html?s_tid=hp_products_matlab. 55 Stevens, ‘Creswell Crags - Recording Report’. 56 Reaper, ‘REAPER | Digital Audio Workstation: Audio Production Without Limits’, 2017, https://www.reaper.fm/. https://soundcloud.com/shonaknoble/before https://doi.org/10.1109/ASPAA.1999.810884 http://pcfarina.eng.unipr.it/Public/Papers/226-AES122.pdf http://pcfarina.eng.unipr.it/Public/Papers/226-AES122.pdf https://uk.mathworks.com/products/matlab.html?s_tid=hp_products_matlab https://www.reaper.fm/ S. K. Noble, Fingal’s Cave: The Integration of Real-Time Auralisation and 3D Models 11 Audioclip 2. An example voice recording after the impulse response is applied. Courtesy the author, 2017. The latest version of Unity, 2017.1, supports importation of ambisonic audio57. The pre-auralised audio files are AmbiX (WAV b-format) and require 4 audio channels which Unity decodes. The Oculus Spatialiser plugin supports ambisonic audio in virtual reality58. It uses an array of eight virtual speakers surrounding the viewer to decode and spatialise audio, which means it gives an impression of sound within a 3D environment59. It is designed for use with ‘broadband’ audio such as wind and wave sounds which feature highly in this soundscape. When 4-channel AmbiX format audio files are placed in the scene, this plugin makes use of the highly receptive head tracking mentioned above to alter the ambisonic orientation of the audio source with the movement and rotation of the HMD60. The Oculus Spatialiser, together with the HMD technology ensures the 3D modelled scene works in unison with the audio and means that what the viewer sees matches what they hear. They can physically move around the space and turn their head and both the scene and the audio update accordingly. Video 1 demonstrates how the speaker’s voice changes with the orientation of the user’s head as they move around the space: Video 1. Screen-grab video of Fingal’s Cave: an Audiovisual Experience in use, courtesy the author, 2017. 57 Unity, Unity - Game Engine’, 2017, https://unity3d.com/. 58 Oculus, ‘Playing Ambisonic Audio in Unity 2017.1 (Beta)’, 2017, https://developer.oculus.com/documentation/audiosdk/latest/concepts/ ospnative-unity-ambisonic/. 59 Madeline Carson et al., “Surround Sound Impulse Response Measurement with the Exponential Sine Sweep; Application in Convolution Reverb” (Univeristy of Victoria, 2009), http://arqen.com/wp-content/docs/Surround-Sound-Impulse-Response.pdf. 60 Oculus, “Playing Ambisonic Audio in Unity 2017.1 (Beta).” https://soundcloud.com/shonaknoble/voice-recording-after-auralisation https://vimeo.com/255942898 https://unity3d.com/ https://developer.oculus.com/documentation/audiosdk/latest/concepts/ospnative-unity-ambisonic/ https://developer.oculus.com/documentation/audiosdk/latest/concepts/ospnative-unity-ambisonic/ http://arqen.com/wp-content/docs/Surround-Sound-Impulse-Response.pdf S. K. Noble, Fingal’s Cave: The Integration of Real-Time Auralisation and 3D Models 12 4 . 2 N a r r a t i v e S o u n d s c a p e This research will afford engagement with and understanding of multiple layers of the cave’s history, encompassing multiple viewpoints, to enable people to write their own experience with the site as much as possible. Voice recordings are layered with ambient wave noise and Mendelssohn’s overture61 to tell the story of the cave’s romanticised history through an abstract narrative. The ambient wave and wind recordings were captured in Fingal’s Cave as part of the HARPS project and the script features folklore, poetry and descriptions of the cave. It introduces the cave in multiple languages; French, Gaelic and English. The voice audio was recorded in a studio environment to reduce room presence on the recordings as much as possible. Folkloric elements were included in the narrative and these tell of a rivalry between the Giants Fionn MacCoul and Benandonner and how Staffa and the Giants Causeway were made and unmade via their interactions62. For the cultural narrative, poetry was chosen for its imagery of experiences within the cave. Much of it describes the acoustics and comes from a time when recording the sounds in reality was simply not possible. John Keats seems in awe and has a thoroughly spiritual experience in his poem Staffa, the Island/Fingal’s Cave63. He cites ‘organs’, a ‘spirit’ and compares the cave to a cathedral. James Hogg also compares the cave to a spiritual space, referring to it as a ‘temple’64. However, Hogg’s experience seems entirely different; his imagery conveys a much darker experience and illustrates the sheer power of the space. The wave audio had to be layered carefully as it tends to sound like undifferentiated white noise. Inspired by the work of Tim Neilson in the animated film Moana65, the idea was to give character to the wave noise and create the feeling that although the ocean is ever present within the cave, it also ‘speaks’ to the cave’s visitors, inspiring poetry and leaving a lasting impression. To this end, the wave audio grows stronger at meaningful points in the narrative. This is coupled with evocative poetry by Sir Walter Scott66 which reinforces the strength and character of the presence of the ocean within Fingal’s cave. Audioclip 3. The full narrative soundscape courtesy the author, 2017. Featuring quotes by C.L.F. Pancoucke, Queen Victoria, D.B. MacCulloch, M. Faujas and Sir Robert Peel. Folklore retold from Visit Scotland and The Gaelic Otherworld (J.G. Campbell). Music: The Hebrides (Fingal’s Cave), Op. 26, Felix Mendelssohn, from MusOpen. Poetry: The Lord Of The Isles: Canto Iv, Sir Walter Scott 1857; Staffa, the Island/Fingal’s Cave, John Keats 1876-79; and Untitled (Entry in the visitor’s book at Ulva), James Hogg 1975. Voice Over Artists: Norman Mackay, Robbie Noble and Shona Noble. 61 MusOpen, ‘The Hebrides (Fingal’s Cave), Op. 26 | Free Music’, 2017, https://musopen.org/music/1570/felix-mendelssohn/the-hebrides-fingals-cave- op-26/; Creative Commons, ‘Public Domain Mark - Creative Commons’, 2017, https://creativecommons.org/share-your-work/public-domain/pdm. 62 Visit Scotland, ‘Scottish Folklore - Ghosts, Myths and Legends | VisitScotland — Phantom Piper, Fingals Cave, Corryvreckan Whirlpool’. 63 Bartleby, ‘Fingal’s Cave. John Keats (1795-1821). Staffa, the Island. Henry Wadsworth Longfellow, Ed. 1876-79. Poems of Places: An Anthology in 31 Volumes. Scotland: Vols. VI-VIII’, 2017, originally published by James R. Osgood & Co., Boston, 1876–79, http://www.bartleby.com/270/3/341.html. 64 Donald B. MacCulloch quoting Hogg (entry in visitor book in Ulva), Staffa, 4th ed., David & Charles, 1975, 148. 65 Tim Neilsen, ‘How Tim Nielsen and Team Made ‘Moana / Vaiana’ Sound So Good | A Sound Effect’, Interview by Jennifer Walden for A Sound Effect, 2016, https://www.asoundeffect.com/moana-sound/. 66 PoemHunter, ‘The Lord Of The Isles: Canto Iv. Poem by Sir Walter Scott - Poem Hunter’, 2017, originally published by Adam and Charles Black, Edinburgh, 1857, https://www.poemhunter.com/poem/the-lord-of-the-isles-canto-iv/. https://soundcloud.com/shonaknoble/narrative-soundscape https://musopen.org/music/1570/felix-mendelssohn/the-hebrides-fingals-cave-op-26/ https://musopen.org/music/1570/felix-mendelssohn/the-hebrides-fingals-cave-op-26/ https://creativecommons.org/share-your-work/public-domain/pdm http://www.bartleby.com/270/3/341.html https://www.asoundeffect.com/moana-sound/ https://www.poemhunter.com/poem/the-lord-of-the-isles-canto-iv/ S. K. Noble, Fingal’s Cave: The Integration of Real-Time Auralisation and 3D Models 13 The final quote is from Sir Robert Peel: “I have stood on the shores of Staffa; I have seen the ‘temple not made with hands’”67. It is read in multiple voices, which are layered and desynchronized. This final notion is intended to draw to mind the ghosts of the past visitors to Fingal’s Cave and to inspire people to explore the cave for themselves. 4 . 3 L i v e A u r a l i s a t i o n To be able to speak within the VR experience and hear your own voice reverberated in the same way as it would be in the cave, live auralisation is required. Since it is not possible to perform real-time auralisation within Unity as it is not supported, it is implemented via external software and delivered as an adjunct to the core Unity experience. Drawing again from stage acoustics research, the real-time auralisation is implemented using a headset microphone for participants to speak into. This live audio is then routed from the microphone via a soundcard to the Reaper Digital Audio Workstation68 where a single ReaVerb69 (a powerful reverberation plugin) is applied each of the w, x, y, z audio channels (the four channels work together to give an impression of sound within the full sphere). Each ReaVerb instance is loaded with an impulse response corresponding to w, x, y, z of the impulse response that was recorded in the cave. The w, x y, z channels are then routed to an ambisonic binaural decoder for headphone playback70. The decoder used was the Ambisonic Toolkit71. It is then fed to headphones via the soundcard output. This is the same process as with the pre-auralised audio, except that the convolution is performed in real-time as the participants are speaking. Consequently, some processing latency is present which causes the speech to sound delayed. Latency was reduced by reducing the recording buffer size, adjusting the sound resolution of the reverb (called the Fast Fourier Transform) and changing the Head Related Transfer Function (HRTF) in the decoder. This is an iterative process and is achieved by experimenting with the settings to keep unwanted feedback at bay while reducing processing latency as much as possible. This ensures responsiveness within the space72. In order to make applying the acoustic characteristics of the cave to live speech suitable for mobile dissemination, the reverberation could have been approximated using the built in Unity Audio Source component73. This takes input from the microphone and applies reverberation which is mocked up by ear, listening and tweaking until the desired effect is achieved. Although this is a viable option, for this project external software performs the live auralisation to make use of the impulse responses recorded inside the cave, as these gave an authentic representation of the acoustic response, rather than a generic, or synthesised one. The cave has a unique acoustic response and since the impulse response was recorded in-situ, it is a more convincing and authentic representation, which is important if the visualisation is to give a considerate understanding of an experience within Fingal’s Cave74. 67 Donald B. MacCulloch quoting Sir Robert Peel (speech delivered in Glasgow 1837), Staffa, 4th ed., David & Charles, 1975, 162. 68 Reaper, ‘REAPER | Digital Audio Workstation: Audio Production Without Limits’. 69 Geoffrey Francis, ‘Supplement to REAPER User Guide The REAPER Cockos Effects Summary Guide’, 2016, 32, https://www.reaper.fm/ guides/ReaEffectsGuide.pdf. 70 Ronan Breslin, ‘Pers. Comm.’, July 24, 2017. 71 Ambisonic Toolkit, ‘ATK for Reaper’, 2017, http://www.ambisonictoolkit.net/documentation/reaper/. 72 Iain Laird, ‘Pers. Comm.’, July 19, 2017. 73 Unity, ‘Unity - Manual: Audio Source’, 2017, https://docs.unity3d.com/Manual/class-AudioSource.html. 74 Laird, ‘Pers. Comm’. https://www.reaper.fm/guides/ReaEffectsGuide.pdf https://www.reaper.fm/guides/ReaEffectsGuide.pdf http://www.ambisonictoolkit.net/documentation/reaper/ https://docs.unity3d.com/Manual/class-AudioSource.html S. K. Noble, Fingal’s Cave: The Integration of Real-Time Auralisation and 3D Models 14 4 . 4 A u d i o v i s u a l I m p l e m e n t a t i o n With the individual elements of the visualisation processed and ready to be brought together, the final stage of implementation of the project was to ensure that the 3D space and auralisation worked together in a coherent and meaningful way. A Unity VR environment is where the 3D model and the pre-auralised narrative soundscape were brought together. Generated as part of the HARPS project, SimVis holds laser survey data of Fingal’s cave which was used for the 3D digital model. For use in Unity, the model was simplified while keeping the optimal level of detail. Figure 5. Point cloud data of Fingal’s Cave, captured by HARPS.75 Screenshot from Cyclone.76 Figure 6. The original mesh of Fingal’s Cave. Screenshot from 3DReshaper.77 75 HARPS, ‘The Historical Archaeology Research Project on Staffa (HARPS) | Society of Antiquaries of Scotland’. 76 Leica Geosystems, ‘Leica Cyclone - 3D Point Cloud Processing Software - Leica Geosystems - HDS’, 2017, http://hds.leica-geosystems.com/ en/leica-cyclone_6515.htm. 77 3DReshaper, ‘3DReshaper | 3DReshaper’, 2017, http://www.3dreshaper.com/en/. http://hds.leica-geosystems.com/en/leica-cyclone_6515.htm http://hds.leica-geosystems.com/en/leica-cyclone_6515.htm http://dreshaper.com/en/ S. K. Noble, Fingal’s Cave: The Integration of Real-Time Auralisation and 3D Models 15 Figure 7. The processed mesh of Fingal’s Cave. Screenshot from 3DS Max.78 It was not possible to use the original texture of the cave captured by the laser scanner due to gaps in the original dataset, so this was created within Unity. The final stage was to scale the model so that it shows in the visualisation at 1:1 scale. To contextualise Fingal’s cave within its positioning on the Isle of Staffa, a 3D digital model of Staffa was created from photogrammetry data also captured as part of the HARPS project and processed using Agisoft Photoscan79. Figure 8. Digital Mesh of the Isle of Staffa from photogrammetry data. Screenshot from Photoscan.80 Figure 9. Textured model of the Isle of Staffa. Screenshot from Photoscan.81 78 Autodesk, ‘3ds Max | 3D Modelling, Animation & Rendering Software | Autodesk’, 2017, https://www.autodesk.co.uk/products/3ds-max/ overview. 79 Agisoft, ‘PhotoScan’, 2017, http://www.agisoft.com/. 80 Ibid. 81 Ibid. https://www.autodesk.co.uk/products/3ds-max/overview https://www.autodesk.co.uk/products/3ds-max/overview http://www.agisoft.com/ S. K. Noble, Fingal’s Cave: The Integration of Real-Time Auralisation and 3D Models 16 The visualisation is experienced in virtual reality (VR) through a head mounted display and head tracking system (an HTC Vive82). This offers a highly immersive experience and gives a good sensation of being surrounded by the virtual environment. The user is tracked in position and orientation, using highly receptive head tracking, ensuring head movement and rendering of the virtual environment are synchronised83. It uses stereoscopic display which means the viewer can perceive depth which in turn informs interactivity. On entering the unity application, the viewer is dropped into a scene where they can see Staffa and Fingal’s Cave and ambient audio surrounds them. The narrative audio plays (this is a linear, 3-minute-long track), and as participants move around and explore their surroundings, the reverberation of the audio changes with their position and the direction they are facing (movements are tracked by the HMD and reverberation is controlled by the Oculus Spatialiser). Once the narrative soundscape finishes, the ambient soundscape continues to play (for a further 7 minutes before looping) and participants are invited to interact with the acoustics and speak to hear their voice auralised. Through use of interactive virtual reality, the exploration afforded in the virtual site surpasses the breadth of exploration available at the actual site. In this digital recreation, viewers can move all around the cave, from front to back and left to right, they are only constrained to the plane of the ocean, compared with being constrained to a narrow causeway on the right side of the cave that stops half-way inside of the real cave. This is designed to increase the ‘otherworldly’ nature of the experience and allow participants to feel like they are ‘walking on water’. Although the nature of the site made recording of both the laser scan data and ambisonic impulse responses difficult, there were workarounds in post processing available to prepare both for public consumption. The reasons for making recording tough, namely limited access and difficult conditions, directly contribute to the reasoning behind the creation of this visualisation. They further add to the allure of the cave, as not everyone will have the opportunity to visit. 5 I m p a c t One aim of this project was to demonstrate that 3D models and interactive real-time auralisation can come together in a meaningful way. This research has proved their integration possible, but what is the impact of the visualisation in terms of engaging with communities of interest and in the field of audio in heritage visualisation? The main interpretive aspect of the experience is the narrative soundscape, designed to give a sense of the different cultures and experiences of the many visitors to Fingal’s Cave throughout history. It is intended to afford engagement with and understanding of multiple layers of histories and meanings; a reflection of the intangible side to its heritage. It is purposefully abstract to allow for personalisation; everyone should take something different from it depending on their own knowledge and experiences. As Ioannidis et al. suggest: “if digital storytelling used the right combination of interest to focus attention, empathy to make visitors feel they are part of the story world and imagination to let them fantasise alternative realities, then it could constitute a successful entertainment experience”84. By its very definition, folklore tells the stories of a community passed by word of mouth through generations. Its inclusion in the immersive experience means that the stories, traditions and beliefs of a community are woven into 82 HTC, ‘VIVE | Discover Virtual Reality Beyond Imagination’, 2017, https://www.vive.com/uk/. 83 Steven LaValle, Virtual Reality, Cambridge University Press, 2017, http://vr.cs.uiuc.edu/. 84 Yannis Ioannidis et al., ‘One Object Many Stories: Introducing ICT in Museums and Collections through Digital Storytelling’, Digital Heritage International Congress (DigitalHeritage) \ IEEE 1, 2013, 422, http://www.madgik.di.uoa.gr/sites/default/files/digitalheritage2013_401_ ioannidisetal_1.pdf. https://www.vive.com/uk/ http://vr.cs.uiuc.edu/ http://www.madgik.di.uoa.gr/sites/default/files/digitalheritage2013_401_ioannidisetal_1.pdf http://www.madgik.di.uoa.gr/sites/default/files/digitalheritage2013_401_ioannidisetal_1.pdf S. K. Noble, Fingal’s Cave: The Integration of Real-Time Auralisation and 3D Models 17 the narrative. The poetry was chosen for its differing imagery of experiences within the cave. The use of poetry, storytelling and voice give it cultural presence, as do the sea sound samples running underneath. Storytelling through an abstract narrative soundscape brings the human experience to an otherwise inanimate visualisation. It is designed to make people want to visit the site, but equally, provide an authentic experience for those who cannot, hence enhancing virtual engagement off site and supporting communities of interest in their continuing interaction with the heritage site. In terms of accessibility, the experience works as an interactive installation and is suitable for dissemination as an exhibit in art galleries, museums and heritage centres. The institution would need a PC with Reaper installed, a Vive and a space large enough to house the Vive. An optional extra would be a screen that shows what the person wearing the HMD is seeing, making for a more inclusive experience. This 3D tool has the potential to maximise visitor usage, for all internal, external, local and international audiences (it could be packaged for download from the internet). The core concept of the visualisation has been designed around contextualising and increasing understanding of the heritage site, both on and off-site. It goes beyond a static 3D model of the site, bringing visuals together with exploration and interactivity. 6 F u t u r e D i r e c t i o n s This research has determined a prototype for a visualisation that acts as a profoundly immersive representation of a heritage site; highlighting the importance of including audio, as well as visual characteristics, in heritage visualisations if they are to give a more authentic and deeply engaging understanding of the site. The next stage in the development of this project would be to engage in a full-scale evaluation. A possible evaluation methodology might follow the work of Pujol-Tost and rate the visualisation for the following factors, outlined as critical for achieving cultural presence: “1) Realistic behaviour and scientific/cultural reliability of the virtual environment; 2) Distinctive cultural elements (place, material culture, everyday life, people’s aspect); 3) Presence of realistic, autonomous human characters; and 4) Communicational aspects of technology (visual realism, affordances for interaction in environment; intuitiveness of interaction in devices)”85 The {LEAP] project86 postulates that Cultural Presence, the sensation of ‘being there’, is determined by narrative and can boost social significance and understanding87. Participants should be recruited in 2 groups – those who have visited the cave and could provide feedback on the application’s success at recreating this experience; and those who have not visited the cave and could provide feedback on the application’s success at inspiring them to visit. Both groups should be asked whether they draw any significance from the experience in terms of engaging with the cave’s heritage. The ‘voices’ of participants could be harnessed as a dataset that captures responses to the cave of people who are unable to visit it in person, allowing them to become part of the ongoing cultural narrative surrounding it. 85 Laia Pujol-Tost, ‘Being There and Then. Cultural Presence for Archaeological Virtual Environments’, EuroVR International Conference 2016, Unpublished, 2016, 2, https://www.upf.edu/documents/3622658/3623728/EuroVR_2016Poster_CameraReady.pdf/24bdae2b-2c0c-0796-5f33- 66744e522a62. 86 Pujol-Tost, ‘Being There and Then. Cultural Presence for Archaeological Virtual Environments’. 87 Forte, ‘Virtual Reality, Cyberarchaeology, Teleimmersive Archaeology’. https://www.upf.edu/documents/3622658/3623728/EuroVR_2016Poster_CameraReady.pdf/24bdae2b-2c0c-0796-5f33-66744e522a62 https://www.upf.edu/documents/3622658/3623728/EuroVR_2016Poster_CameraReady.pdf/24bdae2b-2c0c-0796-5f33-66744e522a62 S. K. Noble, Fingal’s Cave: The Integration of Real-Time Auralisation and 3D Models 18 Through the technical implementation of this project, some areas of further development have arisen. Although this visualisation serves as a demonstration of the core concept, it should be considered a prototype for which the following developments could be made in the future. Critically, the prototype requires the implementation of more impulse responses. Since an impulse response measures the acoustic response of a space from the exact position from which it was measured, it is specific to that point. Indeed, as Sterne affirms, “a single impulse response no more captures the motion of sound in a room over time than a photograph of a person walking captures his or her route”88. So, to truly represent the cave’s acoustics, many more impulse responses measured all around the cave would be needed. This requires further field work on Staffa to record impulse responses from multiple positions to survey the full acoustic footprint of the cave, something the HARPS project intend to do in future field seasons. Though the project makes use of free and open access software wherever possible, more work is needed to design integrated digital infrastructures for experience-based sensory representation that ease financial pressures on both the initial purchase and maintenance89. This research has highlighted a demand for live auralisation to be supported in game development engines such as Unity. Ideally, multiple impulse responses could be loaded into the game engine and the convolution linked to the position of the participant within the virtual scene. There would need to be some experimentation with regards to applying a cross-fade between impulse response positions. The implementation of the impulse responses into the core Unity application would allow an application such as this one to be self-contained and consequently be widely disseminated through heritage organisations to reach new audiences, expanding engagement with the site. This research has determined a prototype for a visualisation that dynamically integrates 3D models and auralisation so that they work in unison in a highly responsive, multisensory virtual experience. It also champions the capturing of the cultural and intangible side to its heritage, moving people to want to visit the cave and equally to conserve it. A c k n o w l e d g e m e n t s Sincerest thanks go to Stuart Jeffrey and the Glasgow School of Art for support throughout this project, as well as Ronan Breslin, Victor Portela, Mike Marriott, Matthieu Poyade, Brian Loranger, Daniel Livingstone and Jessica Argo. Special thanks also go to Iain Laird and Nick Green for their expertise regarding auralisation. Thanks indeed to Helen Slaney for sharing your forthcoming article with Anna Foka and Sophie Bocksberger. Finally, thank you to the VIEW Journal editors and the anonymous reviewers for your constructive criticism and guidance. B i o g r a p h y Shona Noble holds a Master’s Degree in International Heritage Visualisation from the Glasgow School of Art. She is passionate about telling the stories of past people and cultures, conveying many layers of meaning of arts and heritage through compelling narratives and visuals. Her specialisms include researching and creating audience- focussed digital experiences using futuristic technologies, and developing interactive digital content for the arts and cultural heritage. 88 Jonathan Sterne, ‘Space within Space: Artificial Reverb and the Detachable Echo’, Grey Room, 60, 2015, 125, http://sterneworks.org/ reverb--sterne.pdf. 89 Anna Foka et al., ‘Beyond Humanities qua Digital: Spatial and Material Development for Digital Research Infrastructures in HumlabX,’ Digital Scholarship in the Humanities 33, no. 2, 2018, 264–78, https://doi.org/10.1093/llc/fqx008. http://sterneworks.org/reverb--sterne.pdf http://sterneworks.org/reverb--sterne.pdf https://doi.org/10.1093/llc/fqx008 S. K. Noble, Fingal’s Cave: The Integration of Real-Time Auralisation and 3D Models 19 VIEW Journal of European Television History and Culture Vol. 7, 14, 2018 DOI: 10.18146/2213-0969.2018.jethc150 Publisher: Netherlands Institute for Sound and Vision in collaboration with Utrecht University, University of Luxembourg and Royal Holloway University of London. Copyright: The text of this article has been published under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 Netherlands License. This license does not apply to the media referenced in the article, which is subject to the individual rights owner’s terms. Her most recent research focussed on the integration of audio in heritage visualisation and culminated in a virtual reality project that presented an audiovisual exploration of Fingal’s Cave, demonstrating the Isle of Staffa’s cave’s extraordinary acoustics. This project features real-time photorealistic visualisation and is designed to be entirely immersive, allowing uninterrupted exploration of the digitally recreated cave. It includes an innovative interactive element in which a viewer can speak into a microphone and hear their voice auralised as it would be inside the cave, a first in heritage visualisation. http://creativecommons.org/licenses/by-nc-nd/3.0/nl/deed.en_GB http://dx.doi.org/10.18146/2213-0969.2018.jethc150 _Hlk534987600 _Hlk534889623