Title of the Paper: Example Paper for Business Systems Research Journal econstor Make Your Publications Visible. A Service of zbw Leibniz-Informationszentrum Wirtschaft Leibniz Information Centre for Economics Navratil, Jiri; Ubik, Sven; Melinkov, Jiri Conference Paper Performing Arts Across the Continents: Our Way to Digital Humanities and Arts Provided in Cooperation with: IRENET - Society for Advancing Innovation and Research in Economy, Zagreb Suggested Citation: Navratil, Jiri; Ubik, Sven; Melinkov, Jiri (2015) : Performing Arts Across the Continents: Our Way to Digital Humanities and Arts, In: Proceedings of the ENTRENOVA - ENTerprise REsearch InNOVAtion Conference, Kotor, Montengero, 10-11 September 2015, IRENET - Society for Advancing Innovation and Research in Economy, Zagreb, Vol. 1, pp. 227-234 This Version is available at: http://hdl.handle.net/10419/183653 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by-nc/4.0/ www.econstor.eu 227 ENTRENOVA 10-11, September2015 Kotor, Montenegro Performing Arts Across the Continents: Our Way to Digital Humanities and Arts Jiri Navratil, Sven Ubik, Jiri Melnikov CESNET, Czech Republic Abstract One of the strategic projects initiated in Europe in the period 2011-2014 was the project DARIAH-EU which had the main goal to elevate research in Humanities and Arts science using digital technologies and create special infrastructure across Europe. Czech Republic is in the phase of joining this infrastructure. Digital Humanity is a new scientific discipline which appeared in the last decade in many universities over the world and spread into many directions. We joined this activity in the field of performing arts. Our objective was to verify whether modern computer network and audio-visual technologies can enable collaborative work of performing artists when they are distributed across large distances and what are the requirements and limitations. We will describe our experience from the events which we organized or on which we participated during the last 4 years in Europe, US, Malaysia, Korea and Taiwan. The experiments showed that Global Performances as new form of performing arts can be arranged for different type of artists. The GP bring new very interesting impressions for artists and for the spectators. We demonstrated that GP can be used for presentation of performing arts on the festivals, cultural exhibitions and fairs. We believe that, It could have very positive economic effect. It is our IT contribution to Digital Humanities and Arts. Keywords: HD Video, 4K video, 3D technology, cyber performance, live surgery JEL classification: C88 Acknowledgments: This work was supported by the CESNET Large Infrastructure project (LM201005) funded by the Ministry of Education, Youth and Sports of the Czech Republic and used multiple academic networks including Geant, TEIN, GLIF, Gloriad, KREONET, TWAREN and Internet2. Introduction In this paper we describe our way to the field of Digital Humanity (DH). DH describes the emerging scientific discipline that seeks to integrate the principles of humanities and science. It is based on the assumption that mankind has ever larger amounts of data, which allows us to use new ways of analysis and presentation. Due to the digital nature of the data, it can be easily processed using various computational algorithms, more deeply analyzed and then used to acquire new knowledge about mankind, bring new views on our history and our behavior. DH just like other scientific disciplines appeared as result of evolution. Every university, faculty or department involved in the humanities had a profile which was shaped over the years of its existence. However, development of information technologies (IT) and the Internet itself with millions of users brought into many areas entirely new possibilities and each year brings many other directions that will further differentiate. EU reacted to this phenomenon several years ago by opening the project DARIAH-EU (Digital Research Infrastructure for the Arts and Humanities). As seen from the title, it explicitly included the arts in this field. The main task of this project was the 228 ENTRENOVA 10-11, September2015 Kotor, Montenegro establishment of ERIC (European Research Infrastructure Consortium). This was achieved in August 2014. Fifteen members of EU established this consortium. The tasks for ERIC are defined quite broadly which corresponds to the current reality. Although the Czech Republic is not a member of the ERIC, we are interested to participate with this infrastructure because some of the activities that we carry out for years certainly belong to the area covered by ERIC. Most results presented in this paper were achieved due to the activity of CESNET in global networking. CESNET researchers during its existence developed several tools which allow transmitting video signals in HD formats and the latest formats as 4K and 8K which are used in TV and film industry on high-speed networks. In following paragraphs we will describe how we were able to use them for different performing art sessions and also for presentations of one of national heritage project. Our work in the field of art Remote concerts One of our first works in the field of culture was the live transmission of a harp and guitar concert in 3Dfrom Czech Republic to the US. The goal was to create an illusion for remote visitors as they had been sitting in the concert hall close to the musicians. Figure 1 Musicians in the Studio (left) and Remote View via 3D Glasses (right) Source: Author’s illustration We presented this concert in 2013 to the meeting of the APAN cultural working group. The success of it opened us future collaboration with this group. The following two pictures show this performance. In Figure1there are musicians in Prague studio (left) and in the right there is a remote view of them via 3D glasses. We continued with this effort of remote music performances with the aim to demonstrate the capability of current networks and transmission technology to enable such types of art sessions for public. Later, we coordinated a European project E-music, supported by the open call of the pan-European GN3plus project, which brought us an opportunity to collaborate with more partners. In this project we focused on the study of relations between transmission delay and personal feelings of musicians for playing in the distributed environment. 229 ENTRENOVA 10-11, September2015 Kotor, Montenegro Global Performance Global Performance (GP) is a new form of a live performance as results of a joint effort of artists and engineers located across the world, working together in real time. The team includes network engineers and researchers, audio-visual technicians, programmers, musicians, dancers, scene designers and choreographers spanning multiple areas. The first event where we participated took place in Daejeon, Korea during the 36th APAN Meeting in Aug 2013. The performance was called “Dancing beyond time” and it was conducted by Prof. Boncheol Goo from KISTI, Korea. The event began at 08:55 UTC/GMT simultaneously in Salvador, Brazil (BR), Prague, Czech Republic (CZ), Barcelona, Spain (ES) and Daejeon, Korea (KR). Preparation of such an event involved approx. 100 people from nearly 20 organizations located in three continents and it last several months. The original idea was simple, to organize a joint dance of the Avatars. The initial idea of Avatars came out from the famous Kinect project when people can dance in front of the TV screen and the movement is scanned by a Kinect device. The result is a moving virtual person – Avatar which is shown in other window. The idea for this GP developed into music and dance on several places and transferring their audio and video as well as the Avatars into a remote site (Korea) in a synchronized way. There were musicians “Unlimited trio” playing live music in CZ, dancers in KR, ES and BR and the main audience in KR. Video shoot in ultra-high- resolution (4K) along with audio were transmitted to KR. The sound was sent to ES and BR simultaneously. In KR, BR and ES a dance was performed in front of a Kinect sensor. Each skeleton data of dancers in ES, BR and KR were exchanged and sent to the stage at the same time. All dancers could see avatars of each other. To mitigate the latency issue, computer generated skeleton data using Kinect sensors generated by a MaxMSP program were used between sites. During a preparation phase, the director decided to create a more complex scene in Daejeon. The final big screen (size 16x6,5 m) was split into several regions. Each of these regions was dedicated for different video. The left half of the screen was for musicians from Prague with 4K projection. The upper part of the right region was dedicated for a joint dance of Avatars and below them there were projected three original videos of dancers sent from different sites. The performance got true interesting expression when a Korean dancer performed live dance directly in front of the screen. The whole performance had a perfect illusion of a compact live local performance. The audience was approx. 100 people in Daejeonat the conference closing session and additional participants in other locations. The big screen with all videos in the venue in KR is shown in the illustration in Figure2. Figure 2 Two Views of Final Stage of the Global Performance for APAN36 Source: Author’s illustration 230 ENTRENOVA 10-11, September2015 Kotor, Montenegro A slightly different GP was prepared for the APAN38 held in Taiwan. The performance was called „Dancing in Space“. We again collaborated with partners across continents. In this case, one big screen was used to compose video of musicians and dancers together. The main video of a musical trio was transferred from Prague and into it was composed live video of a cellist who played in Miami (US) and dancers who danced in Barcelona (Spain) and in Nantou (Taiwan). The final screen from this performance is shown on Figure3. Figure 3 Global Performance for APAN38 Source: Author’s illustration Work in the field of national heritage Langweil model of old Prague. Langweil model of Prague as a spatial representation of the city is a unique work of art and unique in the world of its kind. The Model from the cardboard on the wooden structure was created in the years1826-1837 by an assistant at the University Library in Klementinum, Antonín Langweil. The model contains over two thousand buildings in the historic center of Prague in a life-like version with all building façade sand ornate details. Approximately half of the buildings were later demolished or rebuilt. Most of building are 3 - 6 cm tall but of course contain various dominant features (towers, columns, etc.) which are around 20 cm tall. The physical model is kept in a kind of “greenhouse” with tinted glass and limited lighting inside. Figure 4 Langweil Model in Prague Museum (left) and Zoomed Details (rights) Source: Photographs are a reproduction of a collection item administered by the City of Prague Museum, the author of the item is Antonin Langweil 231 ENTRENOVA 10-11, September2015 Kotor, Montenegro The digitization of the Langweil model was an incredibly difficult project from a technical point of view. The model could not be touched in any way as there was a risk of irretrievable damage. Digitization had to be carried out without contact, purely on optical principles with special camera. The result of the photography was approximately 250,000 photographs with 16 Mpix resolutions, which represents a considerable amount of data for processing into 3D model. Digitization was done by a professional company Kit Digital in collaboration with the Department of computer graphics and interaction of the Czech Technical University. The digital model is owned by The City of Prague Museum and it is used only in this Museum for local video presentations. We obtained a permission to use one part of the model for our network experiments. We decided to use the model interactively to allow people “to be inside”. In the first step the data was converted into the form usable for CAVE. The CAVE stands for CAVE Automatic Virtual Environment and takes the form of a cube-like space in which images are displayed by a series of 3D projectors. When a visitor enters into CAVE he has a chance to be in new virtual reality. In this case on the streets of old Prague. As the next step, we extended this idea to allow the user to walk in the models remotely. The idea of remote interaction was firstly introduced in the project called „C2C”. The goal of this joint project of CESNET and Institute of Intermedia (IIM) at the Czech Technical University was to send and share data from one CAVE into other immersive environments. For such remote interaction the remote sensor of position was integrated into the system, to allow a remote user to move around in the model, which is stored and rendered locally. The remote user, located somewhere in the world sees the video from the CAVE (or at least the front wall) and use a joystick in his hand to simulate a walk. This principle is illustrated in Figure5. The model was projected on the CAVE walls and on the remote 3D screen according to the position of the person in the CAVE or according to the joystick movements if the remote person. To see the video in 3D, all attendees of such a session must have special glasses. We called this demo „Virtual Walking in historical Prague“, because visitors could move inside the model according their own selection. This project was first time demonstrated on a CineGrid Workshop in San Diego in 2012 and second time on a joint Internet2 and APAN meeting in Honolulu in 2013. Figure 5 Diagram of Using CAVE with Remote Control and 3D Presentation Source: Author’s illustration 232 ENTRENOVA 10-11, September2015 Kotor, Montenegro The scheme of network configuration is shown in Figure6. It indicates also the time for video transfer from Prague to Hawaii and also a backward channel via which we sent the position information to the CAVE. Figure 6 Scheme of Networking for APAN Demo Source: Author’s illustration Discussions of necessary conditions In this part we have to discuss several conditions which play an important role in our work. The first is networking, the second devices for video transmissions and the third presentation devices (large screens, multidimensional screens etc.) Only the perfect integrated work of all three elements can bring results which are well recognized in international community. CESNET is a part of GEANT community (EU academic network), member of GLIF (Global experimental network) and we collaborate with other leading networking partners such as Internet2 in the US and APAN (Asian Pacific Advanced Network). This allows us to bring video into many places on the globe. The video transporting tools which we use in our projects are MVTP-4K and Ultragrid. MVTP-4K is a hardware device based on FPGA technology allowing fast transmission video with minimal internal latency. Ultragrid (UG) is a free software solution which can be used on a PC or MAC platform. The availability of these tools was the second main reason for our invitations to participate at the global project. Each tool has its own advantages. For the demonstrations described in this paper, we used MVTP-4K. This device was originally designed for transmission of 4K video, but the experience showed that it can be used in many other applications due to its very low added latency It can be used for remote access to scientific visualizations, for medical sessions connecting operating theatres with lecture halls and conference venues or for eCulture events and collaboration where we need more parallel videos. The limiting factor is that the same device is needed in all sites. The device is currently commercially available under the name 4KGateway. CESNET has several units available for experiments and we provide them for important events on anon-profit basis. 233 ENTRENOVA 10-11, September2015 Kotor, Montenegro Figure7 MVTP-4K Box Enabling Multiple Video Transmissions Source: Author’s illustration Conclusion In our paper we showed our present work in the field which can be understand as a starting work in a national effort to join the European project ERIC. In the future, we plan to continue in this work and to investigate more deeply the use of immersive visualizations for collaboration in performing arts, and involving other kinds of artistic expressions, such as fine arts and paintings and installations of more permanent infrastructures for the use in university lectures. References 1. 4Kgateway “About” (2015), available at: http://www.4kgateway.com/ (accessed May 30th 2015). 2. APAN (2015), “About APAN”, available at: http://apan.net (accessed May 30th 2015). 3. APAN-35 (2015), “An International Conference of Networking Experts”, available at: https://meetings.internet2.edu/2013-01-jt/ (accessed May 30th 2015). 4. APAN-36 (2015), “Asia-Pacific Advanced Network 36th Meeting”, available at: http://www.apan.net/meetings/Daejeon2013/0101.php (accessed May 30th 2015). 5. DARIAH-EU (2015), “About DARIAH-EU”, available at: https://dariah.eu/about.html (accessed May 15th 2015). 6. DARIAH-ERIC (2015), “DARIAH-EU: Annual Report 2011”, available at: https://dariah.eu/fileadmin/Documents/DARIAH-EU_Annual_report_2011.pdf (accessed May 15th 2015). 7. GLIF (2015), “Connecting research worldwide with lightpaths”, available at: http://www.glif.is/publications/info/brochure.pdf (accessed May 15th 2015). 8. Halak, J., Ubik, S. (2009), “MTPP - Modular Traffic Processing Platform”, Proceedings of the IEEE Symposium on Design and Diagnostics of Electronic Circuits and Systems, Liberec, pp. 170-173. 9. Halak, J., Krsek, M., Ubik, S., Zejdl, P., Nevrela, F. (2011), “Real-time long-distance transfer of uncompressed 4K video for remote collaboration”, ELSEVIER FGCS Vol. 27 No. 7, pp. 886-892. 10. IIM (2015), “Projects”, available at: http://www.iim.cz/?id=148&lang=1#node (accessed May 30th 2015). 11. Langweil (2015), “O projektu” [About the Project], available at: http://www.langweil.cz/projekt.php (accessed May 15th 2015). 12. MVTP4K (2015), “About us”, available at: http://www.ces.net/project/qosip/hw/mvtp-4K- v2.pdf (accessed May 30th 2015). 13. Navratil, J.; Sarek, M.; Ubik, S.; Halak, J.; Zejdl, P.; Peciva, P.; Schraml, J. (2011), “Real-time stereoscopic streaming of robotic surgeries: e-Health Networking Applications http://www.4kgateway.com/ http://apan.net/ https://meetings.internet2.edu/2013-01-jt/ http://www.apan.net/meetings/Daejeon2013/0101.php https://dariah.eu/about.html https://dariah.eu/fileadmin/Documents/DARIAH-EU_Annual_report_2011.pdf http://www.glif.is/publications/info/brochure.pdf http://www.iim.cz/?id=148&lang=1 http://www.langweil.cz/projekt.php http://www.ces.net/project/qosip/hw/mvtp-4K-v2.pdf http://www.ces.net/project/qosip/hw/mvtp-4K-v2.pdf 234 ENTRENOVA 10-11, September2015 Kotor, Montenegro and Services (Healthcom)”, CESNET, Prague Czech Republic, 13th IEEE International Conference on Issue Date: 13-15 June 2011. 14. Ubik, S., Navratil, J., Melnikov, J., Goo, B., Cuenca, D. and Santana I. (2014), „Collaborative visual environments for performing arts“, CESNET, KAIST, i2CAT, Universidade Federal da Bahia, The 11 International konference on Cooperation Design and Visualisation, Sept. 14-17, 2014, Seattle, USA. 15. Ubik, S., Navratil, J., Zejdl, P., Halak, J. (2012), “Real-Time Stereoscopic Streaming of Medical Surgeries for Collaborative eLearning”, 2012 CDVE Mallorca, Spain, Proceedings pp. 73-77. About the authors Jiri Navratil received his PhD in Computer Science from Czech Technical University at Prague in 1984. He worked for 30 years at Computing and Information Center of CTU in different positions linked with High Performance Computing a Communications. During his several sabbatical leaves he worked in Switzerland, Japan and USA in the field of networking. Since 2006 he started work for CESNET - Czech Education a Scientific Network as leader of group supporting special research applications using high speed Internet. In the last years he participated on several multimedia performances organized in frame of large international cooperation in different fields. Author can be contacted at jiri@cesnet.cz Sven Ubik received his MSc and Dr in Computer Science from the Czech Technical University. He is currently a senior researcher in CESNET and the head of the research group Technologies for network applications. He has created a Network Visualization Lab in collaboration with the Czech Technical University. His research interests include novel applications for distance collaboration, digital representation and distance access to culture heritage, 3D models; hardware accelerated video processing and optical networks.. Author can be contacted at ubik@cesnet.cz Jiri Melnikov works in CESNET as an administrator of high-resolution multimedia laboratory and has experience with developing applications for low latency and high quality transmissions. He received his MSc. in Computer Science from the Czech Technical University in 2012. His research interests also include software defined networking and software development for high-resolution tiled displays walls. He currently resides in Prague and can be contacted at melnikov@cesnet.cz