key: cord-0589405-9ugs0tyq
authors: Lee, Lik-Hang; Lin, Zijun; Hu, Rui; Gong, Zhengya; Kumar, Abhishek; Li, Tangyao; Li, Sijia; Hui, Pan
title: When Creators Meet the Metaverse: A Survey on Computational Arts
date: 2021-11-26
journal: nan
DOI: nan
sha: 01ecf1b716cfa2edf0c88b670a16f106c5c9f3f7
doc_id: 589405
cord_uid: 9ugs0tyq

The metaverse, enormous virtual-physical cyberspace, has brought unprecedented opportunities for artists to blend every corner of our physical surroundings with digital creativity. This article conducts a comprehensive survey on computational arts, in which seven critical topics are relevant to the metaverse, describing novel artworks in blended virtual-physical realities. The topics first cover the building elements for the metaverse, e.g., virtual scenes and characters, auditory, textual elements. Next, several remarkable types of novel creations in the expanded horizons of metaverse cyberspace have been reflected, such as immersive arts, robotic arts, and other user-centric approaches fuelling contemporary creative outputs. Finally, we propose several research agendas: democratising computational arts, digital privacy, and safety for metaverse artists, ownership recognition for digital artworks, technological challenges, and so on. The survey also serves as introductory material for artists and metaverse technologists to begin creations in the realm of surrealistic cyberspace.

Art is defined as a vastly diversified range of human activities that create products of imagination and creativity in various channels, including but not limited to visual, auditory, dancing, theatrical (c) Creation of 3D artworks in virtual reality named Tilt Brush. In 2020, the COVID-19 pandemic significantly impacted our lifestyle. The digital transformation has been expedited by years, and its impact can be long-haul [52] -people study and work remotely and spend their leisure time with digital tools and virtual platforms [36] . As such, opportunities emerge for computational arts in the aftermath of this global crisis. While the COVID pandemic has engendered a sizeable shock to industries like filmmaking, where the employment in motion picture and sound recording in the US has dropped by more than 40% in 2021 [162] , other industries architecture in 1919, namely "Proun #5A" (Figure 2a) , was recreated in a 3D virtual environment that allows 3D-object rendering and repositioning of multiple paintings or drawings [11] .

(a) EI Lissitzky's (re)constructed Proun #5A in the metaverse.

(b) Constructing situated-learning metaverse platform for Japanese culture. The latest technology further pushes the boundary towards the realisation of the metaverse that pinpoints the key concept of digital twins and ultimately virtual-physical 105 co-existence [132] for multi-user socialisation and co-creation. Through the lens of augmented reality (AR) headsets such as Meta Version 2 [140] , users can see virtual-physical blended worlds, in which virtual content superimposing on the top of physical counterparts of smart cities [130] . On the other hand, users with virtual reality (VR) headsets can situate in some immersive environments. As such, users, as represented by their avatars, will interact with other avatars in virtual cyberspaces inside the metaverse, i.e., the Second Life [74] . Such interaction can involve item enquiry and selection [54] , and hence numerous items should first be created by multiple metaverse creators.

Between the two ends of the spectrum (AR and VR), mixed reality (MR) requires high levels of scene understanding. This enables virtual objects (including virtual art) to seamlessly interact with the counterpart of physical objects. Decentralised governance of scene information requires an effective and real-time information sharing scheme among peer creators in a virtual-physical shared space [183] . Meanwhile, sensors in real life can get information from the physical world and hence impacts virtual projection in virtual worlds [194] . Based on the above infrastructure, creators in the metaverse can socialise with each other in the virtual worlds, and achieve their common goals in situational contexts [169] , e.g., creating new artistic contents that serve the goal of expressing Japanese culture and architectures ( Figure 2b ) [195] . Additionally, the metaverse is characterised by a virtual marketplace, driven by blockchain technology, which facilitates peer-to-peer item trading (Figure 2c ) in the metaverse [48] . It is worthwhile to mention that the trading of (artistic) items is the fundamentals of building a metaverse community, in which creators or artists will spend a significant amount of time to create novel and creative contents in the metaverse. Further discussion of virtual artwork trading is available in the next paragraph (Section 1.2).

With the ease of duplicating and disseminating digital and media arts, many argue the orthodoxy valuation of artwork (i.e., based on scarcity and exclusiveness) is less relevant for this new type of creative work [30, 31] . However, Non-Fungible Tokens shed light on the conundrum associated with digital artworks' inherent lack of scarcity and authentication [152] . At the heart of virtual art trading, there lies the Non-Fungible Token (NFT), an eminent concept where its appeals go beyond the CMA community. Powered by blockchain technology, NFT can establish publicly recognisable ownership in virtual objects that are susceptible to piracy and forgery. The means by which such ownership is enshrined should not be unfamiliar to readers with some knowledge in cryptocurrency. Records of ownership in NFT, congruous to cryptocurrency, are logged by countless devices across the globe in a decentralised manner. In simpler terms, instead of storing the actual artwork (e.g., an image of a popular meme) like backing up data on a server, blockchains log records of 'who 7 owns what' pointing towards certain objects, where individuals cannot freely subvert these records. Therefore, artists who work on physical arts (such as large size art installations) may also mint new tokens to serve as certificates of ownership, where they can then offer these tokens to art collectors. A close analogy can be where companies including Verisart and Artory take the initiative to utilise the blockchain to verify physical artworks [209] . Though this idea may seem radical at first glance, it does possess some intrinsic appeals towards art gallery and private collectors who wish to have their collection displayed publicly or simply lacks the storage capacity. Additionally, it may also benefit art traders who expect high turnover for their physical artworks as it saves them the transportation cost of shipping the works back and forth. In return, some intermediary may store the physical artworks with a reasonable charge for the traders who frequently exchange ownership through NFTs. Analogously, nascent physical art forms like robotic art may also benefit from a wider acceptance of this new business paradigm. As a sharp contrast to a significant proportion of art dealers' conservative attitudes towards new technologies, say, virtual galleries through VR devices [201] , the market has been more willing to embrace NFTs in recent years. In complementary to the growing public interest in Non-Fungible Tokens as we demonstrate in Figure 3a and rising NFT user count reflected by the amount of user wallets 10 , developed marketplaces for digital artworks powered by NFTs already exist. On these platforms, creators offer a broad-spectrum of objects [132] , including but not limited to drawings, avatars, audio and 3D models. OpenSea, as an example of one of the leading NFT trading platforms, is valued at 1.5 billion USD [151] , with exponential growth in trade volume since its birth 11 . Successful adoption in NFT-based digital arts takes place outside the common NFT art marketplaces: the Cryptokitties, one of the first NFT collectable projects, received universal attention with one rare digital cat being sold at a jaw-dropping price of 600 ETH. Figure 4c shows the cryptokitty that was sold for 600 ETH [55] . Likewise, the blockchain-based game F1 ® Delta Time, illustrated in Figure 4b , also achieved a similar triumph by releasing a limited-edition virtual race car. Some refer to this class of NFTs commonly seen in blockchain-based games as 'smart collectables', which are digital tokens that enable additional functionality beyond plain ownership [56] . Once again, 7 It is worth noting that the word 'who' should not be taken explicitly. Instead, it is individuals' wallets addresses that get recorded. 8 https://bit.ly/3qsyijx 9 https://cryptoart.io/data 10 https://bit.ly/31Ybn5k 11 https://bit.ly/3BU29mR using the game F1 ® Delta Time as an example, the differentiated tokens represent different in-game collectables that come with different specifications proportional to their rarities (Figure 4b ).

(a) The Economist's front page image of 'Alice in Wonderland' sold as an NFT 12 .

(b) F1 ® Delta Time: players collect and exchange in-game elements as NFTs 13 .

(c) The cryptokitty that was sold for 600 ETH 14 . Outside the conventional art industries, we also see participation in NFTs from the wider media community. As demonstrated in Figure 4a , The Economist, one of the most well-known magazines, minted its own NFT for its cover image of 'Alice in Wonderland' featured with cryptocurrency [51] . This is not an unprecedented move of the press industry, as other media outlets, including Times and The New York Times, have already jumped on the NFT bandwagon in the past 15 . Examining the NFT market from users' angle, NFT trades appear to be clustered at the current phase: traders tend to focus on exchanging virtual objects of similar characteristics with others adopting similar specialisation [159] . The clustering phenomenon also takes place in the price determination of similar digital artworks [57] . Additionally, there appears to be some clustering for NFT artists too when it comes to their artworks' undertones: artists tend to exhibit clear emotional undertones with their digital artworks, where positive connotations are more common than negative ones [64] .

However, NFT is not a silver bullet to all the perplexities revolving around digital and media arts. First, it is worth questioning whether the explosion in the trade volume of NFTs truly remarks a reclaim of power by art creators through the decentralised technology, or it simply represents yet another asset bubble, which releases misleading signals that create a detrimental effect on the resale market due to oversupply of artworks [66] . Apart from the debate of whether this financial inventiveness is de facto instrumental to artists [220] , individuals should also stay cautious with the proclaimed technological robustness against fraudulence. It is worth noting that a wide-range implementation of NFT on digital arts is not equivalent to a termination of illegal appropriations: individuals can still take a screenshot of an artwork for personal or even illegal commercial use without being forced to pay the owner. Rather, as highlighted earlier in this section, NFTs should be thought of as licenses that outline the ownership. Although complete annihilation of illegal appropriations is unrealistic, we may find a silver lining in the composition of digital product valuation. In the context of customer value theory originated from marketing, how aesthetically appealing (emotional value) and the underlying social implication of owning the object (social value) may be more relevant to consumers of digital goods [113] . Nonetheless, piracy and forgery can impede social value if digital arts are demanded to complement individuals' social status and the licensing itself is insufficient to secure the sense of exclusiveness. In addition, an NFTbacked virtual artwork is not tantamount to authenticity: there are several documented instances where malicious opportunists beguiled NFT collectors by selling digital artworks imposed as authentic pieces produced by famous artists [132] . Even worse, the possibility of scamming through 'sleepminting' 16 can bewilder inexperienced NFT collectors into considering a counterfeit as genuine. Therefore, one may come to the realisation that the non-fungibility feature of these digital tokens acts like a double-edged sword: we can only establish ownership by making the tokens differentiable, but it also implies they are not like cryptocurrencies that have going rates and are fungible. Consequently, authentication becomes more sophisticated for decision-makers. Moreover, NFT does not grant users immunity to conventional scams such as impersonating as 'customer service' 17 . Thus, both artists and collectors should remain vigilant when conducting trades in digital artworks, albeit the remarkable progress the NFT stands for. Lastly, even when we temporarily drop the concern of NFT-related scams, the question of how precisely should an NFT-backed digital artwork be valued still lacks a commonly accepted conclusion. While some have reached out to the economics of supply and demand, which is ubiquitous in market analysis in countless industries, others have put forward more scenario-specific approaches. One instance is a rating system that values artworks based on key attributes, including past records of bids and sales [65] . Going beyond the discussion of the valuation of digital artworks, some have also inspected the accompanying trade mechanisms of NFTs through lenses such as game theory [111] . Apart from the NFTs' valuation, there are also rising concerns regarding the environmental implications associated with the large-scale adoption of NFTs. That is, the energy consumption that arises from minting and trading NFTs may potentially impede the process of achieving carbon neutrality. Nonetheless, businesses strive to ameliorate the situation by choosing production locations that use renewable energy and designing more efficient blockchains [26] . In short, even if we omit the shortcomings of the NFTs in extinguishing ownership-related hitches, there still exists significant scope for further scrutiny in the practicality of NFTs.

In spite of the underlying challenges coupled with the utilisation of NFTs as a new means of accreditation for ownership, one should not easily dismiss its merits, especially when harnessed for benevolent uses with proper regulation. Specifically, a commonly accessible virtual trading system is restorative to self-employed content creators in virtual and digital art industries, both in the conventional and the emerging art industries that will be discussed in detail in this survey. Also, as aforementioned, the application of NFTs can extend beyond artworks in the absence of tangible physical forms. In theory, they can establish ownership for both the algorithm behinds AI-Dance/ Robotic Art or the entire artwork, both the physical and non-physical components. When it comes to the metaverse, the NFTs possess the ability to rise above mere ownership-verified collection in one's wallet by endowing digital artworks with additional interactive attributes. Thus, given the prominence of this burgeoning trading channel, the art and the wider community should seek to address its flaws and carefully scrutinise its possible prospect of delivering a socially desirable outcome, where the recipients of its benefits go beyond artists and curators.

As a concluding remark for the introductory part of this survey paper, we can obviously see the rising popularity of digital artworks, driven by Non-Fungible Tokens (NFTs). Nowadays, digital artworks have been widely traded via virtual platforms, i.e., the very early stage of the metaverse mainly motivated by economic efficacy. More importantly, the metaverse, as a socialised virtual world, can accommodate various artistic creation and subsequently facilitate virtual art trading. The work of art in virtual worlds can go beyond the fundamental functions of communication through images, sounds and stories. Also, the combination of the metaverse and computational arts could also serve as the key vehicle to encourage people from different cultures and across generations to express their opinions to each other in virtual worlds characterised by perpetuity and openness. We acknowledge that a broader definition of creators could include designers in other domains such as UI/UX for computer interfaces, interaction design, industrial products, and so on. Instead, we focus our review and discussion on the unique functions of computational arts in the metaverse, as defined in a prior survey [132] .

We view the metaverse-driven worlds as an enormous and malleable canvas in the virtual-physical blended realities, while novel forms and representations of digital arts can appear in every aspect of our living environments once such realities are mature. With the background as aforementioned, our survey paper is a state-of-the-art review of computational arts, an interdisciplinary research area with arts and technologies. We did our utmost to provide comprehensive coverage in the metaverse era, such as virtual entities and scenes, calligraphy and poetry, robotic and immersive arts, and so on. More importantly, our survey serves as the first effort of integrated reviews for two complementary facets of computational arts and the metaverse. To the best of our knowledge, the most relevant survey articles pinpoint either a focused area of computational arts, e.g., art collection analysis and AI-generated images [34] and media (video) intelligence [86] , or fundamental developments of the metaverse, e.g., technological enablers and ecosystems of the metaverse [132] , the metaverse applications and industry sectors [164] , the design of virtual worlds and metaverse user perception [45] . In contrast, our survey paper aims to review the latest development of digital arts and investigate the potentials of computational arts in the metaverse era.

We review the latest development of various types of computational artworks, which could serve as the basic virtual building components in the metaverse, including computer-generated imagery like virtual photography and cinematic simulation (Section 2), textual elements -Calligraphy and Poetry (Section 3 and 4), and auditory and musical metacreation (Section 5). Next, the discussion focuses on human-engaged and user-centric artistic creation such as embodied collaboration (Section 6). Then, virtual artworks represented by physical embodiments of robotics are discussed in Section 7. Finally, we investigate the emerging multimedia of extended reality, known as Augmented Reality (AR) and Virtual Reality (VR), and the prominent features of virtual creativity and brainstorming that are highly relevant to the rise of the metaverse (Section 8). After presenting the main body of article reviews, we discuss the research agenda of computational arts in the metaverse era (Section 9).

In recent years, there have been a surge of artists working with digital virtual imagery, often inspired and influenced by the language of traditional photography and cinema, while exploring the new possibilities offered by computer rendering and simulation technologies [173] . The broader context of this trend is that photography and cinema, the earliest forms of "new media" art, are becoming more computational and moving toward virtual worlds and networked spaces [147] . The boundaries between computer-rendered images and photography have been blurred [118] . On the one hand, computer rendering is replacing traditional photography in many functional areas of application. When consumers today shop for a desk or visualise a new kitchen on Ikea's website, they may not realise that the images are likely to be computer renderings rather than traditional photography, yet this has long been the norm in the architectural and product design fields [192] [212] . These images replace the function of photographs for the lay viewer, at which point they become de facto photographic images. As artist and professor Claudia Hart pointed out, computer generated imagery (CGI) is a form of post-photographic technology [91] . In this context, it also has become a new form of photography, namely virtual photography. One of the prominent examples of CGI as photography in popular culture is the virtual influencer @lilmiquela on Instagram, a social media platform catered to image sharing. Taking on a persona of a virtual young female identity of Miquela, the account shares highlights in Miquela's "life", embedding the realistically rendered virtual avatar into real environments, and often even "posing" with various real people and celebrities [193] . This is part of a recent trend, dubbed "celebrity 2.0", of virtual stars created with CGI and simulation technology gaining popularity in the larger culture sphere [47] .

On the other hand, a variety of new visual sensing technologies, homologous to traditional lens-based photography, are expanding new realms of application. Perhaps one of the best known of these applications in popular culture includes the fact that in 2016, former U.S. President Barack Obama became the first president to have an official 3D portrait created by a team led by the Smithsonian Institution, through a process that involved capturing facial form, colour and texture with a mobile Light Stage equipped with fourteen cameras and fifty lights. Combined with data of the rest of the head and shoulders captured by a structured light 3D scanner, the Smithsonian team was able to recreate a complete and accurate 3D representation with which to produce the physical bust using 3D printing [15] (Figure 5a ). In many ways, 3D scanning shares or utilises some of the same techniques as photography, capturing traditional visual information with a camera while using enhanced assistive means, such as structured light pattern or photogrametry, to capture additional spatial distance information to not only record colour and light, but also the three-dimensional forms [127] [24] . In this post-photographic era, 3D scanning can be seen as a form of photography in the broadest sense, or Augmented Photography. This technique has in recent years been extensively studied in the museum field and is being applied to cultural heritage preservation, with the hope that this new augmented photography will not only provide a 2D record as traditional photography does, but also a complete record of the three-dimensional morphological details of the artefacts with surface material and colour properties, thus enabling reconstruction in the three-dimensional space [174] [204] . In addition to still images, CGI can naturally be used to create moving images. In the early days of CGI, this field was only accessible to animation studios or teams because of the time and labor involved, and in the personal creation and research domain, artists often need to collaborate with technology research teams or gain access to sophisticated computing facilities [172] , such as the case of Rebecca Allen using early 3D digitization and animation technology to create a music video for the pioneering German electronic music group Kraftwerk in 1986 [207] . Or in another case, the entire creation is driven by people who are engineers by profession, such as the simulated creatures produced by engineer and visual effects developer Karl Sims, which are exhibited as works of art while winning the Prix Ars Electronica media art award in both 1991 and 1992 [189] . With the increased usability of software, many artists have begun to adopt CGI for personal artistic expression. One notable recent example is British artist Ed Atkins, who uses CGI to create highly personal video artworks, which is noted for its "syncopated montages of sounds and filmic images. [199] " His works, such as Ribbons (2014) (Figure 5b ), often show virtual male characters speaking poetic or mundane phrases with ambiguous meanings, accompanied by slightly exaggerated camera movements, bokeh, lighting and dust effects, creating a rich and layered sense of humour and melancholy at the same time. When it comes to the techniques used, Atkins believes that animating with virtual avatars allows him to work as an independent artist and to blend visual, sound and narrative elements. The avatars allow the artist to take on different persona and express his thoughts and emotions without constraint [10] . With emerging tools targeting virtual human creation such as Metahuman and detailed 3D-scan libraries such as Quixel Megascans, high-quality CGI creation will become even more accessible to creators.

The development of 3D graphics in the field of games is also encouraging. Improvements in usability and graphics quality in game engines have made it possible for cultural and artistic creators to use them as a regular tool [13] . Game engines have the benefit of real-time rendering compared to traditional 3D content creation tools, allowing artists to get visual feedback immediately without having to spend time waiting for rendering results, making the workflow more intuitive and smooth. At the same time, the "virtual production" pipeline based on game engines also enables film production crew to quickly obtain the virtual backgrounds needed for visual effects, which can be shown on large LED screens on set in the studio and captured directly by the camera at the same time as the performances, without the need for complex and troublesome green screen shooting and compositing in post-production [149] . This has become one of the new trends worth exploring in the field of video production. In the realm of using game engines to create movingimage artworks, Lawrence Lek, an artist with a transnational background, uses Unreal Engine to produce art films with science fiction narratives [134] , which has been noted for his exploration of the concept "Sinofuturism" [221] . Lek's narrative themes span artificial intelligence, financial market, simulation, and the identities of Singapore and Asia. In several of Lek's works such as Geomancer (2017) and AIDOL (2019) (Figure 6a ), the protagonists are either an artificial intelligence who want to break out of their constraints and to create art, or a human creator whose relevance has faded due to the popularity of "creative AI," thus highlighting the tension between AI and creativity. The films present large-scale futuristic sci-fi cityscapes, and in about a decade ago, such works would have been almost impossible to be done by a single artist due to software usability and the limitations of computing power. Today, thanks to the relative ease of use of game engines and the continuous improvement of real-time rendering quality, artists can avoid the time and expense associated with CGI and complete long-form video works that emphasise conceptual thinking and narrative expression in a relatively short period of time. In addition to its real-time rendering capability, the game engine also allows for programmed mechanism and interactivity due to its computational nature, which means that the generated image no longer has to be on a fixed timeline, but instead can be full of unexpected dynamic events. Artworks created by taking advantage of this feature can be considered as "artistic simulation," when the image progresses according computation and is visually represented through time. When such artistic expression includes influences from the language of traditional cinema, such as camera movements and editing techniques, it can be also considered as "Cinematic Simulation. " The ideas behind these terms have been practised by artists actively in recent years and discussed in art reviews when referring specifically to the works of these artists, but in-depth theoretical treatments on simulation as art remain sparse [46] . One prominent example of practice is the British artist John Gerrard, who uses the game engine Unigine to create surreal simulated landscapes (Figure 6b ). Gerrard states that while there is always the concept of duration in traditional cinema, his work, though still visually presented as video, is theoretically endless because of the use of real-time simulation techniques [73] . This idea is best reflected in one of his early works, One Thousand

Year Dawn (2005) , in which the simplistic image shows a character standing on a beach facing a sunrise that is set to take a thousand years to complete. Many of Gerrard's recent works feature meticulously reconstructed virtual representation of industrial and agricultural facilities, such as data centers, solar power plants, and pig farms, often in remote and uninhabited locations. Gerrard "simulates" the form and dynamics of these facilities, while the time-of-day and weather in the images are synchronised with the location of the facility. He conceptualises these works as digital portraits of such facilities. During the production process, the artist's team took thousands of photographs in the fields around the facilities, which were then used to reconstruct the sites as detailed virtual environments. In this sense, Gerrard's work can be considered an example of the exploration of the concept of the digital twin within the field of art.

Another notable artist using simulation technology is the New-York-based Chinese-American artist Ian Cheng. Inspired by his educational background in cognitive science, Cheng's work explores the agency and interactivity of virtual intelligence using "live simulation, " or "video games that play themselves" [184] . The work BOB (Bag of Beliefs) (2018-2019) is a snake-like monster lifeform that is alternately driven by multiple sub-agent systems with different personalities and desires, generating corresponding judgements, speculations, reactions and memories to various events and elements that appear in the environment in real time. Meanwhile, viewers can interact with the snake monster through their cell phones and send virtual "offerings" and text messages to the snake monster, thus influencing its response to the environment and objects. In this process, new dynamics are generated between different viewers' behaviours, so that new relationships are formed between viewers through the connection of the snake monster. Cheng considers his work as a construction of a kind of "Minimum Viable Sentience" [37] , and "BOB" is indeed an experiment on the interaction between intelligent systems and people in the context of art. Ian Cheng's other series, Emissary Trilogy (Figure 6c ), is also an exploration of artificial intelligence systems in the form of simulation presented as moving images. In each of the trilogy's environments, multiple AI characters with different goals interact with each other, driving new events in the virtual world and propelling the narrative forward. This is arguably a new way of making narrative moving-images. There is no need to write a script in advance, whether by a human being or AI. Everything is set up so that the AI agents will act according to their respective agenda, and then the story is formed in a generative way through events triggered by the evolution and interaction of the AI agents. This form of work can be considered as "event-driven cinema".

Before we discuss other building blocks for metaverse composition, we highlight the potentials of artistic augmentation of computer-generated virtual scenes and characters. The latest techniques of neural style transfers can achieve real-time processing of videos. The videos of virtual or physical worlds can be alternated by multitudinous user-defined styles [96] (Figure 7a ), which achieve high reasonable granularity of processing the video scenes by providing multiple styles for different object classes [214] (Figure 7b ). This implies that metaverse creators can choose their preferred styles, and offer highly personalised yet artistic experiences to other metaverse individuals, due to the nature of cinema-alike metaverse.

As discussed in the previous section, images and videos (e.g., painting) are excellent candidates to describe and illustrate abstract concepts. In contrast, textual elements are indispensable in the cyberspace of virtual worlds, which serve as crucial containers to express ideas and support user communications in accurate and efficient manners. The existing metaverse virtual worlds, such as Fortnite 18 , allow users to select various skins and items to customise their avatars. Such avatar makers reflect the users' desire to personalise their virtual characters and create unique representations. In a similar way, textual contents in the metaverse can be revealed as dialogues between virtual characters and interaction footprints on virtual objects (e.g., writings on a virtual wall) in the metaverse [120] . However, the current capabilities of mobile interfaces for user interaction with virtual environments suffer from throughput rates and levels of input details [130] . For example, the current input devices may not be able to support virtual pens or brushes for artistic and personalised handwriting in virtual 3D worlds, with full granularity, such as stroke width and legibility. Also, writing calligraphy would be more complicated than the counterpart of building avatars. Instead of requiring intensive training in calligraphy, every participant in the metaverse should enjoy the democratisation of artistic creation [132] . Thus, metaverse users would need alternative platforms and tools for customising their texts with artistic effects. Nowadays, artificial intelligence (AI) can recognise the features of calligraphy and result in promising effects of mimicking experts' calligraphy [223] . Considering that the metaverse has to serve human users from highly diversified backgrounds, the existing AI techniques can quickly adopt multiple linguistic contexts. As demonstrated by GANWriting [108] , AI-generated handwriting can mimic various features of handwriting style, including roundness, stroke width, skew, ligatures, and slant. Such generated handwriting texts are highly realistic, where human evaluators cannot judge whether machines or human beings write it. In addition, ScrabbleGAN [59] can synthesise handwritten English text of varying lengths, which concatenates character-token of unlabelled data into word-level calligraphy of various styles, for instance, "Supercalifragilisticexpialidocious" from Mary Poppins (Figure 8a) . Accordingly, HiGAN [71] significantly improves the similarity with the specific reference of handwriting style, while JokerGAN [219] enhances the text line conditioning for more natural and intuitive handwriting.

(a) Ten different styles of English calligraphy from Scrab-bleGAN [59] .

(b) Style transfer for Chinese calligraphy from Zigan [208] . In the context of Asian calligraphy, a most recent model of few-shot style transfer, namely ZiGAN [208] , demonstrates fast learning of Chinese calligraphy without significant sample numbers of the handwriting of calligraphy masters that are hard to acquire in practical scenarios (Figure 8b ). In addition, the calligraphy artwork can further consider the emotions of metaverse users, by detecting users' emotions. For instance, facial expression and heartbeat rates can be supported by computer vision and smart wristbands, respectively. In ECA-GAN [218] , the calligraphy style can employ various emotions, including peaceful, happy, and sad, to adjust the structures of Chinese characters in both character and discourses levels. Furthermore, the technique of photometric stereo can further convert Chinese calligraphy into 3D virtual objects. Remarkably, the heightmap of the reconstructed 3D surface can present an enriched digital copy pinpointing the beauty of Chinese calligraphy, in terms of strength and personality [103] . This opens the opportunities of circulating artworks as 3D virtual objects made by calligraphists in the metaverse.

The above paragraphs describe the properties of AI-driven artistic representation of characters, and the potential benefits of enriching the metaverse. The latest technology of automatically generated poems, such as Deep-speare, can create Shakespeare's poems that are hardly distinguished by human beings [128] . Furthermore, the existing computational approaches can generate impressive linguistic contents for such artistic texts (e.g., poems [42, 75, 139] ). Accordingly, the union of AI-generated calligraphy and poetry genres can facilitate the linguistic creativity of creators and artists in various scenes inside the virtual-physical blended metaverse. More specifically, the prior studies imply that activities between metaverse users, i.e., user-generated content including discussion topics and scenes, can achieve augmentation linguistically and artistically. Recurrent neural networks (RNN) can process discussion topics and related words, resulting in automatic poetry generation [75, 215] . Moreover, metaverse users can leverage poetry as a channel of two-way communication with gamified elements in virtual worlds, as reflected by earlier examples of human-to-human (Super Atari Poetry [155] ) and human-to-AI (ReadingRites [105] ) games through the existing cyberspace. It is also important to note that AI-driven linguistic creation demonstrates the potentials to match virtual scenes in VR and physical environments in AR. This connects to an analogy to the fusion among calligraphy, poetry and painting, highly influenced by the philosophical and spiritual tradition in ancient China, namely, Taoism and Confucianism. AI can recognise scenes (input images), and accordingly generate various poetry genres, with the following examples.

A hierarchy-attention seq2seq model, namely HieAS2S [139] , can generate two types of Chinese poetry genres: five-character quatrains (WuJue) and seven-character quatrains (QiJue), based on the image recognition module supported by 'GoogLeNet' 19 to understand a scene describing 'red plum' and 'snow' (Figure 9a ). In the context of AI-generated calligraphy, we judge the effectiveness by the level of intimacy to the human handwriting. Similarly, we should also evaluate AI-generated poetry through its fluency, meaningfulness, phonological compliance, and coherence [138, 217] . As such, Deng et al. [42] attempt to narrow down the performance between poetry generated by human beings and machines. In their work, Quality-Aware Masked Language Model drives the iterative polish of quatrain by considering the tokens, segmentation, positions, tones, and rhymes in AI-generated poems (Figure 9b ). Other works show the feasibility of automatically generated poetry across age and culture, e.g., Tang-dynasty poetry [138] , Modern Chinese poetry [188] , Spanish poetry [33] , as well as Homeric poetry [125] .

Although the existing studies show promising performance in the scene-poetry automatic generations, the grand challenges are as follows. First, poetry creations have to maintain consistency between the semantics of poems and images. We have to understand the reasons for topic drift throughout the automatic generation process, and to design preventive mechanisms, perhaps more transparent and explainable tools are required. Also, some words appear more frequently than the rest, but they could deteriorate the model effectiveness [213] . On the other hand, we would like to emphasise that most automatically generated poetry are considered as rigid outputs. Recalling an earlier example (Figure 9a ), the augmentation of five-character quatrains on top of a physical scene, a type of Chinese poem, is a potential scenario of art applications [139] . We expect that the metaverse would serve as a huge creation space that can accommodate flexible and free artwork representations, known as 'visual poems'. For instance, some visual poems 20 explore the highly blended representation between visuals and textual contents. Significant research efforts can focus on the spatial factors in virtual and physical worlds and even virtual-physical reality scenarios.

Musical Instruments for the Metaverse. In 1932, Antonin Artaud published Theatre of cruelty in which he foresaw the emergence of the metaverse and in there, recognized the unique role of musical instruments in this space [9] . Fast forward 60 years, in 1992, Jaron Lanier, one of the founding fathers of Silicon Valley, gave a presentation titled Sound of One Hand at the SIGGRAPH Conference, where he demonstrated how the sound was being synthesized and improvised in virtual reality. This demonstration marked the beginning of the musical instruments being created entirely in virtual space. After almost 20 years of stagnancy in the field, an audiovisual environment for audio composition and performance, named Versum was presented in 2009 [16] . It features audiovisual objects arranged in virtual space in which the composer, performer, and spectator can float freely. It uses virtual space as a canvas for composition, which is explored by different paths so each run creates a meta-composition. A virtual reality musical instrument (VRMI), Carillon was presented in 2015 by Robert Hamilton and Chris Platz which allows users immersed in VR to control using gesture and motion [88] . In 2018, he presented Cortet, another VRMI which can mimic well instruments like piano, or percussion very well [89] . It allows multiple performers to collaborate in a shared virtual space. In 2019, Connexion was presented by Danowski et al., which embedded the idea of magical and mimetic, i.e. an instrument in a quasi-organic form that is visually responsive to the performer's actions [40, 41] . It surrounds the audience with an eight-channel sound system, making the audience immersed from every direction. Most recently, PatchXR has taken a major leap by providing a spatial equivalent of visual programming engines, which allows artists to build their audiovisual instruments using patches, i.e. visual programs which represent the flow of events in the music [100]. It allows artists to turn a place into a music studio on the fly, to compose and play music in a modular and sculptural workflow with the building blocks of sound at the spatial level (Figure 10a ).

(a) PatchXR, a VR musical instrument, turning a room into music studio on the fly [100].

(b) Singer Madison Beer performing in a studio, while audiences watching performance via her avatar in a virtual world [191] .

(c) 360°Egocentric view of the audience watching orchestra in the virtual space [168] . Music Concerts and Festivals in the Metaverse. Nowadays many VR platforms like VRChat, AltspaceVR allow artists to create their world and perform in the virtual environment together, and some of these platforms have millions of users around the world as a potential audience. Given the severe impact on the live-music performance industry by the coronavirus pandemic, the virtual world has emerged as an ideal stage for music concerts and festivals. VRChat platform has been used for the live concert with support from engineers from VRrOOm since 2020 [101] . In 2020, Sony organised a live concert of musician Madison Beer. As shown in Figure 10b , while she stood in a studio in her virtual reality suit, the audience received the appearance of a live concert [191] . Some VR platforms like the Fortnite game platform have also been used to the concert by playing back animation inside the gaming engine, instead of the musician performing live [185] . Minecraft, another VR gaming platform, has been used to organise music and film festivals [211] . One of the major advantages of organising concerts and orchestras in the virtual space is that it allows the audience an opportunity to know their inner workings. For example, during Philharmonia Orchestra conducted by Esa-Pekka Salonen (Figure 10c) , the audience was able to move between each instrumental group, go backstage, stand next to the conductor, or watch from the theatre seats, which is nearly impossible during such events in the real world due to maintenance and security costs [168] . Moreover, conducting concerts in the metaverse offers performers to bring a new perspective of decorating the stage environment as per the theme of their performance, as in Alice in Wonderland. 2020 Burning Man event was organised in the multiverse using the Altspace VR platform [146] . Not to mention, concerts in the metaverse will also provide easy accessibility to physically disabled audiences who often find it challenging to attend real-life concerts.

Though it has generated interest among the audience who are VR users, the technology is still in the nascent stage for organising a concert for a large audience. For example, VRChat allows only a maximum of 40 users in one room. So, even though multiple instances of concert rooms could be created to accommodate a large audience, the avatar of the performer could be present in only one room. Other challenges include audience jumping to restricted areas which can block the projection of the artist's avatar. The major challenge lies in how to simulate the realistic concert environment since the real concert tends to have lots of chaos in terms of the auditory environment, i.e. main music coupled with sounds from the live fans, and other background noises. One possible approach is to sample sound from different areas of a concert area which allows the sound to be activated in an even or uneven manner in six degrees of freedom within the sound field.

Musical Creativity for the Metaverse. While replacing the physical auditorium with a single website is one major key focus for the metaverse, another focus also lies in generating musical arts itself with help of machines that can assist music concerts in the metaverse. The field of Musical Metacreation (MuMe) especially focuses on this goal. Metacreation is the idea of endowing machines with creating behaviours that are deemed to be novel and creative by unbiased human judges [210] . According to Pasquier et al. [171] , the MuMe field is about developing software and systems that can autonomously (or interactively) recognise, learn, represent, complete, accompany, compose, or interpret music. As mentioned in the previous paragraphs, though concerts in the metaverse give the appearance of live performance, it is still far from simulating a real concert-like experience. Music metacreation can fill this missing gap, such as machines generating the background sound depending on the concert theme, concert hall settings, and audience crowd in order to stimulate the chaos/adrenaline rush feelings of a real concert.

Recently deep learning-based approaches have taken the central stage in human-machine collaboration for music generation. These approaches can be divided into two categories based on their input method, whether they use either note sequences or raw audio as their input. Magenta is an open-source music project from Google that uses a regular recurrent neural network (RNN) and two Long Short-Term Memory (LSTM) [181] . It can handle any Musical Instrument Digital Interface (MIDI) files. BachBot 21 is LSTM based approach that aims to generate and harmonize chorales indistinguishable from human-made chorales. It is considered one of the best efforts in handling polyphonic music as the algorithm can handle up to four voices. FlowMachines is a commercial project from Sony which uses Markov constraints as neural network techniques. It has generated the first AI pop-songs [144] . Wavenet 22 is another project from Google based on Convolutional Neural Networks (CNN) which aims to enhance text-to-speech applications by generating a more natural flow in vocal sound [165] . Magenta, BachBot, and FlowMachines use input in the form of note sequences, while Wavenet uses raw audio.

The previous sections primarily demonstrate the role of computational arts for creative creation, solely driven by computers and deep learning. The achievements of deep learning among the novel artworks shed light on the paradigm shift. One possibility is that artificial intelligence achieves a high level of automation in art creations and eventually replace human artists. Nevertheless, we should not neglect the possibility of having human users who interplay with the computerised artistic agents. Within the creator community in the metaverse [132] , it is very likely to see that human users with authoring systems work collaboratively with other AI-driven avatars or computer agents. Also, the metaverse can support enriched user interaction in immersive urban [130] . This envisions that multitudinous sensors capture users' physiological information and their movements.

With appropriate privacy management of such data and informed user consent, such captured data can be further converted into artistic representations.

(a) A human artist paints with a robotic arm.

(b) Body writing.

(c) Sharing the Senses by brain activities [126] . The following paragraphs will discuss some examples that involve collaborative roles of computational arts and other user-centric approaches. As for the collaboration between human and computerised artists, AI-assisted collaborators, by leveraging robot arms and sensors to achieve physical embodiment, can work with human artists. For instance, Artist Sougwen Chung 23 is passionate about collaborative art-making with robots ( Figure 11a ). Chung has designed and programmed about two dozen robots named Dougs, and they are guided by an AI system called recurrent neural networks. In order to make these Dougs her gestural expertise, she has uploaded her paintings online for over a decade. In addition, she has led AI-assisted painting performances on stages and in galleries before the pandemic. However, with the spread of the pandemic, Chung streams her robotic collaborations from her studio to replace live performances.

As revealed by the above example, AI-assisted creation can help metaverse users to collaboratively complete certain artwork creation in virtual 3D worlds. Apart from finding AI-driven partners for creative processes, smart computation devices can surprisingly utilise the internal status of human users [122, 187] , captured by body sensors, to create artworks. For instance, a wearable prototype named "Body Writing" (Figure 11b ), attached to a user's body and collects the user's biological statuses and emotions, connects the body with data through wearable devices. Based on the recognition of the user's sadness, fear, enthusiasm, and other emotions, the body sensors inform AI artists, as user-centric inspiration, for writing poems.

Furthermore, thanks to the advancement of machine learning that is capable of handling complicated signals, our brain activities being captured by Electroencephalography (EEG) electrodes, serving as brain-computer interfaces (BCI), can objectively mine, interpret, and classify social behaviours and emotion recognition. As recorded in a recent survey, numerous contemporary arts (N=61) have been completed by employing BCI from 1965 to 2018 [179] . Dutch artists Lancel and Maat initiated interdisciplinary research called "EEG KISS" [126] at the Art and Science Research Center of Tsinghua University in 2014 24 . With such sensing capability, we can leverage the "Symphony of Intimacy Data" as artwork. As shown in Figure 11c , the brain activity data of participants were captured by EEG smart wearables. When the two participants kissed and stroked each other's cheeks, the participants' emotions and the corresponding brain activity were converted into visualised data, as artistic representations, in real-time.

From the above cases, we can imagine that the internal status of metaverse users can bring some artistic effects to virtual 3D worlds. An imaginary example could be converting users' emotions or brain activities into the artistic aura to their virtual characters (i.e., their avatars). Another essential 23 https://www.washingtonpost.com/business/2020/11/05/ai-artificial-intelligence-art-sougwen-chung/ 24 https://zhuanlan.zhihu.com/p/94572785 source of user behaviours is body movements and gestures, knowing that our body gestures could express emotions and moods, or even formulate a story through art performance like dancing. Back to the date before the prevalent use of machine learning, real-time computation of Fibonacci sequence and the golden ratio was regarded as the standard of judging the dancer's movement [29] . However, such quantitative metrics are hard to apply in enriched virtual environments. Lately, optical sensors (e.g., depth and RGB cameras), supported by convolutional neural networks (CNNs), are commonly employed to capture the dancers' poses and movements [14] . By employing common techniques like dynamic time wrapping (DTW) [58] , the gestures and movements of the user (dancer) can be clustered as high-dimensional feature space and further converted into dance performance in virtual environments [8] , as shown in Figure 12a . In contrast, autonomous avatars can understand sound-motion mappings of human dancers, powered by deep learning, and hence mimic dancing movements with high levels of dance-likeness and emotional expressivity [205] . Such sound-motion mappings, enabled by the Long short-term memory (LSTM) autoencoder, can be extended to the governance of artistic movements of virtual characters driven by certain musical rhythms in the metaverse [196] . With Sparse-Temporal ReID Network, multiple dancers and their virtual postures can be re-constructed through image restorations [92] . In the metaverse, computer agents, represented by autonomous avatars [132] , can parse the user movements, and hence interplay with the human users' avatars. This can be further extended to the case of co-creation in the metaverse through dance performance. In a large display system of co-creative dancing named LuminAI [141] , human artists can perform dance improvisation (i.e., creative movement spontaneously). Then, LuminAI leverages unsupervised learning to learn about the user movements and formulate the human partner's gesture memory. As a result, a LuminAI avatar appears to move and dance interactively (Figure 12b ). It is worthy of pinpointing that the dance performance is not limited to the dancers' body parts. Other factors such as space utilisation and dancer arrangements (i.e., their relative positions) should further be considered [106] . Currently, dancing installation performance can be practically deployed at nightclubs and party gatherings. A prototype named 'canvas dance' can capture indoor activities of multiple dancers. The computers can detect multiple dancers' actions, and coordinate the dancers by giving visual cues in the form of blinking lights on a large display [83] . Apart from dance improvisation, similar sensor and installation setups can support other types of interactive dance, such as Salsa (a Latin dance), Ballet, Contemporary Dance, and Waltz [82, 161, 170] . Furthermore, the dancing art installations discussed above shed light on the possibility that avatars can understand the human users' movements, perhaps with a longitudinal observation in the metaverse, thus resulting in highly collaborative art performance across human users and autonomous avatars. However, several unexplored areas exist in technology-inspired dance performance, including connecting kinetics to the audiences, augmentation of expression for choreographers and dancers, aesthetic harmony of dancers as an aligned storyline, interactive build of the choreography, and integration between dancers (regardless of human or virtual characters) and interactive technologies/environments [78] .

In November 2021, Roblox has released their ambitious investment plan valued 10 million US dollar for metaverse education. In the initial stage, three STEAM (Science, Technology, Engineering, Art and Mathematics) subjects, namely robotics, science, and space exploration, will be the first focal points 25 . The metaverse environments can further extend to dance performance to reinforce the Art. Regarding the feasibility, the latest technique of one-shot learning allows the conversion of human dancers into animated dancers with only one image. This opens the possibility for metaverse users from various backgrounds (e.g., body types, ethnicity, age, and gender) to quickly appear as virtual dancers [69] . Also, a prior work provides personalised learning and multimodal recordings (e.g., annotated videos and audios) to demonstrate the possibility of web-based platforms for remote interactions between dance learners, practitioners and experts [53] . In addition, the CHROMATA platform is designed to enable the scene understanding of cultural heritage and then build a digital twin of cultural heritage. Accordingly, such a metaverse scene can allow dancers to experience immersive dancing on the performance stage in a virtual 3D theatre of ancient Greece [177] .

In the era of intelligent industrialisation, we foresee the highly intelligent robots or robotic arms would achieve high levels of automation in production lines and hence superior production efficiency and quality. According to a report released by the World Robotics 2021 Industrial Robots report 26 , the number of robots increases by 10% yearly, with the existing baseline of three million industrial robots or robotic arms currently operating in factories worldwide. Although Microsoft would like to pinpoint its metaverse with the workplace, the social robots, which may play a significant role in artistic creation and performance, are neglected. Although we mainly spot the usage of artistic robots appeared in the existing literature, human artists in physical environments can work collaboratively with artistic robots that either represent virtual avatars or AI-assisted agents from the metaverse [132] . In addition, artistic robots present a new experience (known as a physical embodiment) to the artists during the creative process, in terms of environments, personal experience and emotions, and potentially inspire human artists to reach an unexplored landscape of art performance and artworks in the virtual-physical blended realities.

In this section, we discuss diversified artistic robots designated for performance arts and artwork creation. First of all, musical robots can play traditional instruments or some adapted instruments. In addition to being able to perform the instructions given by the technicians, some robots have gained the ability to create new rhythms through machine learning algorithms. Noticeably, increasingly research effort is moving towards having robotic musicians play alongside human musicians. Therefore, in order to enrich the audience's experience of a live performance, the robot musician acquires additional interactive features from both audiences' reaction and performers. For instance, human musicians often move their bodies according to the tempo and intensity during the performance. Such captured motions allow the robots to be more in tune with the mood of human performers and audiences.

A robotic marimba player, namely Shimon [93] , are designed to serve the aforementioned functions. Second, wind instruments, such as flute, are sounded by the player blowing into (or over) the instrument's mouthpiece, causing the air to vibrate. The shape of the mouth opening and the direction of the air blown in will influence the sound. The same principle applies to robot performers once the (soft) mechanics of the robots are available. Figure 13a shows an example of a flute robot. Musical performance robots are equipped with artificial lips [190] and air-supplying devices [136] to improve their performance. Also, the robots can group and play music together. For instance, the conductor directs the musical performance in large group performances, such as orchestral or choral concerts. When musical robots join such events, they are able to recognise and respond to the conductor's gestures [39] . Alternatively, fuzzy logic algorithms enable the robot performers to react to music performed by human, to achieve the human-robot joint improvisation performance [198] (Figure 13b ). The integration of traditional dance art forms and robot technology gives dance art a new horizon and artistic meaning. With the increasing demands for robots to subtly convey emotions and gestures, robotics designers of dancing performance are pursuing the development of highly sensitive robots for complicated movements to enrich artistic performances. For example, enabling the robot to dance more naturally, such as changing its movement according to the user's real-time reaction, allows robots to express themselves through different dance movements. Therefore, more varied and characterful choreography becomes a crucial part. Dance robots can acquire new dance movements through machine learning or imitating movements from human dancers. For example, Nakaoka et al. [160] developed a bipedal robot that can imitate the movements of a human dancer of a Japanese folk dance. In addition, improvising dances allow dance robots to perform more human-like, including dance synchronised to music [22] . Moreover, the development of robots with more flexible mechanical frames [167] could allow robots to perform more complex motions since there are no joints in the robot structure like in a human body. Dancers can express their feeling through their bodies to the audience during dance performances [166] , giving robot dancers humanlike characters. In addition, artistic robots can dance interactively with other human artists [14] ( Figure 13c ). In particular, wearable technology like smart wristband that are equipped with IMU sensors (accelerator in particular) can detect the dancer motor movements and hence their skill levels of dancing movements [44] . The captured dancer movements enable the robots to coordinate with the dancer's movements. With the deepening of human-computer interaction research, it is obvious that a variety of mobile robots can dance with humans ubiquitously. Robots with other physical form factors, like spherical robots [200] , can fit different needs of human performers. We would like to highlight that the performance with artistic robots is still in its nascent period. Performers can explore various robots and the artistic effects to create novel robot performances.

As with other art forms, painters use graphics, composition and other aesthetic methods to express the concepts and meanings they wish to convey. There are many different types of painting, using media such as ink, acrylic and computer software. Robots can also draw in different styles [72, 143, 186, 216] . Jean Tinguely presents his Meta-Matics machines, which can create unique abstract art by generating movement, at the Kaplan Gallery 27 . To the best of our knowledge, the painting created by the Meta-Matics machines could be considered as the very first robotic drawing. Apart from digital painting, most of the drawing process requires the use of brushes, paints and paper. The painter controls the strength and speed of the brushes to create different thicknesses of lines. Therefore, there are many studies on brush control approaches [123] . For example, Yao et al. [216] studied the techniques of robot's brush control in Chinese painting, while Karimov et al. [110] presented a brushstroke rendering algorithm for ARTCYBE. Additionally, a painting robot, by Beltramello et al. [17] presented a palette knife technique.

Furthermore, the shade of colour can indicate the contrast of light and dark and the distance relationship. Thus, the robotic painter also needs to consider these factors to make their drawings more human-like. In the case of coloured realistic paintings or black and white paintings, artists often mix pigments to create various colours to enrich the detail of the paint. As a result, robot designers created colour mixing devices [109, 142] for their robots.

Recent technology advancements have demonstrated that some drawing robots own a more human-like appearance and are able to communicate with humans -e.g., robot artists Sophia [182] ( Figure 13d ) and Ai-Da [154] . Ai-Da is the world's first ultra-realistic artist created in February 2019. She draws with her robotic arm, AI algorithms and the camera in her eye. Moreover, Sophia is the world's first robot citizen, with her citizenship declaration in Saudi Arabia. In addition, she can communicate with people while painting, and recently sold her painting as NFT for $688,888 28 .

A robot can also be a theatre actor. Theatre robotic actors not only brings a new form of theatre performance but also can use to inspire and teach STEAM (Science, Technology, Engineering, Art, and Mathematics) education [102, 117] . Figure 14a shows five robots used in a Robot Opera. Education in countries worldwide has now started to provide STEAM education to students at a younger age and cultivate the next generation of computational arts in the era of the metaverse. Therefore, many schools started to offer related subjects, such as programming and robotics, and some even started offering from elementary school [102] . The advantages of using theatre robots for STEAM education are transforming unfamiliar content into familiar context and thus enhancing student engagement. Thus, getting more students interested in the art field through various STEAM activities. In addition to assisting education, theatre robots could stimulate more diverse research related to robots [81, 175] , such as human-robot interaction (HRI). For example, helping to analyse positive HRI using the sensory system [80] , and installing multiple robots into a facility using the Body-sharing multi-robot system [202] . The use of drones for entertainment purposes could use for performance besides aerial photography. For example, drone shows [1] are one type of drone arts that uses multiple unmanned aerial vehicles (also known as drones) to fly in the air for display. The drone show is usually held at night, and drones are equipped with LEDs for various light effects; Figure 14b shows a flying drone performance held in Shenzhen. Since drones are movable objects, they can be dressed up in costumes and performed with human performers 30 (see Figure 14c ), which provide a new form of performance for the audience. However, there are potential dangers in the use of drones if their design is defective. For instance, when the drone is flying, its propellers are operating at high speed, and it may cause injury if a person accidentally touches it during interaction with drones. Therefore, ensuring safety during indoor or outdoor drone shows is essential, and this requires a great number of technical engineering [68, 112] . The drone will follow a pre-programmed route and use GPS or radio frequencies to determine its position and height with respect to other drones to prevent any collisions between drones.

The emphasised blending of virtual-physical realities for the metaverse cyberspace leverages virtual cyberspaces and physical spatial as a canvas for vastly diversified artworks. Examples include paintings and drawing in VR [104] , creating AR overlays on top of physical environments [206] ( Figure 15a and 15b) , and decorating physical surface of heritage sites with AR and designing virtual buildings inside historic buildings [119] . Remarkably, Multiverse.pan 31 utilises light projections to create immersive environments on buildings and their interior structures (Figure 15d ). In addition, immersive environments can support and benefit art performances, e.g., offering visual cues when playing the piano [87] (Figure 15c ). It is important to note that numerous immersive art installations consider arts and data together. Though at first glance, data appears to be detached from the notion of artistic creation, the union of the two seemingly disconnected subjects has come into existence for a long time. Roughly 60 years ago, Jules Prown visualised his study in the history of American art with the use of punch cards [79] . However, data art as one of the fruitful products yielded from this union, extends beyond data visualisation. While the two strands both create visual entities that appeal to individuals' comprehension, they differ greatly in ends. Like the AR/VR projects mentioned above, immersive and light displays now serve as a conveyor of data arts that reshape our spatial environments. For instance, 'Weight of Data' [115] presents rice, sugar and salt as a light-display installation art showing raw crystal with a sense of data transformation (Figure 15e) . As countless examples under artistic contexts exist, we cannot report all of them due to limited space. Instead, we focus on a prominent feature of nurturing creativity in virtual and immersive environments. Recent studies have investigated virtual worlds as a facilitating tool supporting individuals to improve creativity. By leveraging the virtual worlds, it is possible to nurture both the creative process and artwork creation in the metaverse. It is important to note that this concept aligns with the second stage as described in the three-stage metaverse development, namely Digital Native [132] . Due to creativity being a complicated conception, we explained the use of 30 https://www.youtube.com/watch?v=6C8OJsHfmpI 31 https://www.signalfestival.com/en/history/2019/multiverse-pan/ virtual environments such as AR, VR, and eventually the metaverse, in five aspects based on creativity referred to generate ideas or solutions that are novel and suitable for particular goals [6] . According to the componential theory of creativity [6, 76] , four components can affect creativity: domain-relevant skills, creativity-relevant processes, task motivation, and the social environment. In addition, although the componential theory of creativity is not including the physical environment, the previous studies verified that it could influence creativity [6, 76, 156, 163, 197] . Those five elements are influenced by one another, affecting creativity, as shown in Figure 16a .

(a) Interaction between VR and creativity components.

(b) Three sketches of virtual creativity environments-1st column: the closing and opening environments. 2nd column: a virtual environment with/without specific 3D objects. 3rd column: topic-relevant virtual environment, source from [20, 21, 156] . Domain-related skills refer to the specific skills, knowledge, and capacity required to solve problems in a specific domain, [6] , such as the drawing skills for artists. Integrating VR into practice could develop domain-related skills [7] , such as the designing skill for designers [107, 114] . A study applied a VR application supporting students' garden designing and compared it to paper-based design [114] . There are many advantages of using a VR application for garden design. For example, a specific factor in garden designing is the time dimension, which requires the designers to consider the influence of time, such as the daily changes by sun's position, seasonal changes, and the plants' changes by years. That is easy to be simulated in a VR application and provides the opportunities for designers to observe their design in different conditions and reduce the time needed in the real-world [145] , which helps designers practice and develop their designing skills in a virtual environment rather than the real world. Moreover, other skills also can be developed in a VR application, for example, the skills in design reviewing [107] . That is fundamental to identify design errors, such as missing components of products, and misuse of materials. Furthermore, for other domains, such as psychomotor skills [5] , surgical training [148] and nurse training [35] , the researchers mentioned VR could simulate the real world to gain domain-relevant skills developing new ideas and solve problems, without incurring dangerous, expensive costs, and social problems.

Creativity-relevant processes, or called creativity-relevant skills, consist of personality characteristics and cognitive styles, such as independence, risk-taking, openness, work style and skills, and the ability to break out of perceptual in generating ideas. VR is mainly applied in creative methods during creativity-relevant processes, which influence individuals generating responses [6, 76, 77] . For example, integrating VR into brainstorming and adopting the different avatars, improved the creative performance by measuring the fluency, originality, uniqueness, and novelty [25, 27, 84, 85] . Specifically, the number of ideas (fluency) and the number of unique ideas (originality) could be increased by adopting an inventor avatar (such as a scientist) in a brainstorming task by applying 31 https://www.youtube.com/watch?v=aIllsTyxt3Y VR [84] . In addition, the avatars with the identification (i.e., social identification) could contribute to the group creative performance and outcomes [27, 85] . Moreover, adopting VR in creativity-relevant processes could avoid shortcomings and have more advantages than without VR [121, 176] . The shortcoming, such as saving the conversation or capturing the discussion contents, is easily solved by adopting a VR application [158] . For example, it is easy to undo an operation in VR application allowing the participants to practice trial and error in design processes not feasible in the real world, which supports creativity-relevant processes to improve creativity [114] .

Based on the componential theory of creativity, task motivation specifically refers to intrinsic task motivation that is the degree of personal enthusiasm for completing tasks or solving problems. For example, challenging or exciting tasks can stimulate self-challenge and interest and satisfy the individual [6] . Some researchers have suggested that the immersive environment created by VR could be a valuable way to increase personal interest [99, 116] , and learning motivation [4, 19, 137] . The researchers mentioned that immersive systems, including the metaverse, are interesting and inspire motivation and creativity [133] . Another study also emphasised that individuals worked hard to create a solution in an immersive environment, and they found it more fascinating, and dramatic than an ordinary physical environment, [28] . In addition, users are satisfied with the process of creation, and they believe that the virtual environment was accompanied by dramatic challenges that motivated them to perform better, thereby enhancing creativity [90] . Moreover, the VR game, for instance, through priming game [43] , can motivate an individual by delivering illustrations of the goals and achievements to spark the individual's creative performance and eventually gain a big pool of ideas. Furthermore, a study verified that playing VR games encourages individuals to engage in training and motivates them to exercise more often [116] . The studies verified that applying VR to complete a task or solve a problem or learning or training could stimulate individuals' motivations, capture their interests and make them engage in the processes.

The social environment, perhaps socialised virtual worlds, generally includes all extrinsic motivators, which might hinder or stimulate intrinsic motivation and creativity [6] . Some extrinsic motivators might be obstacles to creativity (e.g., excessive time pressure and demanding leaders), which is in stark contrast to a friendly social environment where free sharing of comments and similar motivations can stimulate creativity. For example, to participate in a task anonymously in a virtual environment, which supports an equal social environment. Participants can freely express their ideas and provide feedback, which encourages them more voluntarily to share their ideas, and more free communication and avoids several negative external motivations such as a strict hierarchy, thus improving creativity [60, 98, 121] . Other extrinsic motivators, such as collaborative exploration, co-modification, and mutual co-exploration, encouraged individuals to generate original solutions and inspired them to discover unexpected new solutions [94] . Moreover, VR could also be used to create a virtual environment (e.g., situated learning environment), contributing to enhancing learners' motivation [97, 153] . The participants expressed their feelings and provided their feedback in the interview after using a spherical video-based virtual reality supporting teaching, such as the presence and immersion, which encourage them to be more focused in learning and stimulate their learning motivation (i.e., extrinsic motivation) [97] .

The physical environment refers to the geographic area and factors around human beings [157] . It is worthwhile to mention that the metaverse will contain a huge number of creators or artists across different cultures and geographic areas. Some specific factors in physical environments to be verified could enhance creativity [49, 135, 197] . Therefore, the researchers simulated the physical environment and added specific factors by the use of augmenting such environments with virtual entities (i.e., mixed reality), evaluating the simulated environment that could be used to inspire creativity. Comparing the closing environment (without windows and closed roof), the opening environment with arched glass roofs (left column, Figure 16b ) enabled individuals to contribute more original, and practical ideas to teams [156] . Also, the virtual environment with specific 3D objects (e.g., specific trees and flowers), could influence individual cognition, thereby enhancing individuals' creativity [21] (the lower illustration in the middle column, Figure 16b) . Moreover, the prior study continued to investigate this topic by comparing the topic-relevant virtual environment (right column, Figure16b) and the specific virtual environment mentioned above. Their results showed that both virtual environments could improve creative performance because priming can manipulate individuals' unconscious cognition and behaviour to improve team performance. In particular, the topic relevant environment is a suitable method for implementing priming in VE, which can encourage personal design thinking, especially in ideation stages [20] .

Although the researchers put lots of effort into VR to enhance creativity, there is much room for further exploration. For example, lacking facial expression is still a neglected factor that negatively affects users' experience in the creativity-relevant processes. Adding other senses also a promising method, such as the smell of fresh air, enabled individuals to involve in the creative process peacefully and calmly, which influenced their cognitive, and positive emotions related to task motivation and the social environment enhancing creativity [76, 178] . The advent of the metaverse, perhaps with more intelligent and comprehensive sensing capability, can further open the opportunity to enrich the environments of virtual creativity.

The article surveys numerous recent examples of computational arts with obvious concatenation with the metaverse. We project that innumerable artworks will appear in the metaverse era, considering that our everyday objects in the physical reality would turn into an endless canvas. As such, artists will leverage various opportunities from novel computational devices, new materials and media, and mixing of virtual entities and physical environments, to express their idea, messages, and creativity, being conveyed by artworks. Instead of deepening the discussion of various artwork and creative processes, the grand challenge of metaverse creation primarily links to the design of the metaverse ecosystem and its governance. Some fundamental questions could be: how the metaverse environments encourage content creation by every metaverse user? Also, how to preserve the metaverse creation (e.g., digital heritage)? And, how to achieve a creator economy, like nowadays virtual art trading? In addition, how to ensure the privacy and safety of metaverse users, especially those who are vulnerable like underage content creators, and help artists understand the social impacts of their works? We consider such ecosystem issues are critical to accelerating the momentum of metaverse creation. Thus, we outline a research agenda to reflect several urgent matters regarding establishing a metaverse ecosystem for art and culture activities.

Democratising Computational Arts and Education. The year 2021 marks the twentieth anniversary of the Processing project, an open-source, Java-based programming framework initiated by Casey Reas and Ben Fry [62] . It is designed with the goal to enable artists and designers to more easily employ computer programming for visual media and creative output [180] . Over the years, Processing and other similar creative programming frameworks become vastly popular, with their user bases no longer limited to art and design practitioners. Not only have these frameworks made programming accessible to a wide range of fresh learners, those who have formal training in computer programming are also starting to experiment with these frameworks to create visual outputs and thereby engaging with art and design directly. This close interaction between the arts and computation gave rise to a new community that blends people from both backgrounds. These phenomena reflect an important trend in recent years, which is the democratisation of computational arts and education. Creative programming frameworks, with their open-source nature and ease of use, have made it possible for most people to start creating multimedia output in a relatively short time. This trend is not only seen in programming, but also in the field of CGI and games, exemplified by the open-source 3D content creation software Blender [61] and the professional-grade Unreal Engine that is mostly free and also has its source code openly published [70] . If this perceived trend continues, it is expected that they will bring even greater energy into the computational arts field, though the details of their impacts remained to be further studied.

Barriers of Building Virtual Scenes and Characters. In the field of CGI and video game, independent creators now have access to relatively easy-to-use and inexpensive or even free software, and are able to be self-taught through a large amount of tutorial resources on the internet. Many of the assistive processes and tools, such as motion capture, are also becoming more accessible, so that what used to require a team of dozens of people can now be done by a minimal group or even an individual, greatly stimulating the enthusiasm for creativity. At the same time, there are still issues that need to be addressed and further researched. First, the degree of integration between process and platform still needs to be improved, as content creation platforms often have their own focus areas, and therefore users often need to transfer and process content between multiple platforms in the metaverse, but this process considering the interoperability is sometimes quite cumbersome, not to mention the need for possible future integration to more interactive and networked presentation platforms. Second, software usability still can be improved, especially as most of the software is still designed for team-based, professional pipeline, and although it has become more user-friendly for individual users than before, some still has a steep learning curve, as we expect that every participant in the metaverse can enjoy the democratisation of content creation.

Digital Privacy and Safety for Artists in the Metaverse. In online communities like Transformative fandom, content designers share their works that have been derived from the original content. For example, members of the Harry Potter fandom can write their own stories on the on-screen relationship between Harry Potter and Hermione Granger. Since the work is creative, it often goes beyond the social norms which are generally accepted in real life. So, it becomes imperative to ensure privacy and safety for content creators. A Recent work by Dym et al. [50] reports that members of such communities often rely on implicit understanding from the fellow members for ensuring privacy and safety, and often such threat arises from new members who have recently joined the community, but do not conform to the implicit understanding. Given the recent report on harassment and cyberbullying in VR social network platforms [67] , new mechanisms will need to be developed which can tackle these issues, and especially protect the most vulnerable members, like underage artists. Another concern arises from the perspective of digital content moderation. Since some artists are uniquely positioned to challenge norms of society, whether it is social, political, or economic, it is important to equip them with tools that can help them to understand the potential influence of their contents on society at large. The most common strategies adopted by the society or the government to control such influence are either community moderation, or censorship. However, norms based control measures do not have rooms for including the ideas which are not widely accepted by the society, and which artists often tend to bring-forth through their arts.

Digital Art Trade Beyond Ownership Recognition. The emergence of NFTs kindled the vision of a new business model for digital arts. The capability to establish ownership, however, is not a sufficient condition to produce an effective framework that successfully imitates how traditional art markets works. One area worth addressing is leasing NFTs for personal and commercial purposes. In traditional art markets, a piece of artwork curated by an individual collector or an art gallery may be leased by others for time-limited display or other commercial events. By similar analogy, metaverse residents who hold precious virtual artworks may also face scenarios where they receive requests for leasing their collections. Currently, one notable step taken forward in addressing this issue is the reNFT project 32 , which proposes a rental protocol that enables NFTs owners to profit from leasing their collections to others. Nonetheless, one should recognise the need for collective efforts in enforcing a functioning digital rental market in the metaverse. Specifically, one needs to scrutinise key questions, including how disputes arising from the lease of NFTs should be settled, which may require institutional inputs from metaverse creators. In addition, a similar complexity arises from the different types of rights involved in transactions of the artworks from artists to buyers. The current technologies allow artists to specify what rights of a particular piece of artwork (e.g., the right to commercial use) are to be transferred when minting the token. Nevertheless, it is still important to ensure both artists and buyers understand the legal implications of the terms and act accordingly. Once again, the need for sufficient enforcement for terms specified in the NFT licenses to be honoured immediately becomes a salient issue. Another challenge arises from the fact that all digital goods (and hence digital arts) can be copied, which is detrimental for content designers in the metaverse. Even though a potential customer would be legally bound to the terms and conditions agreed with the content creators before leasing, we need to ensure that the customers cannot make a copy of the art on the technological level. Time-limited attestation of the digital art with the verified VR hardware of the customers could be one potential solution to address this concern. Also, limited research has been done on environmentally friendly NFTs. As it is undesirable to create a virtual world built upon technologies that backfire on our physical home, we encourage readers to consider how NFTs can be used in a sustainable manner.

Intangibility of Metaverse Creation. The current music technologies for the metaverse have many limitations. For example, artists and musicians are used to feeling physical pressure on their fingers when they play instruments in the real-life, and this pressure channels the music sensations throughout the body. Current VRMIs do not have such feedback mechanisms. One of the possible solutions could be to utilise a brain-computer interface that can channel the sensations to musicians. Another approach would be using Haptic Gloves, like the ones proposed by the Facebook Reality Lab [150] . Music concerts in the metaverse though offer an alternative to physical concerts and an opportunity to bring multiverse themes at the concert stage and hall, it needs to tackle a few issues, i.e. audience's avatars may wander to restricted space, the projection of the audience's avatar may overshadow the musician's avatar, as observed in the performance of Jean-Michel Jarre on VRChat platform in June 2020 [101] . As for supporting artists in creating (creative) music, deep learning-based approaches have mostly seen success in classic genres. Other genres are ripe for research. All in all, artists who have given performance in the virtual world are still not using VRMI, for instance, musical metacreation (Section 5), but physical instruments, even though VRMI could provide a richer choice to compose music in an immersive environment, which opens a door to develop training programs that can help artists trained with physical music instruments, adapt to virtual reality music instruments.

Turing Test for Computational Arts. Turing test provides a generic framework to measure the machine intelligence by asking a simple question: Can machines think? Extending this idea to the domain of computational arts, the question becomes Can machine think as well as be creative? Having a generic framework to answer this question is imperative, since we are likely to see novel computational arts in the future cyberspace of the metaverse which we would need to assess. The widely accepted answer considers two factors: 1) An unbiased human can not distinguish whether the given artwork (generated with aid of the machine, but this fact unknown to the judge) has been generated by the machine, or by the human, 2) The given machine-generated artwork has as much aesthetic value as one produced by a human artist. If both of these factors are affirmative, then such an artwork generated with machine-aid could be considered creative. However, this framework has several loopholes. Should the unbiased human judge be an ordinary Joe/Jane or an expert in the field? It has been reported that on several occasions the audience withdrew their previous valuation of the image and music after coming to know that they were computer-generated [23] . The crux lies in the perception that computer-generated art is not really art, and any beauty it may have is purely superficial. This perception arises because, on the most fundamental level, the computer has to follow human instructions to generate anything which diminishes the essence of "beauty" and "novelty". To tackle this issue, we would need to think to the next level: Can computers create? There is a subtle but significant difference between this question and the previously mentioned question of being creative because the baselines for answers to each of those two questions are distinctive. In the former question, the baseline of an unbiased human judge not being able to distinguish may suffice, however for the latter question, it wanders in the domain of computer being able to create something new without any human (or artist) instructions and we still need to come to an answer in the metaverse era.

Technological Infrastructure of Metaverse Creation. The maturity of input and output devices for AR/VR, mobile network, and edge and cloud infrastructures would facilitate the metaverse creation at scale [132, 222] . Generative art and other automatic generation methods of artworks involve demanding computational tasks. Due to insufficient computation resource and short battery life [129] , wearable and mobile computers, such as AR and VR headsets, strongly rely on task offloading [222] . The real-time generations of calligraphy and poems, especially a great number of creators that exist in the metaverse [132] , would be challenging. Furthermore, the latest development of input devices for user interactivity in virtual-physical blended worlds are tackling the bottlenecks of throughput rates [130] and investigating alternative modals for delivering sensation for uservirtuality interaction [18] . Creating artworks in such environments would require more advanced devices to convert user inputs into the desired expression, e.g., the acceleration of the user's arms can reflect a stroke width in mid-air drawing with a virtual brush. We expect technologists or interaction designers to employ varied sensors and multitudinous form factors to pursue subtle and accurate devices or tools for metaverse creation. Also, the limited field-of-view (FOV) on the display screen of the latest AR and VR headsets [131] could impact the user immersiveness and hence potentially the appreciation of visual arts. Such FOV constraints encourage designing immersive recommendation systems that offer precise yet highly personalised artistic content to the audiences [124] .

With the burgeoning virtual art trading, the computational arts's next decade will look radically different from what it is today, driven by the advent of the metaverse. Metaverse cyberspace will open numerous opportunities for creators and artists to reshape our virtual and physical environments in artistic and novel ways. By surveying the most recent works across virtual photography, cinematic simulation, calligraphy, poetry, musical metacreation, immersive arts and virtual creativity, as well as other artworks driven by user embodiment and robotics, we hope to have offered a broader discussion within the community of computational arts. We pinpointed the above vital topics and discussed the research agenda and several fundamental challenges to construct an artistic vision for the metaverse cyberspace. We call for interdisciplinary research requiring significant efforts from both technologists and artists to co-investigate the integrated facets of computational arts and technological infrastructures of such an artistic metaverse.

The drones of the olympic games #Tokyo2020

Art: What it Is and Why it Matters

The Definition of Art

The Effectiveness of a 3D-Virtual Reality Learning Environment (3D-VRLE) on the Omani Eighth Grade Students' Achievement and Motivation towards Physics Learning

Use of a low-cost portable 3d virtual reality gesture-mediated simulator for training and learning basic psychomotor skills in minimally invasive surgery: development and content validity study

Componential theory of creativity

Open Virtual Reality Project to Improve Students' Skills

Digital Dance Ethnography: Organizing Large Dance Collections

Theatre and its Double

Ed Atkins Interview: Something is Missing

Re) Building Proun #5A in the Metaverse

Azimuth to Cypher: The Transformation of a Tiny (Virtual) Cosmogony

Using the Game Engine in the Animation Production Process

Choreographic Pose Identification using Convolutional Neural Networks

President Obama Lands the First 3D Presidential Portrait

Tarik Barri -Versum Demo

Artistic robotic painting using the palette knife technique

Exploring Button Designs for Mid-Air Interaction in Virtual Reality: A Hexa-Metric Evaluation of Key Representations and Multi-Modal Cues

Virtual reality environments and haptic strategies to enhance implicit learning and motivation in robot-assisted training

Contextual priming and the design of 3D virtual environments to improve group ideation

Creative virtual environments: Effect of supraliminal priming on team brainstorming

Real-time dance generation to music for a legged robot

The Turing test and artistic creativity

3D Scanning Instruments

Enhancing collective creative design: an exploratory study on the influence of static and dynamic personas in a virtual environment

Are environmentally-friendly nfts possible?

Proteus vs. social identity effects on virtual brainstorming

Developing cognitive processes for improving inventive thinking in system development using a collaborative virtual reality system

Entertaining with Science, Educating with Dance

The valuation of emerging media arts in the age of digital reproduction. Electronic Visualisation and the Arts

The Economic Valuation of Digital Media Art

Computational Art Inspired by Genome Browsers

Search of Poetic Rhythm: Poetry Retrieval through Text and Metre

Understanding and Creating Art with AI: Review and Outlook

Effectiveness of virtual reality in nursing education: meta-analysis

Learning from Home: A Mixed-Methods Analysis of Live Streaming Based Remote Education Experience in Chinese Colleges during the COVID-19 Pandemic

2020. Minimum Viable Sentience

Global video game market value from 2020 to 2025

Musical robots: towards a natural joint performance

Sound of The Metaverse: A breif History of Virtual Reality Music Instruments and Virtual Music venues

An Iterative Polishing Framework Based on Quality Aware Masked Language Model for Chinese Poetry Generation

Sparking creativity: Improving electronic brainstorming with individual cognitive priming

Let's Dance: How to Build a User Model for Dance Students Using Wearable Technology

3D Virtual worlds and the metaverse: Current status and future possibilities

Silico: A Practice-Based Exploration of Computer Simulations in Art

Celebrity 2.0: Lil Miquela and the Rise of A Virtual Star System

Metaverse for Social Good: A University Campus Prototype

Work environments for employee creativity

Social Norm Vulnerability and Its Consequences for Privacy and Safety in an Online Community

How our NFT auction went

Days of seamless metaverse grow near as digital connectivity improves by leaps and bounds

A Web-Based System for Annotation of Dance Multimodal Recordings by Dance Practitioners and Experts

Searching for the Metaverse

Cryptokitties, cryptography, and copyright

Smart Collectibles: Unlocking The Value of Non -Fungible Tokens

Under the Skin of Foundation NFT Auctions

Dynamic Dance Warping: Using Dynamic Time Warping to Compare Dance Movement Performed under Different Conditions

ScrabbleGAN: Semi-Supervised Varying Length Handwritten Text Generation

Supporting Collaborative Creativity with Educational Visualizations in 3D Virtual Worlds

Blender Foundation

Processing Foundation

Art for Space

The Sentiment of Crypto Art

Enhancing art with information: the case of blockchain art

Crypto art: A decentralized view

Online harassment in virtual reality is 'way, way, way worse' -but can devs change that?

Performance of a networked human-drone team: command response and interaction effects

Single-Shot Freestyle Dance Reenactment

Unreal Engine 4 on GitHub

HiGAN: Handwriting Imitation Conditioned on Arbitrary-Length Texts and Disentangled Styles

Making robots draw a vivid portrait in two minutes

Metaverses as a Platform for Game Based Learning

Generating Topical Poetry

Literature review: Existing methods using VR to enhance creativity

VIRTUAL BRAINSTORMING AND CREATIVITY: AN ANALYSIS OF MEASURES, AVATARS, ENVIRONMENTS, INTERFACES, AND APPLICATIONS

Dance-Inspired Technology, Technology-Inspired Dance

The surprising partnership of art and data

Method and improvisation: Theatre arts performance techniques to further HRI in social and affective robots

Promoting Theatre Methodology for Expressive Robot Movement and Behavior

Canvas Dance: An Interactive Dance Visualization for Large-Group Interaction

Towards an Interactive Dance Visualization for Inspiring Coordination Between Dancers

Avatar-mediated creativity: When embodying inventors makes engineers more creative

Social identity cues to improve creativity and identification in face-to-face and virtual groups

Computational Media Intelligence: Human-Centered Machine Analysis of Media

AI and AR Based Interface for Piano Training

Live performance of "Carillon" by the Stanford Laptop Orchestra in the Bing Concert Hall

Coretet -Virtual Instruments for the 21st Century

Virtual World Construction and the Relationship to Creativity in Art Education

Beginning and End Games: A Parable in 3D

Learning from Dances: Pose-Invariant Re-Identification for Multi-Person Tracking

Shimon: an interactive improvisational robotic marimba player

Enablers and barriers of the multi-user virtual environment for exploratory creativity in architectural design collaboration

NFT sales volume surges to $2.5 bln in 2021 First Half

Real-Time Neural Style Transfer for Videos. 2017 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR

Learning to be a writer: A spherical video-based virtual reality approach to supporting descriptive article writing in high school Chinese courses

Effects of Avatar's Symbolic Gesture in Virtual Reality Brainstorming

Player experience of needs satisfaction (PENS) in an immersive virtual reality exercise platform describes motivation and enjoyment

Jean-Michel Jarre -Alone Together VR Concert

Robot Opera: A modularized afterschool program for STEAM education at local elementary school

Learning the Traditional Art of Chinese Calligraphy via Three-Dimensional Reconstruction and Assessment

HandPainter -3D Sketching in VR with Hand-based Physical Proxy

ReRites (& ReadingRites): Human + A.I. Poetry (& Participatory-Readings)

Requirements on Dance-Driven 3-D Camera Interaction: A Collaboration between Dance, Graphic Design and Computer Science

Application of a virtual reality educational game to improve design review skills

GANwriting: Content-Conditioned Generation of Styled Handwritten Word Images

Advanced tone rendition technique for a painting robot

Quality Management, Transport and Information Security, Information Technologies

Property rights in the Crypto age: NFTs and the auctioning of limited edition artwork

Realization of swarm formation flying and optimal trajectory generation for multi-drone performance show

Investigating the intention to purchase digital items in social networking communities: A customer value perspective

Using immersive virtual reality to support designing skills in vocational education

Weight of Data

Study on the motivational and physical effects of two VR Exergames

Robot-Theater Programs for Different Age Groups to Promote STEAM Education and Robotics Research

2020. 3d Rendering: Is It Photography?

Development of an Augmented Reality Tour Guide for a Cultural Heritage Site

Theophany: Multimodal Speech Augmentation in Instantaneous Privacy Channels

An interactive and multimodal virtual mind map for future workplace

MyoKey: Surface Electromyography and Inertial Motion Sensing-based Text Entry in AR

Application of brush footprint geometric model for realization of robotic Chinese calligraphy

A2W: Context-Aware Recommendation System for Mobile Augmented Reality Web Browser

Generating Homeric Poetry with Deep Neural Networks

EEG KISS: Shared Multi-modal, Multi Brain Computer Interface Experience, in Public Space

Build Your Own 3D Scanner: 3D Photography for Beginners

Deep-speare" crafted Shakespearean verse that few readers could distinguish from the real thing

UbiPoint: towards non-intrusive mid-air interaction for hardware constrained smart glasses

Towards Augmented Reality Driven Human-City Interaction

From seen to unseen: Designing keyboard-less interfaces for text entry on the constrained screen real estate of Augmented Reality headsets

All One Needs to Know about Metaverse: A Complete Survey on Technological Singularity, Virtual Ecosystem, and Research Agenda

Can Immersive Systems Improve Creativity Performance? An Exploratory Study

Embodied metaphors and creative "acts

Designing a musical robot for Chinese bamboo flute performance

Using virtual reality to facilitate learners' creative self-efficacy and intrinsic motivation in an EFL classroom. Education and Information Technologies

Generating Style-Specific Chinese Tang Poetry With a Simple Actor-Critic Model

A Multi-Modal Chinese Poetry Generation Model

Immersive Augmented Reality in Minutes with Meta 2

Visualizing Improvisation in LuminAI, an AI Partner for Co-Creative Dance

Robot artist for colorful picture painting with visual control system

Robot Artist Performs Cartoon Style Facial Portrait Painting

AI makes pop music

Motivational and cognitive benefits of training in immersive virtual reality based on multiple assessments

Burning Man in the Multiverse

The Language of New Media

Immersive virtual reality for surgical training: a systematic review

The Four Technologies Used In Virtual Production

Inside Reality Labs Research: Bringing Touch to the Virtual World

NFT market OpenSea hits $1.5 billion valuation

Visibility and digital art: Blockchain as an ownership layer on the

Applying situated learning in a virtual reality system to enhance learning motivation

Ai-Da (robot

Super Atari Poetry

Opening the mind: designing 3D virtual environments to enhance team creativity

Comprehensive biotechnology

SyncMeet: Virtual Work Environment for Collaborative Manga Creation

Mapping the NFT revolution: market trends, trade networks and visual features

Task model of lower body motion for a biped humanoid robot to imitate human dances

DANCING, Dance ANd Choreography: An Intelligent Nondeterministic Generator

Employment in U.S. motion picture and recording industries 2021

Soundscape quality in suburban green areas and city parks

A Survey on Metaverse: the State-of-the-art, Technologies, Applications, and Challenges

Wavenet: A generative model for raw audio

Towards the development of emotional dancing humanoid robots

A biologically inspired CPG-ZMP control system for the real-time balance of a single-legged belly dancing robot

2020. Philharmonia VR Sound Stage in Tokyo

An empirical investigation of virtual world projects and metaverse technology capabilities

Perceiving Artistic Expression: A Formal Exploration of Performance Art Salsa

An introduction to musical metacreation

Digital Art

Histories of The Digital Now

Methods for 3D Digitization of Cultural Heritage

Autonomous Robots as Actors in Robotics Theatre-Tribute to the Centenary of RUR

Digital Collaboration with a Whiteboard in Virtual Reality

Creating immersive experiences based on intangible cultural heritage

The impact of nature on creativity-A study among Danish creative professionals

Brain-Computer Interfaces in Contemporary Art: A State of the Art and Taxonomy

Processing: Programming for the Media Arts

Magenta studio: Augmenting creativity with deep learning in ableton live

Sophia (robot)

Distributed Metaverse: Creating Decentralized Blockchain-Based Model for Peer-to-Peer Sharing of Virtual Spaces for Mixed Reality Applications

Watch the Absorbing and Tedious Simulations of Ian Cheng

Picassnake: Robot Performance Art

Emerging ExG-based NUI Inputs in Extended Realities: A Bottom-up Survey

Compose Like Humans: Jointly Improving the Coherence and Novelty for Modern Chinese Poetry Generation

Website

Musical skills of the waseda flutist robot WF-4RIV

Making Madison Beer's Immersive Reality Concert Experience

Ikea Catalog Will Be 25 Percent 3d Renders By Next Year

A syncretic approach to artistic research in mixed reality data transfer

Constructing Situated Learning Platform for Japanese Language and Culture in 3D Metaverse

Dance with Melody: An LSTM-Autoencoder Approach to Music-Oriented Dance Synthesis

Virtual reality and the enhancement of creativity and innovation: Under recognized potential among converging technologies?

Human-Robot Artistic Co-Creation: a Study in Improvised Robot Dance

A Dance Performance Environment in which Performers Dance with Multiple Robotic Balls

The Art Market 2021

Body-Sharing Multi-Robot System in Robot Theater towards Social Implementation

Nintendo's net sales from fiscal

3d Scanning and Replication for Museum and Cultural Heritage Applications

Learning Embodied Sound-Motion Mappings: Evaluating AI-Generated Dance Improvisation

LightPaintAR: Assist Light Painting Photography with Augmented Reality

Rebecca Allen on Kraftwerk, Video Games and Artificial Life

ZiGAN: Fine-grained Chinese Calligraphy Font Generation via a Few-shot Style Transfer Approach

Art and blockchain: A primer, history, and taxonomy of blockchain use cases in the arts

Metacreation: art and artificial life

Best virtual festivals 2020: from Lost Horizon to Wireless Connect

Putting the CGI in IKEA: How V-Ray Helps Visualize Perfect Homes

Image to Modern Chinese Poetry Creation via a Constrained Topic-Aware Model

Real-time Localized Photorealistic Video Style Transfer

Stylistic Chinese Poetry Generation via Unsupervised Style Disentanglement

Painting brush control techniques in Chinese painting robot

Rhyming Knowledge-Aware Deep Neural Network for Chinese Poetry Generation

Learning to Compose Stylistic Calligraphy Artwork with Emotions

JokerGAN: Memory-Efficient Model for Handwritten Text Generation with Text Line Awareness

Digital art as 'monetised graphics': Enforcing intellectual property on the blockchain

Sinofuturism and Its Uses: Contemporary Art and Diasporic Desire. Verge: Studies in Global Asias

EdgeXAR: A 6-DoF Camera Multi-target Interaction Framework for MAR with Userfriendly Latency Compensation

Deep imitator: Handwriting calligraphy imitation via deep attention networks