key: cord-0479857-lqrk7lny
authors: Meuschke, Monique; Garrison, Laura; Smit, Noeska; Bruckner, Stefan; Lawonn, Kai; Preim, Bernhard
title: Towards Narrative Medical Visualization
date: 2021-08-11
journal: nan
DOI: nan
sha: 83b442725c36325ca4a0cc84efd2c34b2aa7fa0e
doc_id: 479857
cord_uid: lqrk7lny

Narrative visualization aims to communicate scientific results to a general audience and garners significant attention in various applications. Merging exploratory and explanatory visualization could effectively support a non-expert understanding of scientific processes. Medical research results, e.g., mechanisms of the healthy human body, explanations of pathological processes, or avoidable risk factors for diseases, are also interesting to a general audience that includes patients and their relatives. This paper discusses how narrative techniques can be applied to medical visualization to tell data-driven stories about diseases. We address the general public comprising people interested in medicine without specific medical background knowledge. We derived a general template for the narrative medical visualization of diseases. Applying this template to three diseases selected to span bone, vascular, and organ systems, we discuss how narrative techniques can support visual communication and facilitate understanding of medical data. Other scientists can adapt our proposed template to inform an audience on other diseases. With our work, we show the potential of narrative medical visualization and conclude with a comprehensive research agenda.

Medical visualization research to date has focused primarily on supporting medical experts (radiologists, pathologists, surgeons in diagnosis and treatment and-to a lesser extent-to medical students, in particularly for anatomy education. Medical information and research, however, are also interesting to non-experts, i.e., a general audience that comprises patients and their relatives along with those with an interest in science. Interactive medical visualization aiming at this type of audience requires different design approaches with easy to understand representations [BKV*20] than in systems such as radiology workstations that are aimed at experts. Narrative visualization combines storytelling techniques with interactive graphics to appeal to a general audience [SH10] . It aims to present the data in a traceable progression that is memorable and easier to understand [Fig14b] . There are two types of storytelling: synchronous and asynchronous storytelling [LRIC15] . In synchronous storytelling, the narrator is in direct contact with the audience, e.g., live presentations, whereas asynchronous stories do not require direct audience contact. These stories take the form of recorded videos, static graphics, or visually guided tours through complex processes with interactive visualizations.

Ynnerman et al. [YLT18] coined the term exploranation for merging exploratory visualizations that are traditionally made for experts with explanatory visualization techniques. While this supports visual knowledge acquisition for non-experts, it requires more guidance and automatically-generated content. For medical imag-ing data this may comprise labeled visualizations and animations highlighting relevant anatomical structures, e.g., vessel branching around an associated pathological structure. Visualizations of population-based data that indicate the most frequent tumor locations or metastasis pathways may also interesting to the public. Moreover, visualizations of health survey data demonstrating avoidable lifestyle-related risk factors for diseases can motivate the public to adopt healthier lifestyles.

While "scientific outreach" is already an essential topic for the visualization of astronomy data [BAC*19] , climate data [Böt20] , and cell biology data [KIK*21] , the same has not been true for interactive medical visualization research. Exceptions include epidemiological data, e.g., the COVID-19 Dashboard by Johns Hopkins University which supports map-based visualization, a selection of interesting countries, and time-based visualization of cases and fatalities. Early limited authoring tools were developed for generating interactive medical stories based on volume data [Woh06; WH07] . However, medical data also includes other data types, e.g., clinical images, 3D models, and flow data.

Several techniques for visualizing medical imaging data lend themselves well to narrative medical visualization storytelling principles with limited freedom for exploration. These include clipping planes which are automatically moved, cutaways or automated ghosted views based on structure selection, and automatically generated animated transitions. However, concept-driven content, e.g., submitted to Eurographics Workshop on Visual Computing for Biology and Medicine (2021) arXiv:2108.05462v1 [cs.CY] 11 Aug 2021 informative infographics, may be highly valuable to engage general audiences in scientific communication [RKO*20] .

Scope of this Paper. In this work, we discuss the potential of including interactive exploration of medical data in narrative visualization for a general audience, i.e., members of the general public who are interested in understanding diseases and their treatment but lack detailed medical knowledge or familiarity with scientific visualizations. We further identify three general public subgroups: Patients with a direct link to a specific disease, patient relatives, and people interested in medicine, see Figure 1 . Following an asynchronous storytelling method, we show how to leverage narrative techniques to present medical data in a way that is both compelling and understandable. Our proof-of-concept focuses on the suitability and arrangement of narrative techniques to tell stories about three common diseases that are related to three important structures of the human body: organs, vessels, and bones. Our inspiration for these disease stories draws in part from health websites such as WebMD and UpToDate. Similar to other works dealing with narrative scientific visualization, we choose touch screen as a medium such that the user can interact with the data during the story.

In narrative visualization, stories can be mainly data-driven or concept-driven. We follow the suggestions by Segel and Heer [SH10] that data should enrich the story while memorable visuals and interesting storytelling are the main components of the story. Our key contributions are the following:

• We provide an overview of existing work in narrative visualization and based on an analysis of a corpus of 30 medical stories propose a template to structure medical visualization stories. • We present three proof-of-concept medical stories that are enriched with interactive medical data visualization components to explain information around selected example diseases. • We identify promising areas for future research in narrative medical visualization.

Organization. Section 2 summarizes general narrative techniques based on seminal works in this field. Then, Section 3 gives a brief insight which other scientific visualization areas have used narrative techniques and how. Here, we also describe the associated transition from scientific visualization designed for experts to scientific visualizations for the general public, as well as challenges which arise in this process. Section 4 then describes the core of our paper. Based on the summary in Section 2, we show how narrative techniques can be applied to medical data to generate stories for the general public. We then discuss various aspects of our conceptualized medical stories in Section 5 and identify a research agenda that highlights promising aspects for future work in medical narrative visualization in Section 6. The paper is concluded in Section 7.

A visual data story is composed of a series of specific facts, called story pieces, that are supported by data [LRIC15] . These story pieces are visualized to convey important messages to the audience. Visualizations are enriched with story elements such as labels, arrows, links, and textual explanations to clearly emphasize these messages and avoid ambiguity. The story pieces should be arranged into scenes on the basis of a meaningful genre and design pattern to support the author's communication goal. General goals include to inform or entertain the audience. In the following, we summarize existing techniques to generate and transition between scenes. In addition, we summarize genres and design patterns with suggestions for their use in medical visualization.

Segel and Heer [SH10] derive general design elements of narrative visualizations and examine the range of user guidance and interaction. Hullman and Diakopoulos [HD11] build on this work to analyze 51 narrative visualizations, examining the rhetorical devices used. Stolper et al. [SLRS16] extend this summary by novel datadriven storytelling techniques. Based on these ground-breaking works in the field of narrative visualization, we summarize existing story elements, how to connect them to form scenes, how to transition between scenes and how to construct a scene path.

Visualizations are best complemented by other means of communication and highlighting techniques need to guide the user through a story [KM13] .

Text narration. Text is the simplest way to explain data. Long-form texts can be used to explain key points in detail and to introduce or summarize a topic. Headlines or captions can serve to draw attention to a story. Tooltips can provide details when a user hovers their cursor over an element [Fig14b] . Text can also be used in the form of annotations or labels to designate important structures.

Audio narration can be used to enhance visualizations [SH10] . This allows the viewer to focus more on the visuals, since the narrative is temporally linked to the visual elements.

Moreover, graphical properties can be used to draw the reader's attention. Elements can be highlighted using wrapped shapes, specific colors or techniques such as motion or close ups [SH10] .

To understand the explanatory nature of the interplay between story elements, connections must be made between them. Stolper et al. [SLRS16] found three basic types to connect story elements.

Interaction is an efficient way to connect story elements. Interactivity refers to the different ways a user can manipulate the visualization, e.g., filtering, hovering, zooming, rotating, and translating and also to how the user learns these methods (explicit instruction, tacit tutorial, initial configuration) [SH10; SLRS16] .

submitted to Eurographics Workshop on Visual Computing for Biology and Medicine (2021) Figure 2 : General genres of narrative visualization according to Segel and Heer [SH10] .

The level of interaction ranges from passive narration, where no interaction is provided, to free exploration, where the user has no interaction constraints [WH07] . Passive narration can be interrupted and the user can temporarily take control and change the presentation, e.g., by using dynamic queries to change the visual style of an object. Afterwards, the passive narration continues.

Color is another option to link story elements. Consistent colors should be used to represent objects or attributes that appear in multiple visualizations [SLRS16] . Color can also be used to connect text and visualizations by assigning text the same color as the associated visualized objects. However, the choice of color schemes and the design of the color map play a crucial role. Crameri et al. [CSH20] presented guidelines to design color charts for scientific data, including perception effects to be considered.

Animations can also be used to link objects that help users relate complex processes in an understandable way. Care must be taken to ensure that the user does not lose the focus while context information is needed for orientation. Therefore, smooth transitions between different camera positions are required, where focus objects should be visually emphasized.

Moving within and between visual scenes without disorienting the user is a fundamental aspect of storytelling. Segel and Heer [SH10] identified six types of transitions. One way is to keep the object change between scenes to a minimum, maintaining object continuity. The number and style of objects should not be fundamentally changed between two cuts. Related to this is the concept of familiar objects, which states that commonly used symbols should be used to represent facts. Another category involves meaningful movement of the virtual camera. The view angle of the camera should change between two scenes or when moving within a scene, but not so much that completely different views are created. Also, strong changes in the camera movement speed between adjacent scenes should be avoided. Continuity editing is an established technique from the film industry which creates the impression that the story was shot in one piece without cuts. Another option is to use animated transitions. Based on morphological transformations, objects of one scene can be changed into objects of another scene.

Data-driven stories are usually characterized by an author-specified order. The story is thus given a structure that is supported by frequent navigation aids. In addition to the specification of a strict path (linear story), there is the possibility to provide the user with several paths to choose from (user-directed story) [SH10] .

Commonly used techniques to navigate through a story are next/previous buttons and scrolling. Flowchart arrows can help to convey the intended narrative structure of the story. To navigate to a specific location, menu selections or interactive maps can be provided. To show the user where s/he is in the story section header buttons, breadcrumbs in the form of points, and timelines in the form of progress bars or checklists are often used.

To communicate the story in an understandable way, consideration must be given to how story elements are arranged and combined. Segel and Heer [SH10] have defined seven genres: magazine style, annotated chart, partitioned poster, flow chart, comic strip, slide show, and film/video/animation, as depicted in Figure 2 . These genres differ in the number of scenes shown, and the arrangement of story elements within a scene. The choice of genre depends on the data complexity, as well as the intended audience and medium.

For narrative medical visualization, magazine styles, where a 2D image is embedded in text, could be adapted to integrate 3D models, with the text around explaining visible structures. In contrast, flow charts can be used to show medical processes such as disease treatment in an abstract way. Annotated charts can be used to present statistical information, e.g., the prognosis as a function of the selected therapy. The combination of images and diagrams in a partitioned poster is well suited to provide overviews or summaries of medical explanations. Slide shows are commonly used in business presentations. For the application to medical data, the user's attention should be kept by interactive components, where the s/he is encouraged to interactively explore the data. Comic strips consist of highly abstracted illustrations that contain only brief annotations. An interesting scenario would be the cartoon-style illustration of medical aspects for children. Videos, and animations would be well suited to support the exploration of 3D medical data. Optimal views on surfaces, such as vessels and organs, could show structures of interest, e.g., the resection of a tumor.

Depending on author intent and the audience, a story can be told in different ways. Bach et al. [BSB*18] described eighteen narrative design patterns that can be used individually or in combination to tell data stories. Each pattern has a specific purpose, with five overarching groups. There are argumentative, structuring, framing, emotional, and engaging patterns.

Argumentative patterns include comparisons, concretizations, and repetitions, to present, support, reinforce, contradict, or discuss a particular statement. They can be used, for example, to compare submitted to Eurographics Workshop on Visual Computing for Biology and Medicine (2021) treatment options, to present information that users should remember (e.g., preventable risk factors), or to present the benefits of protective measures, such as vaccinations. Structuring patterns include concepts such as revealing, slowing down, and speeding up. Framing patterns determine how the story content is perceived through techniques such as creating familiar settings, making guesses, defamiliarization, breaking conventions, hiding data, and using physical metaphors. Structuring and framing patterns are important to present medical data, which usually contains different data types, such as volume data, 3D models, quantitative values and qualitative flow information. To communicate this data to general audiences, it must be simplified, details must be omitted, and interesting aspects, e.g., in statistical diagrams, should be revealed. Slowing down, and speeding up could, e.g., be used to show blood flow animations. In time ranges where interesting flow occurs, the animation is slowed down and in less interesting ranges the animation is sped up [KPV*14] . Emotional patterns such as directly assessing the audience and presenting individual stories are designed to help understand and share important feelings in the story. To engage the user into the story, techniques such as rhetorical questioning, call to action, and interactive exploration can be used.

Numerous works combined narrative techniques and information visualization [TRB*18; GP01] . In contrast, there is little research on combining scientific visualization with narratives [MLF*11]. In this section, we summarize the transition from expert-driven visualizations of scientific data, i.e., spatio-temporal data, to non-expert visual representation of these data. We also provide insights into the challenges that arise during this transfer, especially for medical data. Finally, we present selected scientific applications outside of medicine, where narrative techniques have already been used.

Traditionally, visualizations were used by experts to gain detailed insights into complex data. Experts have a deep background knowledge of the respective domain and are able to evaluate and interact with complex visualizations.

In contrast, a general audience includes people with varying levels of expertise who differ in terms of age and cultural backgrounds [BKV*20] . Bringing scientific results to a general audience is challenging, as it [...] "is quite a different matter to compel attention and understanding in a diverse, hurried, skeptical population of readers than to communicate with an eager, familiar group of associates" [DiB90] . Therefore, the purpose of the visualization should be clearly defined in the context of the target audience in order to fulfill the intended communication goals [BKV*20] .

Results from cognitive science show that embedding data in a narrative makes it more exciting and memorable [MLF*11]. For this purpose, complex scientific results need to be reduced, summarized and generalized by means of simplified and understandable visualizations. Compromises have to be made in terms of accuracy and completeness, since showing too many details can make it difficult to convey a clear message [BKV*20] .

To create a narrative visualization, the target audience must be defined as precisely as possible [BKV*20] . The background knowledge and the goals of the audience are decisive for the design, the level of interaction allowed and how strongly the audience is guided through the story. With regard to medical data, different audience groups such as scientists, students, patients, health care providers, or policymakers are conceivable, as shown in Figure 1 .

Several challenges need to be considered when designing narrative visualizations for rich scientific data [BAC*19; YLB20] . We summarize the main challenges and their relation to medical data.

Varying Spatial-Temporal Scales. In scientific data, the spatial and temporal scales of objects can vary greatly [BAC*19] . Navigation and interaction aids are needed that identify points of interest both spatially and temporally. In medical data, the sizes of structures can vary greatly. For example, organs, such as the liver, are several centimeters in diameter, while embedded structures, such as vessels or cells, are many times smaller. Similarly, time scales can range from hours that a treatment needs to years in long-term follow-up of diseases.

Varying Data Sources. Another problem are different types of data coming from different sources [BAC*19] . Medical data can include radiological and histological image data, numerical values, and statistical information, which can be acquired with different devices, e.g., different scanners. Moreover, biomedical simulation data may be relevant.

Data Access Issues. Another challenge that is particularly relevant to medical data is making data available to the general public. From an ethical point of view, mere anonymization of data is not sufficient for their use in public scenarios. One solution to this could be to use data derived from data donors.

Interaction and Navigation. The exploration of medical data by the general public requires to reduce complexity in terms of interaction and navigation compared to systems for experts [YLB20] . Otherwise, users can lose their desire to use the visualizations. The design of the user interface should be tailored to the communication goal of the story without noticeably restricting the exploration.

Occlusion management. In 3D scenes, special attention must be paid to resolve object occlusion. Virtual X-ray approaches and volumetric probes to adapt the opacity of occluding objects either automatically or interactively [ET08] are suitable for narrative medical visualization. Whenever interesting objects are occluded in medical data, often smart visibility techniques, such as ghosted views and cut away techniques, are applied [LSBP18] .

To not overwhelm people with visual exploration opportunities, they should be guided through the story [YLB20] . In 3D medical visualization, this can be realized through automatic views, limited rotation capabilities, or predefined parameter settings. The user should always know where s/he is in 3D space.

Flexibility and Performance. Due to different data types and many possible scenarios, a system to interactively explore medical data should be flexible regarding the integration of new interactions submitted to Eurographics Workshop on Visual Computing for Biology and Medicine (2021) and rendering styles. In addition, robustness is important to make software available to the general public.

Typical places where the general public comes into contact with scientific data are museums, planetariums, exhibitions, and science centers. Ma Recently, Ynnerman et al. [YLB20] summarized how storytelling is used in the Norrköping Visualization Center C. In addition to dome projections and VR setups, users can explore volume data using multi-touch displays. These data comprise full-body CT scans, which are visualized by direct volume rendering (DVR). From a pre-defined image gallery, the visitors can select a transfer function, which should be applied to the data. Visitors can interact directly with the visualizations, they can perform single and multi touch gestures, e.g, rotate the volume or cut through it, but they cannot select objects very precisely. Similar setups are used to explore a virtual human mummy [YRA*16] and biological structures that would not be visible to the naked eye [HSFT18] . However, besides pre-defined transfer functions, textual descriptions, and videos no further guidance through the complex data is provided.

Narrative techniques have also been used to communicate potential future climate changes to the general public based on simulated data [BKV*20] . In order for the user to draw conclusions, various visualization aspects must be taken into account. The choice of appropriate color scales is important to draw the user's attention. Furthermore, combinations of visualization techniques, such as color and contour lines to show correlations, must be carefully explained, e.g., by audio guidance.

In this section, we describe how narrative techniques can enrich medical visualization so that users are able to easily understand, absorb, and interact with the data. Our intended communication goal is to inform people interested in medicine about a disease. To demonstrate the potential of narrative medical visualization, we first derive a template comprising potential stages of a story about disease data, as detailed in Section 4.1. Next, in Section 4.2, we define an example scenario where the target audience comes into contact with medical data. Then, we select three common diseases for story generation, discussed in Section 4.3, followed by explaining the story preparation including data preprocessing and selection of an authoring tool in Section 4.4. Based on the defined template and selected diseases, finally the medical stories are designed and presented in Section 4.5.

Many university hospitals, scientific institutes or online encyclopedias put freely accessible blogs online to inform a general audience about the development, diagnosis and treatment of various diseases. We analyzed a total of 30 blogs for three selected diseases: Liver cancer [0]-[0], brain aneurysm [0]- [0] , and pelvic fracture [0]- [0] according to their basic structure, since we have the same communication goal and want to address the same audience.

The basic structure of these blogs is very similar. First, a short and understandable definition of the disease is provided, and statistical aspects such as the annual incidence and age-related distribution between men and women are described. Next, an anatomical overview shows the location and function of the structures affected by the disease. This provides the baseline for understanding what is normal before introducing the disease itself. Using the example of liver cancer, schematic sketches are used to explain where the liver is located, its function, and important nearby structures.

Subsequently, typical symptoms are explained usually as textual enumeration. Afterwards, the diagnosis is explained. This comprises frequently used examination methods, e.g., MRI, as well as their sequence and reliability in order to make a diagnosis. The procedure of each diagnostic method is briefly summarized and associated inconveniences for the patient are explained.

The diagnosis is typically followed by an overview of possible treatment options. Therapeutic procedures are summarized including treatment risks and the associated chance of cure is estimated. Typically, 5-year prognoses are provided.

Finally, disease prevention is explained, where risk factors for the development are summarized. A distinction is usually made between preventable and congenital/genetic risk factors. This concluding consideration of risk factors serves as an appeal and clarification that one's own behavioral patterns can have a strong influence on the development of life-threatening disease. The reader should be sensitized to think about their own habits and to adapt their lifestyle in a positive way.

Based on this analysis, we derived a sequence of seven stages forming a template, as shown in Figure 3 , that can be used as a basic pattern for applying narrative techniques to disease data.

People interested in medicine come typically into contact with medical data either in museums or science centers or through Internet research on their home computers, where no specific doctor patient connection exists. Similar to the Visualization Center in Norrköping [YLB20] , asynchronous storytelling based on a touch display could be used to interactively inform about diseases. While users at home would probably be more likely to use a tablet or their phone, larger interactive displays could be used in a science center or museum. We show how narrative techniques can help medically interested people to inform themselves about diseases at home or in a science center. This is an asynchronous scenario where the user interactively explores the data on their own. The goal is to give a general overview of a disease, not focusing on anatomical variations or severity of a disease.

For selecting example diseases, we oriented ourselves to the basic structures of the human body, which are visible in radiological image data: organs, vessels, and bones. Below, we outline our motivation for selecting three specific disease examples.

Liver Cancer. Regarding organ diseases, we selected liver cancer. The number of new cases of liver cancer has doubled in the last 35 years [LJY*19]. Accordingly, the interest in learning more about this disease on the part of the general public is likely increasing. In Germany, approximately 8790 people (6160 men, 2630 women) are newly diagnosed with liver cancer each year. The average age of onset is 69.9 years for men and 72.1 years for women. The increase in annual new cases is associated with an increasing number of patients with liver cirrhosis, the high rate of new hepatitis B infections, and increasingly frequent obesity.

Brain Aneurysms. For vessel diseases, we selected brain aneurysms, which are localized dilations of the brain vessels. In addition to older people, younger people are also frequently affected, making this condition of interest to the general public. About 3-5 % of all people likely have a brain aneurysm [BRR*17]. In most cases, these aneurysms are found by chance and remain asymptomatic. Brain aneurysms are more common in people over the age of 40, where women are affected more often than men by a ratio of 5:3. Their greatest danger is that the vessel wall ruptures, which leads to internal bleeding. The low incidence of rupture suggests that 80 % to 85 % of all brain aneurysms will never rupture [SMC*09] . Due to the low rupture rate and the existing treatment risks for the patient, physicians must limit treatment to high-risk patients.

Pelvic Fracture. For bone-related diseases, we used a data set showing a pelvic fracture, which is a break in any of the bones that form the ring of bones at hip-level [GFJ*18] . Severe cases show multiple fractures and/or an unstable fracture. This can occur as a result of high-energy trauma, e.g., car accident (20-22 %), or in frail or older patients from minor trauma, such as a fall (5-30 %). Highenergy trauma-related pelvic fractures often come with accompanying injuries that require immediate treatment. Pelvic fractures represent 3 % of skeletal injuries, with 5-16 % mortality (unstable pelvic fractures 8 %). All age groups can be affected, but trauma has a higher incidence in young males, while in older populations is more associated with women.

For medical story design, some data preprocessing was necessary, which we detail in Section 4.4.1. Essentially, this involves preparation of radiological image data. We also had to choose an authoring tool to create scenes and define transitions between them, as discussed in Section 4.4.2.

For each story, an anonymous patient data set was used, comprising different radiological data types, such as ultrasound, MRI, and CT images. As this data was anonymized, we had no access to patientrelated meta information. Therefore, any introductory patient information at the beginning of each story is fictitious. In the following, we shortly describe our data sets and necessary preprocessing steps.

Liver Cancer. We used a data set of a patient with a stage 1 liver carcinoma provided by our clinical partners as sample data. In this data, small and medium-sized tumors without lymph node involvement and metastases were diagnosed in the liver based on ultrasound and CT Angiography (CTA). In addition, the liver showed no cirrhotic changes. Due to multiple tumors, surgical removal was not an option. Instead, the tumors should be treated with ablation. Based on the CTA data, the liver as well as the tumors were segmented using HepaVision [BSL*02]. Moreover, other surrounding structures such as the heart and ribs were segmented and transformed to 3D surfaces.

Brain Aneurysms. For brain aneurysms, we used a data set from Berg et al. [BRB*15] , where a brain aneurysm was incidentally found during CTA. The vasculature was segmented using the pipeline presented by Mönch et al. [MNP11] and converted into a volume grid. Computational fluid dynamics (CFD) simulations were used to calculate the blood flow behavior. Finally, particles are traced in the resulting vector field to depict the blood flow. All simulation details can be found in the work by Berg Pelvic Fracture. We obtained a CT dataset of a woman with a pelvic fracture including pre-operative and post-operative scans. We performed direct volume rendering (DVR) in 3D Slicer [KPV14] in order to generate videos and images to support our story. In addition, we provide interactive 3D scenes based on the Virtual Surgical Pelvis (VSP) model [SLK*16] , which is in use as a virtual educational tool to teach pelvic anatomy [SHK*16]. The VSP consists of anatomical surface models based on expert segmentation of a cryosection data set. Selected VSP structures were embedded as interactive 3D models.

We created the stories using PowerPoint 365 MSO version 2105. PowerPoint offers numerous possibilities to combine and visually arrange narrative elements. Animations and transitions can be defined and different file formats, such as images, videos and interactive 3D models, can be integrated. PowerPoint thus offers all the functionalities we need to design basic concepts that show the potential of narrative visualizations for medical data. In addition, the wide availability of PowerPoint makes it a good choice for an interactive narrative visualization intended for a general audience.

We use the derived stages shown in Figure 3 as the basic structure for designing our medical stories. Thus, some scenes for the three diseases are very similar. To avoid redundancy, we use the example of liver cancer to show a complete design of a medical story. For the two remaining diseases, we focus on illustrating diseasespecific aspects. Visual placeholders are inserted for parts of the stories whose design is similar to the liver scenario. We attached all created stories as supplemental material.

We have chosen the slideshow format as basic design genre. For each stage, one or multiple slides are prepared as scenes. Within the scenes, other narrative genres such as magazine style or partitioned poster are used. Smooth transitions are defined between scenes, with the timeline of stages visible in each scene. This way the user always knows s/he is in the story.

All three stories introduce a patient case to capture the user's attention. This consists of a catchy headline and a long-form textual description of the patient case, see Figure 4 (A1). In addition, the patient description is read aloud as a voiceover. This patient description should help the audience to relate to the case and motivate them to continue with the story. By pressing the start button, the user begins the actual story. Below, we provide detailed insights into the scenes.

Disease Definition. Within the first scene, we use the magazine style to introduce the affected anatomy by an interactive 3D model to provide an initial orientation to the topic. Inspired by the work of Garcia [ASB*16], statistical parameters of the disease are depicted as information graphics with icons and laddered text to quickly absorb information. We exclude visual representations of the disease at this stage to avoid overwhelming the user. Via voice narration, the user is encouraged to rotate the model using their fingers. Since free rotation of 3D objects is difficult to handle for inexperienced users, we limit rotation around a vertical axis with a finger movement from left to right.

In the liver cancer story, we embed an interactive 3D liver model alongside textual components and information graphics, see Figure 4 (B1) . We do not yet show 3D models of the tumors or unnecessary surrounding anatomy. One of the most important statistical parameters is the annual incidence, which is emphasized by the larger font size. Other parameters, such as the distribution between men and women and their average age of onset, are represented by annotated information graphics and icons to aid recognition and memorability. We apply similar concepts to the other two stories.

Anatomy. Again, we use the magazine style to describe the anatomy scenes. We describe the anatomical structures that are necessary to understand disease development. These facts include the location, importance, and function of the key anatomical structure(s). Finally, we introduce the disease by super-or juxtaposing the pathology on the normal anatomy with a crossfade transition.

Continuing the liver cancer story, the anatomical stage defines four key facts: (1) the liver is the largest and the most important organ to digest food and remove toxins, (2) it is located in the right upper part of the abdomen, below the heart, (3) it is supplied and drained by a vast network of blood vessels, and (4) in liver cancer, abnormal growth of cells in the liver forms tumors. We use several scenes to represent these key facts.

First, an automatic rotation of the 3D liver model gives an overview of its anatomical shape, where a long-form textual description with highlighted keywords provides more details. Following familiar objects and object continuity concepts, the story transitions to show anatomical context around the liver: labeled 3D models of the ribs, heart, and liver vasculature. The last scene transitions to show the disease: surrounding structures fade away and the fully-opaque liver becomes translucent to reveal tumors within. The accompanying text discusses development of a liver tumor due to abnormal cell growth.

The aneurysm and pelvic fracture stories both follow a similar introduction and flow of elements. However, a unique characteristic is the complex anatomical structure of the pelvis with multiple bones and closely related vessels, organs, and nerves. In addition, for this patient we have pre-and post-operative data available, which makes it possible to show treatment effects on real data. To communicate these anatomical peculiarities and the treatment process, we combine hotspots, 3D models, and DVR, see Figure 5 . The user can interactively explore anatomical structures by clicking on the hotspots (A) by highlighting the corresponding anatomical name in the text description or clicking on a structure of interest.

Symptoms. Using the partitioned poster genre, a visual overview of frequently-occurring symptoms is provided. For each symptom, we artistically create an icon with an accompanying caption. The use of icons as opposed to purely text-based listings aims to increase memorability of symptoms.The symptoms are displayed one after the other to give the user time to process each icon.

For liver cancer, we create icons and accompanying text for the following critical symptoms in advanced liver cancer: unexplained weight loss, loss of appetite, pain/pressure in the upper abdomen, increased temperature, weakness/fatigue, abdominal swelling, and yellowing of the skin, see Figure 4 (B2). We also identify key symptoms of pelvic fracture, with the same storytelling mechanisms.

We had to adjust the aneurysm story, since we focus on accidentally detected aneurysms without symptoms. Treatment bears considerable risks, which can exceed natural rupture risk. Therefore, the aneurysm story communicates how rupture-prone aneurysms can be detected as shown in Figure 6 . The first scene shows a 3D aneurysm model representing an incidental finding (see Figure 6 (A) ). An illustrative superimposed magnifying glass helps to quickly see the aneurysm in the complex vascular tree. Next, aneurysm rupture is shown illustratively (see Figure 6 (B). Using information graphics and textual descriptions, arranged in magazine style, it should be made clear that a rupture occurs very rarely but is very dangerous. Here, the information graphics are only indicated by a placeholder, as these would be similar to the graphics of the liver definition stage.

Diagnosis. The diagnosis is discussed using a variety of media. Key to this stage is informing the audience on how the diagnosis is achieved comprising diagnostic questions and imaging modalities. Each diagnostic question is illustrated as an individual scene using the magazine style. Important questions for liver cancer include, e.g., the size and location of tumors as well as the the exact tumor type. For this purpose, the 3D translucent liver model including the tumors is shown. The tumors are enriched by glyphs and annotations to visually communicate the the main aspect of each related question (see Figure 4 (C2-C3) ). Simple and clean vector illustrations describe diagnostic procedures, e.g., liver biopsy to determine the exact tumor type (see Figure 4 (D3)). Procedures with anatomical views different than previously presented include a rotating 3D navigator model of the organ that helps for view orientation.

The size and location of aneurysms in the brain vessels is also a critical diagnostic question, which we handle slightly differently (see Figure 6 (C)). This entails beginning with a localizing overall view of the brain vessels with the translucent surrounding skull before a zoom and rotation transformation focuses the camera tightly on the aneurysm to emphasize its location and shape. Panning and zooming allows the user to get even closer to the aneurysm, a detail view is shown in the form of a 3D aneurysm model, with blood flow suggested by animated particles, see Figure 6 (D).

Additional scenes give an impression of diagnostic imaging modalities used. Here, the magazine style is again used to combine image and text information. For each modality, a real image is incorporated as an example, e.g., ultrasound, CT, and MRI in liver cancer, where in each the tumors are highlighted by contours, see Figure 4 (A4). We use similar concepts to show imaging modalities employed in brain aneurysm and pelvic fracture diagnosis.

Treatment. The treatment stage provides an overview of typical therapy options and key aspects that influence treatment decisions. We do not consider rarely performed treatments that can only be done in special centers.

The first treatment scene uses a flow chart combined with longform textual descriptions. The chart in form of a directed graph describes the basic treatment approaches and their key aspects. By clicking on one of its nodes, the user gets more information in a magazine style about the selected treatment. Each treatment is shown as a 2D vector illustration of its key moment to provide clarity. We again use a navigator icon with labels to indicate key aspects of the procedure. An exception to this are the metal implants used in fracture treatment, such as in the pelvis data (see Figure 5 (D)). Similar to bones in a CT scan, these can be easily visualized by DVR, where optimal views from different perspectives can be shown. In case of palliative treatments where the target is more towards symptom alleviation, we use icons to create consistency and repetition between this stage and the earlier symptom stage.

For example, in liver cancer there are essentially two treatment strategies, curative and palliative [ASS*20] (see Figure 4 (B4) ).

To the right, we detail the key aspects that determine treatment in list form. This includes: the number of tumors present, their size, whether they have grown into blood vessels or into surrounding tissue, whether they have already metastasized, how functional the liver still is, and the patient's general health. On selection of the curative therapy chart element, the user is directed through the set of curative therapies. Here, the user learns that the goal of curative therapy is to cure the cancer, methods of which include (1) surgical removal of the tumor(s), (2) ablation, in which the tumor(s) is destroyed by heat or microwaves, and (3) radiation (Figure 4 (C4-A5) ). The user then is taken through the palliative treatment scenes. We use icons to indicate alleviation of symptoms that were introduced earlier in the story, as well as new icons, such as those for chemotherapy, cytostatic drugs, and high calorie foods that may prolong the patient's life and relieve symptoms (see Figure 4 (B5) ).

Prognosis. In addition to general statistics, the prognosis of a disease also includes several parameters depending on the severity and the chosen treatment. Since a detailed presentation of all dependencies and resulting parameters would overwhelm the general user, we limit ourselves to the most important parameters to give insight into the liklihood of a cure. Based on the partitioned poster genre, we produce an infographic-style illustration that makes use of color and laddered text to aid in information absorption and memory retention of key prognostic information. We use glyphs for men and women to convey that, unlike incidence, there are no significant prognostic differences between men and women.

The prognosis for liver cancer depends on the cancer stage and the condition of the liver [ASS*20] . We emphasize the relative 5year survival, which is around 18 % for men and women, in the largest typeface with an attention-drawing accent color along with icons indicating men and women (see Figure 4 (C5)). We use a smiley symbol to indicate the group where tumor removal often experiences a positive outcome. For stage I tumors, the 5-year relative survival is around 62 % in women and around 54 % in men. In stage IV, however, it is only 2 %. We repeat the use of larger typeface with accent color for the survival with gender symbols at a slightly smaller size. For both the aneurysm and pelvic fracture stories we use a similar presentation of laddered text with accent colors for percentages with symbols to indicate affected genders.

Prevention. In this stage, we focus on illustrating avoidable risk factors to give the user a sense of agency. Risk factors such as age or genetic factors that a person cannot influence are excluded since they are not actionable for the user. Similar to the symptoms, we use the partitioned poster genre and utilize icons to better recognize and understand the presented information, e.g., a martini glass for the recommendation to reduce alcohol consumption.

The main risk factor for liver cancer is cirrhosis, caused typically by chronic hepatitis B virus infection, depicted with a syringe and caption to vaccinate against hepatitis B, or high alcohol consumption, which we depict with an alcoholic beverage icon and a caption to limit alcohol consumption (Figure 4 (D5) ). Another risk factor is obesity which is depicted by a scale symbol with an arrow indicating increasing weight along with the caption "Keep weight in a healthy range.". Smoking also increases the risk of the disease, which we depict with a barred cigarette icon with the caption "Quit submitted to Eurographics Workshop on Visual Computing for Biology and Medicine (2021) Figure 4 : All scenes of the liver cancer story covering the seven derived stages. The narrative sequence is A1, B1, ..., D1, A2 and so on. Figure 5 : Excerpts from the pelvic fracture story. A unique characteristic is the complex anatomical structure of the pelvis with multiple bones and closely related vessels, organs, and nerves. Hotspots, 3D models, and DVR are combined to highlight these aspects and treatment.

smoking." We can similarly use such icons to show risk factors for the brain aneurysm and pelvic fracture stories.

In the following, we outline general issues that arise during medical story creation. We cover principle decisions regarding scene design, such as story element style and underlying design patterns. Furthermore, we discuss necessary story adaptations to address other diseases or other communication goals, e.g., education.

Content production. For our purposes of showing an initial concept for narrative medical visualization, the functionality of Pow-erPoint for story creation was sufficient. However, its content production feature set remains limited, with significant manual effort required to integrate multiple media types to tell a comprehensive story. We used several external tools to bridge this gap, including Adobe Illustrator to produce icons and treatment illustrations. While PowerPoint is able to embed 3D surface models, medical Figure 6 : Excerpts from the aneurysm storyboard. There is a trade-off between the rupture risk and the treatment risk. 3D models, conceptdriven visualizations, and blood flow depictions are combined to communicate this dilemma.

image feature extraction and advanced visualization, e.g., DVR, are not supported, which led to a need to use additional software.

Scene design. Following the suggestion of Böttinger et al. [BKY20] to make scenes "as simple as possible without risking scientific credibility," we extract as much information as possible from real medical image data to generate data-driven stories. However, for creating simple and clearly understandable scenes for our target audience, visual abstraction and easy-to-use interaction techniques are necessary [VCI20] . We used textual and verbal descriptions and avoided technical terms where possible. We added 2D vector illustrations to show important treatment concepts in a simplified way. While 3D data-driven models with surgical instruments would also be possible, this creates visually complex scenes that we felt may overwhelm or scare the user.

For touch-based interaction, we use interaction types that do not require a high degree of accuracy. These include single-touch gestures, e.g., rotation around a predefined axis and clicking on objects, as well as familiar multi-touch gestures, such as panning and zooming. Such interactions take advantage of user familiarity with everyday objects such as smartphones and tablet PCs without requiring extra equipment, such as a mouse or keyboard.

To select an appropriate narrative genre, there are currently no guidelines on which genre is most suitable for a given context. Since, we want an interactive and multi-media environment for our stories, slideshows seemed most appropriate. Within the scenes, we often combine 2D/3D representations with textual descriptions arranged in the magazine style to explain disease stages in a short and memorable way. In principle, we could also use another genre for the story or parts of the story, such as comics. Thus, a validation and derivation of guidelines of contextual genre recommendations is an important point for future research.

To inform the general public about a disease, we use all design patterns defined by Bach et al. [BSB*18] (cf. Section 2.3). The two most important patterns in our stories are framing by hiding data and structuring by revealing data. For example, within the 3D models only the most important structures are depicted while structures irrelevant to the story are excluded, e.g., other abdominal organs in the liver scenario. In addition, combinations of 3D structures, annotations/glyphs, and textual descriptions are revealed sequentially to avoid confronting the user with a visual overload. We use emotional and engaging patterns to capture and maintain the user's attention. We use an initial patient description to help users relate to the story, and incorporate interactive components encouraging users to take action. Argumentative patterns in the form of memo-rable icons convey avoidable risk factors. Our aim is to raise awareness of a disease and to encourage taking action and adopting a healthier lifestyle.

Evaluation of Medical Stories. Our stories were designed by scientists with many years of experience in the visualization of medical data. One of our co-authors is a medical illustrator and thus brings a lot of experience regarding the design of illustrations for the general public. In this forward-looking paper, we have basically conceptualized what medical stories could look like. We have not yet done any evaluation with the intended audience which is fundamental for future work.

Communication goals. In addition to informing, educating an audience would be another important communication goal. For example, one possible scenario would be to teach students about various medical conditions. The use of engaging patterns in the form of interactive exploration to maintain students' enjoyment during the learning process is particularly important [RKO*20] . An option could be to integrate an interactive quiz, where the user could check how much they have learned from the story. Moreover, argumentative patterns such as spaced repetition should be used to help the user remember the most important facts.

Furthermore, details on demand would be useful for learning. In currently available apps, such as "language learning apps", the respective topic is explained using a meaningful example. In addition, the user has the option of viewing further language-specific examples to consolidate the learning material. This details-on-demand principle could be applied to medical data. Upon request, the user could see additional records, e.g. a stage IV liver cancer dataset, with other anatomical features.

Application to other diseases. In this paper, we focus on diseases which can be diagnosed based on radiological image data. However, several diseases require other technologies for diagnosis. Examples are respiratory diseases such as asthma, heart rhythm disorders, or infectious diseases such as HIV, which are diagnosed by pulmonary function tests, electrocardiogram, and blood tests, respectively. While our basic division into the seven stages can also be applied to such diseases, the media used must be adapted accordingly. For example, 3D models based on real data cannot be generated for infectious diseases. Similarly, treatment options would have to be adapted. The division into curative and palliative methods is not always possible, since many diseases cannot be cured.

This section focuses on open research challenges focusing on the combination of narrative techniques and medical data.

Authoring tools for narrative medical visualization. When presenting medical information such as incidence rates or prognostic factors, any authoring tool suitable for data story creation would be suitable. However, presentation of medical imaging data offers additional challenges. Providing efficient authoring tools tailored to medical imaging data is a key aspect of advancing the use of narrative techniques in this area. Such an authoring tool would need to support data preprocessing (e.g., segmentation, smoothing, filtering) and provide techniques to add narrative elements. Here, also advanced labeling [OP14] and animation creation [PM20] techniques for medical data are needed. Existing expert tools for analyzing medical data, e.g., MeVisLab [AG] , 3D slicer [KPV14] , or VTK [SML98] could be chosen as a basis. These are powerful medical image processing and visualization frameworks freely available for non-commercial research. Their modular character allow fast integration and testing of new algorithms. They could be extended with narrative modules which allow scene generation scenes based on a selected scene type and transitions between scenes. Key features should be interaction tracking, e.g., to create animations and the definition of which user interactions should be provided.

An alternative would be to provide an authoring tool to create web-based stories. This would allow even more people to access the stories. For the creation of web-based stories, libraries such as VMTK [ISMA18] or VTK.js could be combined for the preparation and visualization of imaging data and D3 [Bos12] for the incorporation of information visualization and interaction.

General pattern design for medical data. The definition of dedicated patterns to capture medical narratives helps authors to structure their stories. In this paper, we propose a pattern for the narrative presentation of disease data comprising seven stages. However, there are many other medical aspects, such as healthy metabolic processes, pregnancy, and medical procedures, for which this pattern would not be suitable. Other patterns should be derived to structure such topics.

Narrative medical visualization for patients. In addition to medically interested people, patients could benefit from narrative medical visualization. Patients come into contact with medical data through their physicians who explain planned procedures and diagnosed diseases, including possible treatments and their prognosis. The patient typically sees excerpts from imaging data or schematic drawings on information sheets. However, since patients often have little medical knowledge, communication difficulties can occur, which could lead to uncertainty and fear. In a doctor-patient conversation, the patient could instead be informed about his or her condition using personalized visual representations. Here, narrative visualization could help the patient to understand diagnostic and therapeutic procedures such that a suitable therapy plan can be made in consultation with the physician. This would strengthen opportunities for participatory medicine. Support experts with narrative medical visualization. In addition to the general public, medical experts are also a potential target group for narrative visualization. Many expert visualization tools partly consist of very complex visual representations where the experts need extensive explanations and there is a steep learning curve. Especially for prospective experts, such as medical students, narrative techniques could help to understand complex medical data [RKO*20] . In contrast to stories developed for a general audience, expert narratives would need more emphasis on data preservation and, consequently, less abstraction. It would be interesting to further explore how expert tools can be enriched with narrative techniques and what impact this has on understanding.

Narrative medical visualization evaluation. To assess the usefulness of storytelling in a medical context, it is important to measure the quality of the visualization w.r.t. achieving communication goals. Evaluation of narrative visualizations is a difficult and complex task with which only a few works have currently dealt [MKK15; BDF15] . The basic idea is to evaluate design decisions made, techniques employed, and media used in terms of aspects such as comprehensibility [Fig14a] , memorability [BBK*15] , and effectiveness in engaging a general audience [BDF15; MHL*17].

In contrast to medical expert visualizations, which are often validated in controlled lab studies, typically within a short time frame, evaluation of narrative medical visualization will require a different approach to account for various scenarios and to reflect realworld uses. To get meaningful results, we have to reach a wider and more diverse audience than the usual small group of experts used in lab studies. Kosara et al. [KM13] suggested to use crowdsourcing platforms for such larger visualization studies. In addition, objective measures of comprehensibility, memorability, and userengagement have to be defined. Eye-tracking studies could help to better understand how different medical-related audiences view and interact with visualizations.

Advanced media for narrative medical visualization. Many of the existing visualization and interaction approaches were originally designed for a standard desktop computer set-up with mouse and keyboard input. However, in the last years new advances have been made in human-computer interaction technology [BYK*21]. New display technologies such as multi-touch displays have become very popular as they are intuitive, easy to use, and allow a direct manipulation of the visual data representation by multiple users. Furthermore, VR environments are increasingly being developed for medical scenarios such as surgical planning and medical education. It would be interesting to investigate how these mediums could be combined with narrative techniques to communicate medical data to general audiences. Furthermore, concepts such as voice and gesture control as well as human avatars to answer questions may bring complex scientific data closer to a general audience [YLB20] . A more detailed investigation is needed regarding their suitability for medical data.

The use of narrative techniques represents an important research focus with the goal of communicating complex scientific data in an understandable way to a general audience without specific expertise. To date, however, there are few approaches leveraging narrative techniques for medical data, although the audience for this is large. Patients, relatives and people interested in medicine can benefit from custom medical representations and expand their knowledge. In addition, physicians and medical students could benefit from the use of narrative techniques.

In this paper, we provide a first proof of concept demonstrating how narrative techniques could help explain disease characteristics to a wider audience. The combination of data-driven and illustrative presentations seems to be a powerful way to present complex diagnosis and treatment methods in an understandable way. Important points for future research are the development of appropriate authoring tools and the validation of narrative medical presentations.

Sometimes more is more: iterative participatory design of infographics for engagement of community members with varying levels of health literacy

Challenges in liver cancer and possible treatment approaches

OpenSpace: A system for astrographics

Beyond memorability: Visualization recognition and recall

Storytelling in information visualizations: Does it engage users to explore data

Reflections on Visualization for Broad Audiences". Foundations of Data Visualization

Challenges and Open Issues in Visualization for Broad Audiences". Foundations of Data Visualization

js -Data-Driven Documents

Reaching Broad Audiences from a Research Institute Setting". Foundations of Data Visualization

The computational fluid dynamics rupture challenge 2013 -Phase II: variability of hemodynamic simulations in two intracranial aneurysms

Unruptured intracranial aneurysms: An updated review of current concepts for risk factors, detection and management

HepaVision2-a software assistant for preoperative planning in living-related liver transplantation and oncologic liver surgery

The misuse of colour in science communication

Visualization in the earth sciences

A taxonomy of 3D occlusion management for visualization

How to tell stories using visualization

Narrative visualization: A case study of how to incorporate narrative elements in existing visualizations

Pelvic Fracture Care"

What storytelling can do for information visualization

Visualization rhetoric: Framing effects in narrative visualization

What Biological Visualizations Do Science Center Visitors Prefer in an Interactive Touch Table?": Educ Sci

The vascular modeling toolkit: a Python library for the analysis of tubular structures in medical images

HyperLabels-Browsing of Dense and Hierarchical Molecular 3D Models

Storytelling: The next step for visualization

Interactively Illustrating Polymerization using Three-level Model Fusion

3D Slicer: A Platform for Subject-Specific Image Analysis, Visualization, and Clinical Support". Intraoperative Imaging and Image-Guided Therapy

From Visualization Research to Public Presentation-Design and Realization of a Scientific Exhibition

The trends in incidence of primary liver cancer caused by specific etiologies: results from the Global Burden of Disease Study 2016 and implications for liver cancer prevention

More than telling a story: Transforming data into visually shared stories

Visual narrative flow: Exploring factors shaping data visualization story reading experiences

Towards a taxonomy for evaluating user engagement in information visualization

Scientific storytelling using visualization

Optimization of Vascular Surface Models for Computational Fluid Dynamics and Rapid Prototyping

Survey of labeling techniques in medical visualizations

A survey of medical animations

Reaching Broad Audiences in an Educational Setting". Foundations of Data Visualization

Narrative visualization: Telling stories with data

The online anatomical human: web-based anatomy education

Pelvis: Atlas-based surgical planning for oncological pelvic surgery

Emerging and recurring data-driven storytelling techniques: Analysis of a curated collection of recent stories

The role of computational fluid dynamics in the management of unruptured intracranial aneurysms: a clinicians' view

Storytelling and visualization: An extended survey

Visual Abstraction". Foundations of Data Visualization

Story telling for presentation in volume visualization

Story telling aspects in medical applications

Reaching Broad Audiences from a Science Center or Museum Setting". Foundations of Data Visualization

Exploranation: A new science communication paradigm

Interactive visualization of 3D scanned mummies at public venues

Liver cancer

Liver cancer

Understanding Liver Cancerthe Basics

Liver Cancer

URL: https : / / liverfoundation . org / forpatients / about -the -liver / diseases -ofthe-liver/liver-cancer

Liver Cancer

Liver and Bile Duct Cancer-Patient Version

What is an Aneurysm

What You Should Know About Cerebral Aneurysms

What is a cerebral aneurysm? URL: https : / / www . urmc . rochester . edu / submitted to Eurographics Workshop on Visual Computing for Biology and Medicine (2021) encyclopedia/content.aspx?ContentTypeID= 85&ContentID=P08772

Unruptured intracranial aneurysms

Pelvic Fractures/Acetabular Fractures

Types of Pelvic Injury