IJTM/IJCEE PAGE TEMPLATEv2 International Journal of Human Factors Modelling and Simulation 3(1), 90-106, 2012. doi:10.1504/IJHFMS.2012.050078 Applying Cognitive Science to Digital Human Modelling for User Centred Design Peter Thorvald*, Dan Högberg & Keith Case Virtual Systems Research Center University of Skövde, Sweden Mechanichal and Manufacturing Technology Loughborough University, UK Peter.Thorvald@his.se, Dan.Hogberg@his.se, K.Case@lboro.ac.uk *Corresponding author Abstract. To build software which, at the press of a button, can tell you what cognition related hazards there are within an environment or a task, is probably well into the future if it is possible at all. However, incorporating existing tools such as task analysis tools, interface design guidelines and information about general cognitive limitations in humans, could allow for greater evaluative options for cognitive ergonomics. The paper discusses previous approaches to the subject and suggests adding design and evaluative guiding in Digital Human Modelling that will help a user with little or no knowledge of cognitive science to design and evaluate a human- product interaction scenario. Keywords: Digital human modelling, cognition, context, situatedness, ecological interface design, system ergonomics, HTA, usability simulation. Peter Thorvald is a lecturer at the School of Technology and Society at the University of Skövde. He received his PhD from Loughborough University in 2011 and his MSc in Cognitive Science at University of Skövde in 2006. Peter is a member of the Virtual Systems Research Centre and the User Centred Product/Workplace Design research group at University of Skövde and his main research focus is information design for manual assembly in the automotive industry. Dan Högberg is an Associate Professor in the School of Technology and Society at the University of Skövde. His research interests include methods and support systems for designers and engineers to consider human-machine interaction related matters in development processes, for example, the development and integration of digital human modelling. He received a BSc in Product Design Engineering from the University of Skövde in 1998, an MSc in Engineering Design from Loughborough University, UK in 1999 and his PhD from Loughborough University in 2005. He is a member of the User Centred Product/Workplace Design research group at University of Skövde and the Virtual Ergonomics Centre (VEC) in Sweden. Keith Case is Professor of Computer Aided Engineering in the Mechanical and Manufacturing Engineering Department at Loughborough University where he leads the Product Realisation Technologies and Innovative Digital Manufacturing Research Groups. Keith has a manufacturing engineering BSc and a PhD from Nottingham University, is a Fellow of the Ergonomics and Human Factors Society, a Fellow of the British Computing Society and a Chartered Engineer. His PhD thesis was entitled ’An Anthropometric and Biomechanical Computer Model of Man’ (1975) and was concerned with developing the SAMMIE Digital Human Modelling System. Other research interests include virtual manufacturing, inclusive design and the application of genetic algorithms. 1 Introduction In Digital Human Modelling (DHM), the term ergonomics usually refers to modelling physical aspects of humans with the main focus being on anthropometry and physical strain on the body. This is also reflected in the DHM tools that exist on the market, e.g. RAMSIS, JACK, SAMMIE, V5 Human (Case & Porter, 1980; Bubb, 2007); tools that mainly, if not exclusively, model physical ergonomics. This paper proposes ways of bringing cognition into the equation and provide users of DHM tools with an aid in evaluating cognitive as well as physical ergonomics. Computer modelling of human cognition has traditionally mainly been done off-line in the sense that the cognitive system is viewed as a hardware independent program, effectively disregarding the surrounding environment and even the importance of a human body. However, in later years, there has been an increasing interest in viewing the human as part of a complex system, incorporating the environment and the human body in cognitive modelling. This has led to new theories regarding how humans cognize within the world and has allowed us to regard the body and the context as part of the cognitive system. Human cognition is not an isolated island where we can view our surrounding context as merely a problem space. We are very much dependant on our body and our surroundings to successfully survive in the world. Previous suggestions on integrating cognition in DHM tools have largely taken their basis in symbol processing architectures such as ACT-R, Soar etc. (Bernard et al., 2005; Gore, 2006; Carruth et al., 2007); architectures that disregard embodiment and situatedness of cognition. This paper places the computer manikins used in DHM tools within a context, a context where cognitive offloading and scaffolding onto the environment is supported. The main advantage of using DHM and incorporating the suggested functionality is that it can be used very early in the system development process. It also allows the designer to consider the spatial information that the physical array incorporates. In traditional usability methods, this is seldom the case as design iterations are often done offline in the sense that they only incorporate some (if any) physical properties of the domain where the system is to be implemented. 1.1 Human performance modelling For as long as experimental psychology has been of interest in science, trying to model human performance has also been pursued. Pew (2007) describes three major movements within this field of study; manual control models of human control, task network models that ultimately predict success and performance time of systems and cognitive architectures that utilizes theories on human performance to predict behavior. 1.1.1 Cognitive Modelling in DHM During the last decade, there have been several attempts at incorporating cognitive modelling in DHM, most of which have focused on using cognitive architectures to predict human performance. A research group at Sandia National Laboratories in New Mexico have created a framework based on a modular and symbol processing view of human cognition and others have focused on a rule based system built on architectures such as ACT-R and Soar (Bernard et al., 2005; Carruth et al., 2007). Though not built on exactly the same architecture, several others have gone about the problem in similar ways, ultimately trying to reach a state where the system can, at the press of a button, perform a cognitive evaluation (Gore, 2006). However, the methodology upon which these architectures are built is challenged by researchers that recommend a more situated view on cognition as a whole. This view, originating in the 1920s from the Russian psychologist Lev Vygotsky, argues that human cognition cannot be viewed separately from its context and body (Clark, 1997). There is no clear-cut line between what happens in the world and what happens in the head; the mind “leaks” into the world. A view already expressed in the DHM community is a need to stop dividing human factors into “neck up” and “neck down” and instead view the human as a whole (Feyen, 2007). This view finds much support in the work on social embodiment by Lawrence Barsalou and colleagues. They discuss how the embodiment of the self or others can elicit embodied mimicry in the self or others (Barsalou et al., 2003), ultimately arguing for a holistic view of the human where the body and mind are both necessary for cognition. Whereas the discussion on embodiment and situatedness is beyond the scope of this paper, it shows us how earlier approaches to modelling cognition in DHM are at best insufficient and that a new approach is needed. This paper discusses two separate approaches to modelling cognition in DHM. The first is a mathematical one where a plausible way towards creating a mathematical model of cognitive behaviour is suggested. The second has a much lower technological level as it tries to consider the human as a system with a physical body, acting within an environment. 2 Existing Mathematical Approaches Whereas past attempts at incorporating cognitive ergonomics in DHM can be criticized, there are other approaches that deserve mentioning. These are, more often than not, based on theories aimed at quantifying behaviour and trying to predict reaction times, body movement etc. (Shannon, 1948; Hick, 1952; Fitts, 1954; Freivalds, 2009). Shalin et al (1996) described a number of potential approaches and classified them into the following categories: • Predetermined Motion-Time Systems (PMTS) o MOST o MTM • Mathematical models o Signal detection theory o Information theory • Symbolic computational models o ACT o Soar 2.1 PREDETERMINED MOTION-TIME SYSTEMS Digital Human Modelling systems usually had their origins either in military or manufacturing applications and SAMMIE (Bonney et al., 1972) is an example of the latter type. At this time economic efficiency of work was as important if not more important than workplace ergonomics and so such DHM systems often contained predetermined motion-time systems. In the case of SAMMIE this was a representation of MTM-2, but SAMMIE soon became a purely ergonomics system and the MTM component was embodied in a separate system (AUTOMAT) (Bonney & Schofield, 1971). In PMTS, task performance is predicted by the addition of expected times for sequential motor processes and is very vulnerable to inaccuracies in the estimations of physical task demands. These demands require elaborate and accurate task performance models that may or may not be present (Shalin et al., 1996). While PMTS, by their computational nature might seem very suitable for inclusion in DHM software, they do not consider the mental demand that is involved with performing a task but regard tasks as sequential and rather offline in a cognitive sense. This results in a lack of concern for distribution of cognition (Hutchins, 1995) and also social, physical and mental context. 2.2 MATHEMATICAL MODELS Mathematical representation models of cognition as found in information theory and detection theory are mainly used to describe mental tasks and predict error rates (Shalin et al., 1996). Also, the field offers specific insights into the description and effect of noise (Shannon, 1948) as well as predicting reaction times as results of spatial properties of the task and the number of choices (Hick, 1952; Fitts, 1954). Information theory quantifies information by calculating the entropy (number of options) in a task. For instance, choosing between eight parts in assembly requires three bits (log28) of information. Using tables that predict task performance, such as reaction times, offers standardized reaction times for a given number of bits in the task. For instance, the reaction time (not response time) for a task with eight options, three bits, would result in a reaction time of 800 milliseconds (Freivalds, 2009). Detection theory, or signal detection theory, was founded as a method of quantifying results from stimulus detection. Among the fundamentals is the classification of responses seen in figure 1 (Wickens, 2002). Response present Response absent Stimulus present Hit Miss Stimulus absent False alarm Correct rejection Figure 1. Matrix of the classification of response and stimulus. Detection theory also takes into account the biases that the respondent may or may not have which are based on the consequences of false alarms or misses (Wickens, 2002). For example, an oncologist, examining x-rays for possible tumours may be biased towards finding a tumour where there is none (False alarm), as a failure to recognize a tumour (Miss) might have larger repercussions than a false alarm. These are often referred to as false-positives or false-negatives. 2.2.1 SYMBOLIC COMPUTATIONAL MODELS As mentioned, the traditional approach to cognitive science, although heavily challenged in recent years (Searle, 1980; Harnad, 1990), is to view human cognition as a symbol processing system, effectively disregarding the context as merely a problem space. The difficulty in considering multiple task dimensions in the traditional mathematical models has given rise to the use of symbolic models for application to task analysis. The models are based on the belief that intelligence is symbol processing in the brain, much like a formal computer system. It is a matter of following a set of basic rules for manipulating symbols and searching over a set of stored problem-solving operations (Shalin et al., 1996). 3 A Mathematical Model of Cognition in DHM The aforementioned mathematical approaches all have in common that they are quantifiable. Whereas they might be subject to challenge on their philosophical basis in cognitive science, the ability to compute them gives them an advantage. Such an approach has great potential impact due to the context in which it can be used. An expert evaluation system, such as that discussed in the rest of the paper, will be limited due to its need for expert analysts. However, a mathematical model would consist of a number of parameters and variables that need to be completed for the system make an accurate computation of cognitive strain or whatever the system is designed to do. To create such a model, the first task would be to create a way to quantify information. Within information theory, information entropy is measured in bits (Shannon, 1948; Freivalds, 2009), which is a binary expression of the amount of information required to decide between two equally likely alternatives. It is calculated using: 𝐻𝐻 = 𝑙𝑙𝑙𝑙𝑙𝑙2𝑛𝑛 Where H is the entropy and n is the amount of equally likely alternatives. Using this mathematical expression, calculating the entropy for a decision with eight equally likely alternatives would result in three bits as: 𝑙𝑙𝑙𝑙𝑙𝑙28 = 3 Three bits, when written in binary ranges from 0-111 which corresponds to 0-7, or eight states. Information entropy can also be calculated for alternatives that are not equally likely by using this formula (Freivalds, 2009): 𝐻𝐻 = Σ𝑝𝑝𝑖𝑖𝑙𝑙𝑙𝑙𝑙𝑙2 � 1 𝑝𝑝𝑖𝑖 � where pipi is the probability of i i is the number of alternatives from 1 to n Once entropy has been calculated, it is merely a matter of assigning each bit a value. Looking at Hick’s law (or Hick-Hyman law) reaction time for instance is calculated by: 𝑅𝑅𝑅𝑅 = 𝑎𝑎 + 𝑏𝑏𝐻𝐻 where RT is response time H is the information entropy in bits a is the intercept b is the slope or information processing rate The information processing rate, or the bandwidth of information processing if you will (expressed in bits/s), is sensitive to disturbances. Consider the biases made in detection theory, mentioned earlier, and it is plausible that a task with a high demand on accuracy would affect the speed of the response. Creating a model which takes the information processing rate into account would require quite a bit of empirical data. What would be sought in such an endeavor is something similar to table 1. Table 1. Linear model of choice reaction time according to Hick's law. Choices Bits Reaction time (ms) 1 0 150 2 1 300 4 2 450 8 3 600 Finding the parameters for such a table would, as mentioned, require a substantial amount of empirical data but once it is gathered, it could be applied to DHM and to DHM-tools. Simply calculating the entropy of a task would allow us to find the reaction time as long as we have found the slope-intercept relationship first. Similarly, this methodology can be extended to include not only reaction times but also response times. Fitts’ law for instance (Fitts, 1954; MacKenzie, 1995), is often coupled with Hick’s law and performs similar calculations to determine the time required to move into a target area. This might be moving a finger to a button, moving a cursor to an icon etc. The general rule being that the time it takes to move into the target area is a function of the distance and the size of the target. A model, like the one described, could potentially have great impact in society. However, it might not be as easy as it sounds to create it, as it also needs to be generalizable to a number of problems. It needs to be able to handle choices of buttons on a car’s dashboard while at the same time being able to calculate which lever to pull in an overhead crane to raise the hook The model presented in this chapter, while full of potential were it to be realized, has inherent problems. Any model intended for use by laymen, needs to be simple and generalizable, something that this model would have difficulties with. However, it is still worth keeping in mind that perhaps it is not the absolute values of e.g. a reaction time that is most important but rather how a particular choice holds up against several others and in that context, the model might be very successful. A significantly different way from mathematics to handle cognition will be discussed henceforth in this paper. Instead of focusing on the quantifiable parts of information entropy, softer values of how a human cognizes will be presented. 4 Cognition as a System For a system design to become successful, the incorporation of human factors is essential. To a large part, physical ergonomics is very well accounted for in today’s system design practices, but the cognizing human is often neglected. On the one hand, as technology increasingly demands more human processing abilities, the modelling of human cognition becomes more important. The range of human behaviours need to be known to design for human-related control systems (Bubb, 2002). However, improved knowledge of human behaviours must not excuse ‘bad design’. It should become more important to design systems that are compatible with how human cognition actually works in order to make the “entire system” (i.e. including the human component) work in an effective and efficient manner, with the overall objective that the system output shall be high and stable (high productivity and quality). So perhaps instead of calling for raised knowledge of cognitive strengths and limitations, one should focus on developing technologies that comply with the whole human system, physical and mental. System ergonomics can be used to describe a more or less complex task’s mental demands on a human. It does so in three ways (Bubb, 2002). 1. Function The main consideration of function is what the operator has in view and to what extent the task is supported by the system. It is largely defined by the temporal and spatial properties of the activities to be performed. When and where should the task be performed? 2. Feedback The feedback allows the user to identify what state the system is in. If a performed task has resulted in anything, what task was performed etc. It is very important to allow the operator to recognize if an action had any effect on the system and also what the result of it was (Norman, 2002). For example, even if a computing task on a PC takes some time to calculate, the operator is informed that the computer is working by a flashing light or an hourglass on the screen. Figure 2. A seat adjustment control which exhibits excellent natural mapping or matching between the system and the user’s mental model. 3. Compatibility Compatibility is largely about the match between systems or between the system and the user’s mental model of the system. The operator should not be required to put too much effort into translating system signals. Compatibility relates information sources to each other. A very simple and obvious example from the automotive industry is described by Norman (2002) with a seat adjustment control from a car. A similar seat adjustment control can be viewed in figure 2. It is obvious in the figure that the system (the adjustment control) corresponds well to the result of the task of manoeuvring the controls. The control maps very well to the response of the seat and to the user’s probable mental model. However, the compatibility is not exclusively relevant to the psychological issues but a designer also needs to consider the physical compatibility of the user and the system. Controls might for example be spatially located away from the physical reach of the human. Though these three points are hardly sufficient for a comprehensive design tool, they are of great help in an initial state of system design and will prove helpful to us in developing a more detailed design aid. 5 Methods for Interface Design and Evaluation In Human-Computer Interaction (HCI) there are several evaluation methods with great use for certain situations. As the aim of this paper is to present a proposal for a design tool, we shall take a closer look at a few of these methods along with a task analysis tool. 5.1 Task Analysis All good design processes include some sort of task analysis. To be able to design a system that fits both task and human, we need to know as much as possible about the task. A fairly quick and dirty task analysis which provides a good basis for further development is the Hierarchical Task Analysis (HTA) (Annett, 2003). A HTA is a tree diagram of the task structure and serves several purposes. It gives us a good overview of the system or the task and subtasks that need to be performed, and assists in achieving common ground within a design group. It can also even serve as a task evaluation tool, allowing a designer to find global problems that can be missed when using usability inspection methods such as cognitive walkthrough (Polson et al., 1992), heuristic evaluation (Nielsen, 1993; Nielsen, 1994) etc. Global issues are mainly related to the structure of the task and the relation between the subtasks whereas local issues are within a subtask with a very limited scope. Figure 3. A very simple HTA of the process of making a pot of coffee. The creation of a HTA is fairly simple. First, identify the overall task to be performed, which in our very simple example, illustrated in figure 3, is making a pot of coffee. The HTA in figure 3 shows this process and also shown are the plans within which each subtask should be performed. In this example it is limited to doing the tasks in order or doing two subtasks first in any order and then continuing with the third. However, these plans can be very variable and flexible including elements such as selections (do one but not the other), linear or non- linear, or even based on a specific condition (if X then do Y, else Z). The finished task analysis is then used as a basis for further inspections and design iterations. 5.2 Ecological Interface Design Ecological Interface Design (EID) is spawned from Cognitive Work Analysis (CWA), which was developed as an analytical approach to cognitive engineering by the Risø group in Denmark (Vicente, 1999). CWA was developed to aid in the design of very critical human-machine systems such as nuclear power plant control rooms to make them safer and more reliable. It is an approach that allows the operator to handle situations that the system designers had not anticipated. CWA is made up of five phases to analyse within a system. These phases are work domain analysis, control task analysis, strategies analysis, social-organisational analysis and worker competencies analysis (Sanderson, 2003). Having these analyses allows the designer and the operator a better understanding of the system and already this enables the operator to better respond to unforeseen events. The idea behind EID is to create interfaces based on certain principles of CWA. It is very closely related to the principles of ecological psychology and direct perception, concepts developed by J.J Gibson in the 70s (Gibson, 1986). Gibson argued that there is enough information in the visual array to directly perceive information and that mental processing of visual information is not necessary. Though this claim is highly challenged, Make Coffee 1. Add Water 2. Add Coffee 3. Press Button 1.1.Fill Pot with Water 1.2. Pour water into Coffee Maker 2.1. Place Filter 2.2. Add Coffee Do in any order 1-2 Do 3 Do 1.1-1.2 Do 2.1-2.2 EID is largely built up around these principles in that its goal is to create interfaces containing objects that visually reveal information on their function. A related goal of EID is to make affordances visible in interface design. Affordances, another concept created by Gibson, are the action possibilities of a specific object (Gibson, 1986; McGrenere & Ho, 2000). The ideas surrounding affordances and EID can also be found in other areas of the scientific literature. In product design, one tends to discuss similar issues in terms of semantics (Monö, 1997). 5.3 Usability Inspections Usability inspection methods are predictive evaluation methods, usually performed without end user participation (although this is not a prerequisite). Usability experts simulate the users and inspect the interface resulting in problem lists with varying degrees of severity (Nielsen & Mack, 1994). 5.3.1 Cognitive Walkthrough A cognitive walkthrough is usually performed by usability experts considering, in sequence, all actions incorporated in a predefined task. Its focus is almost exclusively on ease of learning and the method contains two phases. First the preparations phase where the analyst defines the users, their experience and knowledge; defines the task to be analysed and identifies the correct sequence of actions to achieve the goal of the task. In the second phase, the analysis phase, the analyst answers and motivates a set of questions for each action within the task (Polson et al., 1992). 1. Will the user try to achieve the right effect? For example, if the task is to fill up the car with petrol and a button first has to be pressed from inside the car to open the gas cap, does the user know that this has to be done? 2. Will the user notice that the correct action is available? Simply pressing the button for the gas cap would not be a problem but if the button has to be slid or twisted in some way the user may not think of this. 3. Will the user associate the correct action with the desired effect? Is it clear that this is what the specific control is for? Unambiguous icons and names of controls are important to this aspect. 4. If the correct action is performed, will the user see that progress is being made? The importance of feedback, discussed earlier, comes into play here. These questions, though applicable to many tasks, are merely guidelines towards conducting a successful cognitive walkthrough. The method’s advantage is its focus on detail; it identifies local problems within the task and considers the users’ previous knowledge and experiences. However, it rarely catches global problems related to the overlying structure of the task and can be viewed as fairly subjective. It also requires a detailed prototype for evaluation although this would probably not be a problem if it is complementing a DHM tool where a virtual prototype is likely to already exist. Also, it is not just about presenting information but it is about how the information is presented. A robot would not have problems with different types of knobs or buttons as it has no preconceived notions of how they should look and does not expect things to be in a certain way. Humans do and this is why we have to stick to consistency and standards. 5.3.2 Heuristic Evaluation Just as in the case of cognitive walkthrough, heuristic evaluations are usually performed by usability experts sequentially going through each action within a main task with a basis in a set of heuristics (Nielsen, 1994). The method was developed by usability expert Jacob Nielsen and a set of his heuristics can be found through his publications (Nielsen, 1992; Nielsen, 1993; Nielsen, 1994). Examples of Nielsen’s heuristics are • Match between system and the real world o Similar to the matching and mapping concept discussed in system ergonomics, the system should speak the users’ language, matching the real world in terms of terminology and semiotics. • Consistency and standards o Also related to the matching concept is using accepted conventions to avoid making users wonder whether different words, icons or actions mean the same thing in different contexts. • Recognition rather than recall o Options should be made visible to avoid making the user having to remember how or where specific actions should be performed. • Aesthetic and minimalist design o Dialogues and controls should not be littered with irrelevant or seldom used information. Heuristics can be added and subtracted to fit certain tasks before the evaluation commences. The method results in problem lists with motivations and rankings of the severity of the problems found. 6 An Expert Design Guide for DHM The evaluation and design tools discussed in previous sections are developed for interface design in different settings than DHM. However, the design guide proposed in this section is a hybrid of these, adapted for use under the specific conditions that DHM provides. The method strives to take into account global as well as local issues through the use of action based interface inspections and a task analysis focusing on the structure of the task. As stated earlier in this paper and by others (Pheasant & Haslegrave, 2006), every good design process starts with a task analysis. For our purposes, a hierarchical task analysis is suitable as it complements the inspection methods incorporated in this design guide. The HTA serves several purposes; it gives the designer a better understanding of the task and it provides a common understanding of the task within a development group. The task analysis can also be used as an evaluation tool of the task itself. It allows the designer to identify problems in the task structure that could result in problems with automatism (Thorvald et al., 2008), it can identify recurring tasks and give them a higher priority in the interface etc. Complementary to the task analysis, the designer should consider who the users are and what a priori knowledge they have. This resembles the guiding system for utilising traditional DHM tools in development processes suggested by Hanson et al. (2006), where the users’ anthropometry and tasks are defined before the actual analyses or simulations are performed. The sequence-based walkthrough will take its basis in the task analysis performed. For each subtask (box) of the HTA, a set of questions, based on Bubb’s points regarding system ergonomics (Bubb, 2002), will act as guidelines for the design. • Function – When and where should the action be performed? o Will the user identify the action space where the correct action should be performed? What do the physical and geographical properties of each control convey to the user? o Frequency of actions – a frequently recurring action should take precedence in taking up place and intrusiveness in the physical and cognitive envelope. o Importance of action – Safety critical systems should also take precedence in the available information space. o Minimalism of design – avoid taking up space with irrelevant or rarely needed information. Hick’s law: Reaction time is a function of the number of choices in a decision (Hick, 1952). In figure 4, there is an example of what a virtual interface, modelled in a DHM-tool can look like. In this case the picture shows a fighter jet cockpit used for evaluation where the pilot needed to locate a “panic button” to bring the aircraft back into control under extreme physical and mental load conditions. Figure 4. Two views of a cockpit modelled in the DHM tool SAMMIE. The action spaces that the user has to identify when performing an action are the controls in front of, and to the right and left of the steering control stick. Preferably, a frequently performed action control should be placed on the control stick or directly in front of it as these are the spaces that best correspond to the physical and cognitive reach of the pilot. Also safety systems, as in the case of the evaluation in figure 4, should be placed so that they are easily accessible for the user. Knowing that certain controls are rarely used, they can be placed to the right and left to avoid having too many options in terms of ‘pushable’ buttons at the same place. The intrusiveness and affordances of such “high priority controls” should also be accentuated in terms of their design. • Feedback o Will the user understand that a correct or faulty move has been made? o Is the system status visible? Understanding what has been done and what is in progress of happening with the system can prove vital in many cases. Surely we can all relate to a situation where we have pressed the print button more than once only to find out that we have printed several more copies than needed. While this may be a minor problem, one can easily imagine the problems that can arise in more critical domains. What if there were no indications for what gear the car’s gearbox was in? The driver would have to test each time to see if the car is in reverse or drive. In an incident at a hospital, a patient died as a result of being exposed to a massive overdose of radiation during a radiotherapy session. The problem could easily have been avoided, had the system provided the treating radiology technician with information of the machines settings (Casey, 1998). • Compatibility o Does the system match other, similar systems in terms of semantics, semiotics etc.? o Does the system match the real world and the plausible mental model of the user? o Are demands on consistency and standards of the domain met? o Action-effect discrepancies – is it obvious beforehand that a certain action will have a certain effect? Accurate mapping between systems and mental models is a key concept in the compatibility section. This includes trying to adhere to consistencies and standards of the organisation and the specific field. There should also be clear connection between action and effect. Neglecting these consistencies can lead to serious problems as in the case with an aircraft’s rudder settings. The sensitivity of the rudder could be set through a lever placed to the side of the pilot’s seat. However, between the simulator for the aircraft and the actual aircraft, the lever was reversed, moving in the opposite direction for maximum and minimum sensitivity almost resulting in a crash (Casey, 2006). 7 Conclusions & Future Work In ergonomics, it seems to be common practice to separate human factors into “neck up” and “neck down”. Though this approach may make it easier to study ergonomics, it does not portray an entirely accurate picture of the human. The evidence for a tight coupling between mind and body is so overwhelming that instead of talking about mind and body, perhaps we should be talking about the human system. The aim of this paper has been to consider past and current approaches towards integrating cognition into DHM tools and outline potential new design guides and models to help designers to achieve this integration in a better way. The originality of the design guide lies in the combination of methods from the HCI-field and the new application to DHM. The guide is not complete and needs extensive further development and testing. However, it is a pragmatic start towards including functionality to consider cognitive ergonomics in DHM tools. The mathematical model on the other hand is merely a theoretical suggestion for what potentially could be realized in the future. It also serves as contrast to the expert design guide presented later in the paper. Obviously, the conceptual guide for DHM tool development needs to be detailed and tested on real problems to prove its contribution to the field of ergonomics design and evaluation. Small scale testing with, and evaluation on, the tool will be carried out and the tool will be compared to other HCI and cognitive ergonomics evaluation tools such as CWA, usability testing, user studies, etc. Another approach towards the objective to consider the ‘full’ human in DHM tools is the conceptual ideas of creating DHM Personas as presented and discussed in Högberg, et al. (2009). This concept is based on the idea of giving the manikins in the DHM tool a degree of personality by describing characteristic ‘user types’. The basic approach to portray user types in terms of narrative texts and images is a widespread design method (Nielsen, 2002; Pruitt & Grudin, 2003; Cross, 2008). The idea to map such descriptions on computer manikins is however a newer approach, and resembles the ideas of Högberg and Case (2006) as well as Conradi and Alexander (2007). Figure 5 shows two manikins as DHM Personas, where the descriptions applied on the manikins convey certain capacities and give the manikins personality traits. This attempt is to enrich the DHM tool user’s understanding of end user requirements and about user diversity in the targeted population, both related to physical and cognitive ergonomics, but also for what we may term as pleasurable or emotional ergonomics (Jordan, 2002; Siddique, 2004). Figure 5. Example of DHM Personas. An interesting possibility would be to integrate DHM Personas in the methods for the consideration of cognitive aspects in DHM as suggested in this paper. This as an attempt to even further take into account the ‘whole’ human, and the diversity of humans, in the human-system interface being designed or evaluated. For example, one may imagine the product or workplace designer doing a HTA or CWA with ‘different hats on’, as described by the DHM Persona, hence even further increasing the chance that the ‘entire” user and user diversity is considered in the design process. References Annett, J. (2003). Hierarchichal Task Analysis. In D. Diaper & N. Stanton (eds.). The Handbook of Task Analysis for Human-Computer Interaction. pp. 67-82. Mahwah, New Jersey: Lawrence Erlbaum Associates. Barsalou, L. W., Niedenthal, P. M., Barbey, A. K. & Ruppert, J. A. (2003). Social Embodiment. In B. H. Ross (ed.). The Psychology of Learning and Motivation. San Diego, CA: Academic Press. Bernard, M. L., Xavier, P., Wolfenbarger, P., Hart, D., Waymire, R., Glickman, M. & Gardner, M. (2005). Psychologically Plausible Cognitive Models for Simulating Interactive Human Behaviors. In Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting. pp. 1205-1210. Bonney, M. C., Case, K., Hughes, B. J., Schofield, N. A. & Williams, R. W. (1972). Computer Aided Workplace Design using SAMMIE. In Ergonomics Research Society Annual Conference, Cardiff, April. Bonney, M. C. & Schofield, N. A. (1971). Computerized work study using the SAMMIE/AUTOMAT system. International Journal of Production Research, 9 (3), 321- 336. Bubb, H. (2002). Computer Aided Tools of Ergonomics and System Design. Human Factors and Ergonomics in Manufacturing, 12 (3), 249-265. Bubb, H. (2007). Future Applications of DHM in Ergonomic Design. LECTURE NOTES IN COMPUTER SCIENCE, 4561, 779-793. Carruth, D. W., Thomas, M. D., Robbins, B. & Morais, A. (2007). Integrating Perception, Cognition and Action for Digital Human Modeling. In V. G. Duffy (ed.). Digital Human Modeling, HCII 2007. pp. 333-342. Berlin: Springer-Verlag. Case, K. & Porter, J. M. (1980). SAMMIE - A Computer Aided Ergonomics Design System. Engineering, 220, 21-25. Casey, S. M. (1998). Set phasers on stun and other true tales of design, technology, and human error. Santa Barbara, CA: Aegean. Casey, S. M. (2006). The atomic chef : and other true tales of design, technology, and human error. Santa Barbara, CA: Aegean Pub. Co. Clark, A. (1997). Being There: Putting Brain, Body, and World Together Again. MIT Press. Conradi, J. & Alexander, T. (2007). Modeling personality traits for digital humans. Society of Automotive Engineers., SAE Technical paper (2007-01-2507). Cross, N. (2008). Engineering design methods: strategies for product design. Chichester: Wiley Feyen, R. (2007). Bridging the Gap: Exploring Interactions Between Digital Human Models and Cognitive Models. In V. G. Duffy (ed.). Digital Human Modeling, HCII 2007. pp. 382-391. Berlin: Springer-Verlag. Fitts, P. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of experimental psychology, 47 (6), 381-391. Freivalds, A. (2009). Niebel's methods, standards, and work design. New York: McGraw-Hill Higher Education. Gibson, J. J. (1986). The ecological approach to visual perception. Hillsdale, NJ: Lawrence Erlbaum Associates. Gore, B. F. (2006). Human Performance: Evaluating the Cognitive Aspects. In V. G. Duffy (ed.). Handbook of digital human modeling. Mahwah, New Jersey. Hanson, L., Blomé, M., Dukic, T. & Högberg, D. (2006). Guide and documentation system to support digital human modeling applications. International Journal of Industrial Ergonomics, 36 (1), 17-24. Harnad, S. (1990). The symbol grounding problem. Physica D: Nonlinear Phenomena, 42 (1-3), 335-346. Hick, W. E. (1952). On the rate of gain of information. The Quarterly Journal of Experimental Psychology, 4 (1), 11-26. Hutchins, E. (1995). Cognition in the Wild. Cambridge, MA: MIT Press. Högberg, D. & Case, K. (2006). Manikin characters: user characters in human computer modelling. Contemporary Ergonomics, 499-503. Högberg, D., Lundstrom, D., Hanson, L. & Warell, M. (2009). Increasing Functionality of DHM Software by Industry Specific Program Features. SAE Technical paper (2009-01-2288). Jordan, P. (2002). Designing pleasurable products: An introduction to the new human factors. London: Taylor & Francis. MacKenzie, I. S. (1995). Movement time prediction in human-computer interfaces. In R. M. Baecker, W. A. S. Buxton, J. Grudin & S. Greenberg (eds.). Readings in human- computer interaction. pp. 483-493. McGrenere, J. & Ho, W. (2000). Affordances: Clarifying and Evolving a Concept. Proceedings of Graphics Interface 2000, 179-186. Monö, R. (1997). Design for Product Understanding. Skogs Boktryckeri AB. Nielsen, J. (1992). Finding usability problems through heuristic evaluation. In Proceedings of ACM, Monterey, CA. 373-380. Nielsen, J. (1993). Usability Engineering. San Francisco, CA: Morgan Kaufmann. Nielsen, J. (1994). Heuristic evaluation. In J. Nielsen & R. L. Mack (eds.). Usability inspection methods. pp. 25-62. New York: John Wiley & Sons, Inc. Nielsen, J. & Mack, R. L. (1994). Usability inspection methods. Wiley New York. Nielsen, L. (2002). From user to character: an investigation into user-descriptions in scenarios. In DIS 2002, Designing Interactive Systems, London. 99-104. Norman, D. (2002). The design of everyday things. New York: Basic Books Pew, R. W. (2007). Some history of human performance modeling. In W. Gray (ed.). Integrated models of cognitive systems. pp. 29-44. New York: Oxford University Press. Pheasant, S. & Haslegrave, C. M. (2006). Bodyspace: Anthropometry, Ergonomics and the Design of Work. CRC Press. Polson, P. G., Lewis, C., Rieman, J. & Wharton, C. (1992). Cognitive Walkthroughs: A Method for Theory-Based Evaluation of User Interfaces. International Journal of Man- Machine Studies, 36 (5), 741-773. Pruitt, J. & Grudin, J. (2003). Personas: practice and theory. In Proceedings of the 2003 conference on designing for user experiences, San Francisco, CA. 1-15. Sanderson, P. M. (2003). Cognitive work analysis. In J. M. Carroll (ed.). HCI models, theories, and frameworks: Toward an interdisciplinary science. pp. 225-264. San Francisco, USA: Morgan Kaufmann Publishers. Searle, J. (1980). Minds, brains, and programs. Behavioral and Brain Sciences, 3 (3), 417-457. Shalin, V., Prabhu, G. & Helander, M. (1996). A cognitive perspective on manual assembly. Ergonomics, 39 (1), 108-127. Shannon, C. (1948). A Mathematical Theory of Communication. Bell System Technical Journal, 27, 379-423, 623-656 Siddique, Z. (2004). Conceptualizing Emotional Ergonomics and Exploring Ways to Empower Workplace Dynamics. Contemporary Ergonomics, 540-544. Thorvald, P., Bäckstrand, G., Högberg, D., de Vin, L. J. & Case, K. (2008). Demands on Technology from a Human Automatism Perspective in Manual Assembly. In Proceedings of FAIM2008 Skövde, Sweden, June-July 2008. 632-638. Vicente, K. J. (1999). Cognitive Work Analysis: Toward Safe, Productive, and Healthy Computer-Based Work. Lawrence Erlbaum Assoc Inc. Wickens, T. (2002). Elementary signal detection theory. Oxford University Press, USA. 1 Introduction 1.1 Human performance modelling 1.1.1 Cognitive Modelling in DHM 2 Existing Mathematical Approaches 2.1 PREDETERMINED MOTION-TIME SYSTEMS 2.2 MATHEMATICAL MODELS 2.2.1 SYMBOLIC COMPUTATIONAL MODELS 3 A Mathematical Model of Cognition in DHM 4 Cognition as a System 5 Methods for Interface Design and Evaluation 5.1 Task Analysis 5.2 Ecological Interface Design 5.3 Usability Inspections 5.3.1 Cognitive Walkthrough 5.3.2 Heuristic Evaluation 6 An Expert Design Guide for DHM 7 Conclusions & Future Work References