Motion-capture-based walking simulation of digital human adapted to laser-scanned 3D as-is environments for accessibility evaluation H O S T E D B Y Available online at www.sciencedirect.com http://dx.doi.org 2288-4300/& 20 (http://creativeco nCorrespondin E-mail addre kanai@ssi.ist.ho m.tada@aist.go.j Peer review u Journal of Computational Design and Engineering 3 (2016) 250–265 www.elsevier.com/locate/jcde Motion-capture-based walking simulation of digital human adapted to laser-scanned 3D as-is environments for accessibility evaluation Tsubasa Maruyamaa,n, Satoshi Kanaia, Hiroaki Datea, Mitsunori Tadab aGraduate School of Information Science and Technology, Hokkaido University, Sapporo 060-0814, Japan bNational Institute of Advanced Industrial Science and Technology, Tokyo 135-0064, Japan Received 17 September 2015; received in revised form 8 December 2015; accepted 21 March 2016 Available online 25 March 2016 Abstract Owing to our rapidly aging society, accessibility evaluation to enhance the ease and safety of access to indoor and outdoor environments for the elderly and disabled is increasing in importance. Accessibility must be assessed not only from the general standard aspect but also in terms of physical and cognitive friendliness for users of different ages, genders, and abilities. Meanwhile, human behavior simulation has been progressing in the areas of crowd behavior analysis and emergency evacuation planning. However, in human behavior simulation, environment models represent only “as-planned” situations. In addition, a pedestrian model cannot generate the detailed articulated movements of various people of different ages and genders in the simulation. Therefore, the final goal of this research was to develop a virtual accessibility evaluation by combining realistic human behavior simulation using a digital human model (DHM) with “as-is” environment models. To achieve this goal, we developed an algorithm for generating human-like DHM walking motions, adapting its strides, turning angles, and footprints to laser-scanned 3D as-is environments including slopes and stairs. The DHM motion was generated based only on a motion-capture (MoCap) data for flat walking. Our implementation constructed as-is 3D environment models from laser-scanned point clouds of real environments and enabled a DHM to walk autonomously in various environment models. The difference in joint angles between the DHM and MoCap data was evaluated. Demonstrations of our environment modeling and walking simulation in indoor and outdoor environments including corridors, slopes, and stairs are illustrated in this study. & 2016 Society of CAD/CAM Engineers. Publishing Services by Elsevier. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Keywords: Digital human model; Walking simulation; Laser-scanning; Accessibility evaluation; Motion-capture; Human behavior simulation 1. Introduction Owing to our rapidly aging society, accessibility evaluation to enhance the ease and safety of access to indoor and outdoor environments for the elderly and disabled is increasing in importance. In the International Organization for Standardiza- tion (ISO) 21452, “accessibility” is defined as “provision of buildings or parts of buildings for people, regardless of disability, age or gender, to be able to gain access to them, /10.1016/j.jcde.2016.03.001 16 Society of CAD/CAM Engineers. Publishing Services by Elsev mmons.org/licenses/by-nc-nd/4.0/). g author. sses: t_maruyama@sdm.ssi.ist.hokudai.ac.jp (T. Maruyama), kudai.ac.jp (S. Kanai), hdate@ssi.ist.hokudai.ac.jp (H. Date), p (M. Tada). nder responsibility of Society of CAD/CAM Engineers. into them, to use them and exit from them” [1]. Accessibility must be assessed not only from the general standard aspect specified in ISO 21542 [1] but also in terms of physical and cognitive friendliness for users of different ages, genders, and abilities, as recommended in ISO/IEC Guide 71 [2]. An example of the former is dimensional criteria such as corridor width and slope angle, which can be evaluated only by taking account of the shape of the environment. Physical and Cognitive friendliness involves more human-centered criteria, such as tripping risk and ease of wayfinding, which can be evaluated by taking account of both human movement and the environment. Conversely, human behavior simulation has been progres- sing recently in the areas of crowd behavior analysis and emergency evacuation planning [3], and there is a high ier. This is an open access article under the CC BY-NC-ND license www.sciencedirect.com/science/journal/22884300 http://dx.doi.org/10.1016/j.jcde.2016.03.001 www.elsevier.com/locate/jcde http://dx.doi.org/10.1016/j.jcde.2016.03.001 http://dx.doi.org/10.1016/j.jcde.2016.03.001 http://dx.doi.org/10.1016/j.jcde.2016.03.001 mailto:t_maruyama@sdm.ssi.ist.hokudai.ac.jp mailto:kanai@ssi.ist.hokudai.ac.jp mailto:hdate@ssi.ist.hokudai.ac.jp mailto:m.tada@aist.go.jp Fig. 1. Overview of the research. T. Maruyama et al. / Journal of Computational Design and Engineering 3 (2016) 250–265 251 possibility that the simulation can be applied to accessibility evaluation. In human behavior simulation, the activities of a pedestrian model can be predicted in 2D indoor and outdoor environment models [4]. Moreover, human behavior simula- tion using kinematic digital human models (DHMs) even in a 3D environment model has become possible because of recent advances in computer performance [5]. However, in previous human behavior simulations [3–5], environment models represent only “as-planned” situations, in contrast to “as-is” environments. Furthermore, in the simula- tion, a pedestrian model cannot generate the detailed articu- lated movements of various groups of people including the elderly, children, males, and females. In contrast to an “as- planned” environment, an “as-is” environment exhibits the following characteristics: – It represents the current environment correctly. – It includes extra objects which were placed in the environ- ment after building construction (e.g., furniture, dust bins, and fire extinguishers). – It includes small barriers and obstacles on the floor that were ignored in the CAD data. The presence of extra objects, small barriers, and small obstacles strongly affects the accessibility of an environment. Basing the accessibility evaluation on really existing facilities is therefore important. However, 2D or 3D CAD data of the existing state of the facility are not always available. In this situation, we have to construct the 3D environment model manually by measuring the environment. This is labor-intensive and tedious, and the constructed environment model is sometimes inaccurate. Using an as-is environment model, which is auto- matically constructed from the laser-scanned point clouds, there- fore offers advantages in terms of both cost and accuracy. Accessibility evaluation based on an as-is environment model is also easily applied to as-planned environments by converting the 3D mesh or solid model to dense point clouds. Therefore, accessibility evaluation systems using as-is envir- onment models have a wide range of applications. For these reasons, the as-is environment model is superior for accessi- bility evaluation of a current environment. Considering this background, the final goal of our research was to develop a virtual accessibility evaluation by combining realistic human behavior simulation using DHMs with “as-is” environment models. As shown in Fig. 1, our research group has already developed a prototype system of environment modeling and basic walking simulation for a DHM to achieve the first step of this goal [6]. In this system, the walking simulation algorithm used only single reference motion- capture (MoCap) data for flat walking and enabled a DHM to walk autonomously in as-is and flat indoor environments while adapting its strides and turning angles accordingly. However, the algorithm does not allow the DHM to walk on non-flat terrain surfaces such as slopes and stairs. Therefore, in this research, we develop an algorithm that enables a DHM to walk autonomously even on the non-flat terrain of as-is indoor and outdoor environments. The strides, turning angles and footprints of the DHM can be adapted to different laser-scanned environment models including corri- dors, slopes, and stairs. In the algorithm, first, a flat walking motion of the DHM is generated using our previous algorithm [6] based on single reference MoCap data for flat walking. Then, the pelvis and swing ankle joint positions during flat walking are changed to adapt to walking on the non-flat terrain by increasing or decreasing the positions based on our analysis of a preliminary experiment, in which real human walking motions in various environments were measured using a MoCap system. Finally, all the joint angles are determined T. Maruyama et al. / Journal of Computational Design and Engineering 3 (2016) 250–265252 by analytically solving inverse kinematics (IK) to achieve the adapted pelvis and swing ankle joint positions. Moreover, we develop an algorithm that extracts a stair tread boundary for enabling the DHM to walk on stairs while estimating suitable footprints on the treads. Demonstrations of our environment modeling and walking simulation in indoor and outdoor environments including corridors, slopes and stairs are illustrated. Modeling and simulation efficiency and accuracy are also evaluated. The rest of this paper is organized as follows. In Section 2, related work of the research is introduced and our contribu- tions are clarified. In Section 3, the 3D environmental modeling algorithm for the human behavior simulation is described. In Section 4, the preliminary experiment and the MoCap-based adaptive walking simulation algorithm is described in detail. Finally, the demonstrations, efficiency, and accuracy of the environmental modeling and walking simulation are shown in Section 5. 2. Related work This research primarily relates to three areas: human behavior simulation, 3D environment modeling from laser- scanned point clouds, and digital human modeling for walking simulation. Human behavior simulation must deal with the interactions between the pedestrian models and the environ- ment models, for example in navigation and footprint estima- tion. In addition, the presence of extra objects, small barriers, and small obstacles, which are included in an “as-is” environ- ment model, strongly affects accessibility evaluations based on human behavior simulation. For these reasons, in Section 2.1, the previous work on human behavior simulation is summar- ized with reference to the type of environment modeling and representation used (i.e., as-is or as-planned). Conversely, the previous work on walking simulation in Section 2.3 is summarized without reference to the type of environment modeling and representation used. 2.1. Human behavior simulation In human behavior simulation, studies have been conducted on crowd behavior analysis and emergency evacuation plan- ning [3,4]. Many different methods exist for pedestrian simulation, such as social forces [7] and cellular automata [8], as reviewed by Duives et al. [4]. However, these simulation algorithms are in 2D environments, and precise 3D simulation in as-is environment models is basically unfeasible. Recently, an evacuation-planning simulator that enables kinematics-based walking simulation in a 3D environ- ment model was proposed by Kakizaki et al. [5]. However, their work used only “as-planned” CAD data of a building as the environment model. In addition, the pedestrian model cannot generate the detailed articulated movements of various groups of people including the elderly, children, males, and females. This is because the generation of pedestrian motion is based only on solving IK without the measurement data of a real human. On the other hand, as in the simulation by Pettre et al. [9], a navigation graph representing free space and environmental connectivity can be generated automatically from a 3D mesh model. In their research, the walking path of each pedestrian model is automatically selected by specify- ing start and goal points on the navigation graph. Each pedestrian model can then walk automatically along the path. However, in their study, there is no discussion as to whether the 3D mesh model was as-is or as-planned. Additionally, they used a simple-shaped 3D mesh model, which is too rough to capture the detail of an as-is environment. In contrast to these studies, our proposed human behavior simulation focuses on generating detailed articulated move- ments of a DHM in 3D “as-is” environment models using novel laser-scanning technology. Furthermore, to achieve reliable accessibility evaluation based on human behavior simulation, our proposed human behavior simulation integrates the follow- ing technologies: – Automatic construction of an “as-is” environment model from the laser-scanned point clouds, which is used for the simulation. – Automatic estimation of the walking path, walking trajec- tory, and footprints of the DHM in the as-is environment. – Adaptation of the flat walking motion of various people of different ages, genders, and body dimensions to various as-is walking environments, including stairs and slopes. 2.2. D as-is environment modeling 3D as-is environment modeling from massive laser-scanned point clouds has been actively studied. The goal of these studies was to automatically reconstruct indoor environments [10] and extract semantic objects such as floors, walls and ceilings [11,12], household goods [13], and a constructive solid geometry model of environments [14] from massive laser-scanned point clouds. Existing algorithms and related techniques have been reviewed in the literature [15]. However, these studies focus on general object recognition for environ- mental modeling and do not necessarily aim for human-like walking navigation in the environment models. Moreover, these algorithms neglect to model small barriers and obstacles on a floor as part of the environment model, which is an important element for accessibility evaluation. In contrast to these studies, our environment model, which is automatically constructed from laser-scanned point clouds, can represent walk surfaces that include small barriers, obstacles, and navigation information for the human behavior simulation required for accessibility evaluation. 2.3. Digital human modeling for walking simulation Many algorithms have been developed in digital human modeling for walking simulation. Among them, a variety of human walking patterns can be synthesized using PCA [16]. Although such simulation systems can generate human-like T. Maruyama et al. / Journal of Computational Design and Engineering 3 (2016) 250–265 253 walking motion, they still require a large collection of MoCap data in advance. Therefore, many researchers tackle generating an arbitrary human motion from only a small number of existing MoCap data. Motion synthesis and editing [17], motion rings [18], and machine learning [19] are typical examples. They require a small number of MoCap data to adapt the DHM's strides, turning angles, and footprints to its walking environment models. However, it is generally difficult to persuade elderly persons, who are the main targets of accessibility evaluation, to join prolonged MoCap data collec- tion experiments. Conversely, human-like walking motions have been generated in recent physics-based walking simula- tions using motion controllers and game-engines [20,,21]. The review literature also summarizes a physics-based walking simulation [22]. However, the naturalness of a walking motion is sensitive to motion controller parameters, which are difficult to tune. In addition, Al-Asqhar et al. [23] proposed a motion- retargeting algorithm in which joint positions relative to the walking surface can be flexibly adapted even to dynamically changing walking terrains. However, in their study, the resultant motion is not validated. On the other hand, Reed et al. [24] introduced stair-ascending and stair-descending algo- rithms into a motion simulator called human motion simulation framework (HUMOSIM). HUMOSIM generates lower-limb motions based on behavior-based IK to achieve realistic motions. However, in their study, MoCap data collections are still required for solving behavior-based IK. In contrast to these studies, our walking simulation algorithm Fig. 2. Overview of 3D environmental model – Generates human-like walking motion even on non-flat walking terrain such as slopes and stairs, using only the single reference MoCap data for flat walking. – Adapts the strides, turning angles, and footprints of the DHM to different as-is environment models including corridors, slopes, and stairs, even on non-flat terrain. – Used the analysis of MoCap data from a real human subject, acquired in various walking environments including slopes and stairs. – Provides fast and autonomous walking simulation directly in the point cloud-based environment models including slopes and stairs. – Was validated by MoCap data of a real human acquired in non-flat walking terrain. – Provides a small difference in joint angles between the DHM and MoCap data. 3. 3D environment modeling from laser-scanned point clouds Fig. 2 shows an overview of the 3D environmental modeling and walking simulation of the DHM in our study. For the walking simulation of the DHM, first, a 3D environment model was constructed automatically from the 3D laser- scanned point clouds of the as-is environment [6]. As shown in the “Environmental modeling” section of Fig. 2, the 3D environment model consists of two point clouds Q and W, navigation graph GN, and tread boundary of staircase Bm;n, where Bm;n represents the boundary lines of the nth stair step in the mth stair. Q ¼ ðqi; niÞ represents the down-sampled ing and walking simulation of the DHM. T. Maruyama et al. / Journal of Computational Design and Engineering 3 (2016) 250–265254 points with normal vectors, and W ¼ Wk � Q is the set of walk surface points, where qi and ni represent a down-sampled point and a normal vector at qi, respectively. In this study, we developed a novel algorithm for estimating the tread boundary of staircase Bm;n. The rest of this section is organized as follows. First, the algorithms of the environment modeling, which were already described in a previous paper [6], are briefly introduced in Sections 3.1, 3.2, and 3.3. Then, the details of the proposed algorithm for tread boundary estimation are described in Section 3.4. 3.1. Down-sampling and normal vector estimation Multiple laser-scanned point clouds are first merged to make one registered point cloud. This registered point clouds contain huge number of points; hence, it is down-sampled using a voxel grid. In this study, about 1000 points per square meter is sufficient for modeling and simulation. Normal vectors at the down-sampled points are then estimated using principal component analysis (PCA) on the local neighboring points [25]. These point clouds with normal vectors Q ¼ ðqi; niÞ are used as a part of the 3D environment model to express the geometry of the entire environment [6]. 3.2. Extraction of walk surface points The set of walk surface points representing walkable surfaces such as floors and stair steps is automatically extracted from the down-sampled points Q [6]. First, if the angle between a normal vector ni at a point qi and a vertical vector v ¼ ½0; 0; þ1� is smaller than the thresh- old ε (we set ε¼ 201), then the point is added into horizontal points QH ¼ qjH located on a horizontal plane. Finally, the horizontal points QH are clustered into a set of walk surface points W ¼ Wk � QH using a region-growing algorithm based on a k-nearest search [6]. 3.3. Navigation graph construction A navigation graph representing the environmental path- ways for navigating the DHM during walking simulation is Navigation graph Position vector Radius Node Edge Fig. 3. Overview of 3D e generated from walk surface points Wk � W. To generate a navigation graph from laser-scanned point clouds, we extend the algorithm of Pettre et al. [9], which generates a navigation graph from a simple-shaped 3D mesh model [6]. In our algorithm, the navigation graph is generated from the walk surface image Ik, which is generated by projecting walk surface points Wk onto a horizontal plane. As shown in Fig. 3 (a), the navigation graph GN ¼ V; E; c; th i comprises a set of graph nodes V and a set of edges E. Each node vk AV represents free space in the environment and is generated on a medial pixel obtained from the image Ik. Each has a position vector tðvkÞ and the attributes of a cylinder cðvkÞ, whose radius rðvkÞ and height hv represent the distance to the wall and walkable step height, respectively. Each edge ek represents the connectivity of the environment and is generated between two adjacent nodes with a common region [6]. The down-sampled points Q, walk surface points W, and navigation graph GN were also used in our previous walking simulation [6]. In this study, the tread boundary Bm;n is newly estimated as part of the 3D environment model for the walking simulation on stairs. Details are described in the following subsection. 3.4. Tread boundary estimation The tread boundary estimation is an essential function in realizing autonomous walking simulation of a DHM. This is especially true in indoor environments including stairs since the original point clouds do not contain any semantic information about the environment. The tread boundary Bm;n is estimated from walk surface points Wk � W for estimating suitable footprints of the DHM and avoiding swing foot collisions with treads when the DHM walks on stairs, where Bm;n represents the boundary lines of the nth stair step in the mth stair. In our algorithm, the tread boundary was estimated from the walk surface points Wk which were extracted by a region-growing algorithm based on the normal vectors of each point, as described in Section 3.2. These normal vectors were estimated by PCA. However, the normal vector at a point located near the actual boundary of a stair tread is often tilted toward the neighboring region due to the nature of PCA. As a Tread boundary estimation Tread Tread boundary nvironment modeling. Fig. 5. Overview of the preliminary experiment. T. Maruyama et al. / Journal of Computational Design and Engineering 3 (2016) 250–265 255 result, the extracted walk surface points Wk of a stair tread will often miss the point clouds near the actual tread boundaries. Therefore, in our algorithm, the point clouds on the actual tread boundary of stairs are first extracted at high speed and with high precision using a neighboring search algorithm. The actual tread boundary is estimated using these points. The details of the algorithm are given below. As shown in Fig. 3(b), first, a convex hull Hm;n is generated from the walk surface points Wm;n, where Wm;n represents the walk surface points of the nth stair step in the mth stair. In the figure, pk represents the kth vertex of Hm;n. Then, a set of neighboring points Nk of pk is extracted from Q using a k- dimensional tree [25] to exactly extract the entire set of points of the nth stair step. These points pu � Nk are inserted into Wm;n. Finally, as shown in Fig. 3(b), the tread boundary Bm;n is constructed as a convex hull of Wm;n. In this research, the Ramer Douglas Peucker algorithm [26] is applied to Bm;n in order to reduce the number of vertices in Bm;n. 4. MoCap-based adaptive walking simulation in as-is environments After environmental modeling, a walking simulation of the DHM was performed directly in the point clouds-based environment models. As shown in the “walking simulation” section of Fig. 2, to generate human-like walking motion, we could select and use only single reference MoCap data for flat walking from the gait database (DB) containing the data of 139 subjects provided by the DHRC [29]. As shown in Fig. 4, our DHM has 41 degrees of freedom (DOF) in total and the same body dimensions as the subjects in the gait DB. As shown in Fig. 2, the walking simulation consists both of macro- and micro-level simulations. The macro-level simulation computes the macro-level behavior of the DHM, including the walking path and walking trajectory of the DHM during the simulation. In contrast, the micro-level simulation computes the Fig. 4. Detailed structure of our DHM. micro-level behavior of the DHM, including the joint-level synthesis of the DHM during one-step walking. In addition, a preliminary experiment is conducted to construct our simulation algorithm and to validate resultant walking motion of the DHM. Details are given in the following subsections. 4.1. Preliminary experiment for measuring human motion in various environments We conducted a preliminary experiment to discover the differences in real human walking motion between various walking environments in cooperation with the Digital Human Research Center (DHRC) of the National Institute of Advanced Industrial Science and Technology (AIST). As shown in Fig. 5, we acquired MoCap data of human walking motion on a flat terrain, 3-deg slope, 6-deg slope, and stairs using a MoCap system (Vicon MX system and Vicon Nexus [27]) for three male subjects (All subjects are 23 years old). These MoCap data were analyzed using commercial software (Visual 3D [28]) to obtain the joint angles of the subject during walking. These data and acquired knowledge were used to design our walking simulation algorithm and validate the DHM walking motion. In particular, these data were used in the following processes: – As described in Section 4.3.3, the swing ankle trajectory of the DHM is adapted to non-flat walking terrain using a cubic curve, which was modeled on our analysis of the preliminary experiment. – As described in Section 4.3.5, the swing ankle trajectory of the DHM is modified to avoid collisions with stairs based on a cubic spline interpolation, again modeled on our analysis of the preliminary experiment. – As described in Section 5.3, the simulated walking motion was validated by comparing the knee and hip joint angles of the DHM with the MoCap data on actual human motion. Fig. 6. Overview of the macro-level simulation (a) Global path findings (b) Walking trajectory generation. Fig. 7. Updating next subgoal position xt and locomotion vector v. T. Maruyama et al. / Journal of Computational Design and Engineering 3 (2016) 250–265256 4.2. Macro-level simulation As shown in Fig. 6(a), in the macro-level simulation, first, the user is asked to input start and goal positions as a start node vs and a goal node vg, respectively. Then, a set of DHM's walking paths VW ¼ Vi is found automatically using a depth- first search repeatedly over the navigation graph GN. Each path Vi consists of a set of graph nodes and edges, which are connected from vs to vg. If multiple walking paths exist between vs and vg, all of them can be extracted. Then, one suitable walking path VP AVW is automatically selected from VW by using the equation P ¼ argmin i ðci � BÞ ð1Þ where B ¼ β1; β2 � � ðβk A½0; 1�Þ and ci ¼ ½di; bi� represent the user's path preference and path cost vector of the path Vi. Each element βk specifies the degree of preference between the travel distance and the narrowness of the path. In the vector ci, di A½0; 1� and bi A½0; 1� represent the normalized travel dis- tance along Vi and the normalized narrowness of Vi, respec- tively. Each path cost vector ci is automatically estimated from each path Vi. Once the walking path VP has been determined, as shown in Fig. 6(b), DHM's walking trajectory SO ¼ si is generated automatically, where SO represents a sparsely discretized sequence of the DHM pelvis positions si during the walking simulation. The trajectory SO is initialized as a sequence of the center points s2k ¼ t við ÞþhPZ of each node vi AVP and interpolated points s2k þ1 ¼ Iðvi; vjÞþhPZ, where Iðvi; vjÞ, hP, and Z ¼ ð0; 0; þ1Þ represent a point located at the centroid of the common region between two adjacent nodes vi; vj AV P, a vertical distance between the heel and the pelvis of the selected DHM, and a vertical unit vector, respectively. Finally, an optimization algorithm is applied to make the trajectory SO more natural and smooth, while avoiding contact with the walls [6]. 4.3. Micro-level simulation based on single reference MoCap data After the walking trajectory SO is determined, the one-step walking motion of the selected DHM is generated along the trajectory SO in the micro-level simulation. The processes (A7-A12) in Fig. 2 show an overview of the walking simulation algorithm. In our algorithm, first, a next footprint xf is determined on the walk surface points W (Fig. 2 (A7)). Then, a virtual flat walking motion of the DHM is pre- generated using our previous algorithm [6] based on the single reference MoCap data for flat walking (Fig. 2(A8)). As a result, a pelvis position trajectory f P ϕð Þ and an ankle position trajectory of a swing leg (hereafter, called swing ankle position trajectory) f A ϕð Þ for one-step virtual flat walking are obtained, where ϕA½0; 1� is the normalized phase of one-step walking. Next, the trajectories fP ϕð Þ and f A ϕð Þ are adapted to the actual walking environment of the DHM (Fig. 2(A9)). Consequently, the adapted trajectories f 0 P ϕð Þ and f 0 A ϕð Þ are obtained. At the same time, the stance foot angle θstE ϕð Þ and swing foot angle θswE ϕð Þ are interpolated based on the “elevation angle” repre- sentation of each foot (Fig. 2(A10)). After that, in the case of walking on stairs, a collision-free ankle position trajectory f 00 A ϕð Þ, which has no collisions with stairs, can be further obtained by modifying the adapted ankle trajectory f 0 A ϕð Þ (Fig. 2(A11)). By contrast, in the case of walking on other terrains, f 00 A ϕð Þ is directly copied from f 0 A ϕð Þ. Finally, the one-step motion of the DHM is generated based on the adapted pelvis trajectory f 0 P ϕð Þ, swing ankle trajectory f 00 A ϕð Þ, and interpolated elevation angles θstE ϕð Þ and θswE ϕð Þ using IK (Fig. 2(A12)). Details are given in the following subsections: 4.3.1. Estimating next footprint (A7) As shown in Fig. 7, when the DHM passes through a point on the trajectory sk AS O, the system sets a next subgoal position xt as xt ¼ sk þ2, which serves as a temporal target position during the simulation. The subgoal position xt is continuously updated as the DHM walks along the trajectory SO. Then, as shown in the figure, the next locomotion vector v is determined as v ¼ xt �xcð Þ=‖xt �xc‖, where xc represents the current pelvis position of the DHM. After that, as shown in Fig. 8, the next footprint point xf , which represents a heel placement of the swing leg, is Estimating next footprint on a flat, sloping or bumpy terrain Estimating next footprinton a tread Target posture Stride Search space Locomotion vector Current posture Virtual flat walking (A8) Tread boundary Fig. 8. Estimating next footprint xf . Current posture Initial posture Target posture Generating target posture Swing ankle interpolation Stride ( ) ( ) Gait DB [29] Reference MoCap data Stance leg interpolation ( ) Fig. 9. Pre-generating virtual flat walking motion. T. Maruyama et al. / Journal of Computational Design and Engineering 3 (2016) 250–265 257 determined on the set of walk surface points W. The following algorithms are executed to locate xf on different terrains: staircase treads and others such as flat, sloping or bumpy terrain. As shown in Fig. 8(a), in the case of walking on a flat, sloping or bumpy terrain, a cylindrical search space CF is generated centered at a point pf horizontally located ahead of the current heel position xhs by a specified stride length w. If the multiple walk surface points are included in CF, the walk surface points having the maximum point number Ws are extracted from W in order to locate the heel of the swing leg on the widest walkable region inside of CF. Finally, the next footprint point xf is determined as a centroid of Ws. Note that any user-defined stride length w that is different from the original stride length in the reference MoCap data from the gait DB can be specified in our simulation. On the other hand, as shown in Fig. 8(b), in the case of walking on stairs, the next footprint point xf is located so that 3/4 of the total foot length Lf is placed on the tread according to a previous observational result of the HUMOSIM study [24]. 4.3.2. Pre-generating virtual flat walking motion (A8) After determining the next footprint point xf , a one-step virtual flat walking motion of the DHM with a stride length w 0 , which is different from w, is pre-generated on the basis of the following procedure. Since xf is determined by searching for walk surface points as described in previous section, the horizontal distance between xf and xhs is different from w, where xhs represents the heel position of the swing leg of the DHM. Therefore, the stride length w 0 is newly determined as w 0 ¼ distH xf ; xhs � � , where distHðxf ; xhsÞ is the horizontal dis- tance between xf and xhs. Fig. 9 shows an overview of our method. In that method, single reference MoCap data selected from the gait DB is used for generating human-like walking motion. In the figure, an initial posture FI, representing the full-body posture at the initial contact frame of the next walking step (ϕ ¼ 1), is obtained from the reference MoCap data. In the algorithm of this method: – a target posture Ft with the stride length w 0 at the next walking step can be generated by applying a cyclic- coordinate-descent IK (CCDIK) [30], which is an iterative IK solver for redundant link-mechanisms, to the initial posture FI – a range of motion (ROM) of the joint angles of the DHM can be satisfied by introducing the ROM constrains into the CCDIK – the ith joint angle of the DHM's stance leg θDHMi ðϕÞ is estimated by cubic spline interpolation only using the stance leg angles θDBi ðϕjÞ at the middle-stance phase ϕj A0:4; 0:5; 0:6 obtained from the single reference MoCap data – the swing leg motions are estimated by solving the IK analytically to achieve interpolated ankle positions fDHMA ðϕÞ, where fDHMA ðϕÞ are estimated by cubic spline interpolation of the original ankle positions f DBA ðϕjÞ from the MoCap data – small angle differences in joint angles can be provided between the DHM and reference MoCap data, which are approximately 101 and 51 (maximum) in the knee and hip joints, respectively. As shown in Fig. 8(a), a pelvis position trajectory fP ϕð Þ ¼ ½xP ϕð Þ; yP ϕð Þ; zP ϕð Þ� and ankle position trajectory of the swing leg fA ϕð Þ ¼ ½xA ϕð Þ; yA ϕð Þ; zA ϕð Þ� for one-step virtual T. Maruyama et al. / Journal of Computational Design and Engineering 3 (2016) 250–265258 flat walking can be obtained as results of this process (A8). The details of this process are described in our previous paper [6]. 4.3.3. Adapting joint position trajectories (A9) After generating the virtual flat walking motion of the DHM, the trajectories f P ϕð Þ and fA ϕð Þ of the virtual flat walking motion are adapted to actual walking environments. Fig. 10 shows an overview of the adaptation for the ankle trajectory fA ϕð Þ as an example. An adapted pelvis position trajectory f 0 P ϕð Þ ¼ x 0 P ϕð Þ; y 0 P ϕð Þ; z 0 P ϕð Þ � � and adapted ankle position tra- jectory f 0 A ϕð Þ ¼ x 0 A ϕð Þ; y 0 A ϕð Þ; z 0 A ϕð Þ � � , which are adapted to the changes in the terrain height ht per step, are obtained from x 0 P ϕð Þ ¼ xP ϕð Þ y 0 P ϕð Þ ¼ yP ϕð Þ z 0 P ϕð Þ ¼ zP 0ð ÞþdP ϕ 0 P � � þgP ϕ 0 P � � ϕ 0 P ¼ ðy 0 P ϕð Þ�y 0 P 0ð ÞÞ=ðyP 1ð Þ�yP 0ð ÞÞ 8>>>>>< >>>>>: ð2Þ x 0 A ϕð Þ ¼ xA ϕð Þ y 0 A ϕð Þ ¼ yA ϕð Þ z 0 A ϕð Þ ¼ zA 0ð ÞþdA ϕ 0 A � � þgA ϕ 0 A � � ϕ 0 A ¼ ðy 0 A ϕð Þ�y 0 A 0ð ÞÞ=ðyA 1ð Þ�yA 0ð ÞÞ 8>>>>>< >>>>>: ð3Þ where dP ϕ 0 P � � ¼ zP ϕ 0 P � � �zP 0ð Þ gP ϕ 0 P � � ¼ ðyP 1ð Þ�yP 0ð ÞÞðyA 1ð Þ�yA 0ð ÞÞ htϕ 0 P 8< : ð4Þ Fig. 10. Adapting the ankle position trajectory fA ϕð Þ to the terrain height ht. Fig. 11. Elevation an dA ϕ 0 A � � ¼ zA ϕ 0 A � � �zA 0ð Þ gA ϕ 0 A � � ¼ vA1 þvA2 �2htð Þϕ 0 A3þ �2vA1 �vA2 þ3htð Þϕ 0 A2þvA1ϕ 0 A ( ð5Þ In the above equations, the adapted pelvis height z 0 P ϕð Þ is obtained by increasing or decreasing the original zP ϕð Þ by the linear function gPðϕ 0 PÞ of Eq. (4) depending on the changes in the terrain height ht. By contrast, as shown in Fig. 10, the adapted ankle height z 0 A ϕð Þ is obtained by increasing or decreasing the original zA ϕ 0 A � � by the cubic function gAðϕ 0 AÞ of Eq. (5) to respect the original ankle positions of the swing leg at the beginning of the swing phase. This is because the swing foot of a real human is rotated about a toe contact position (toe locker) during the phase, as observed in [31]. In Eq. (5), the cubic function gAðϕ 0 AÞ is determined so as to satisfy its boundary conditions gA 0ð Þ ¼ 0, gA 1ð Þ ¼ ht, _gA 0ð Þ ¼ vA1, and _gA 1ð Þ ¼ vA2. We set vA1 ¼ 0 and vA2 ¼ ht based on our analysis of the preliminary experiment to measure human walking motion on the stairs described in Section 5.1. 4.3.4. Elevation angle interpolation (A10) The stance and swing foot elevation angles θstE ϕð Þ and θswE ϕð Þ are then interpolated to determine the ankle joint angles during one-step walking. As shown in Fig. 11(a), a foot elevation angle θE represents the angle between the sole and the vertical axis lz, whereas an ankle joint angle θJ represents the relative angle between the foot and the lower leg. Using the elevation angle representation enables the DHM to adapt the foot orientation to the actual walking environment. As shown in Fig. 11(b), the foot elevation angles θstE ϕð Þ and θswE ϕð Þ are interpolated using cubic spline curves so that they satisfy θstE ϕ st i � � ¼ θstE_flat ϕsti � � and θswE ϕ sw j � � ¼ θswE_flat ϕswj � � , where θswE_flat ϕ st i � � and θstE_flat ϕ sw j � � are the elevation angles of the pre-generated virtual flat walking at ϕsti (i ¼ 1; 2; 3; 4) and ϕswj (j ¼ 1; 2), respectively.ϕsw1 ¼ ϕmin is dynamically determined, whereas ϕsw2 is fixed as 0.6. The walking phase ϕmin is determined as the walking phase when θ sw E_flat ϕð Þ takes a minimum value between ϕ ¼ 0:1 and ϕ ¼ 0:5. By contrast, ϕsti gle interpolation. T. Maruyama et al. / Journal of Computational Design and Engineering 3 (2016) 250–265 259 is determined as fixed values ϕsti ¼ 0:2 � i. The angles θswE_DHM, θswE_flat 1ð Þ, θstE_DHM, and θstE_flat 1ð Þ and the angular velocities _θ sw E_flat 0ð Þ, _θ sw E_flat 1ð Þ, _θ st E_flat 0ð Þ, and _θ st E_flat 1ð Þ are also used as boundary conditions for the swing and stance foot of the DHM, respectively. The angles θswE_DHM and θ st E_DHM represent the current elevation angles of the swing and stance foot, respectively. The angular velocities are estimated by fitting a line locally to each of the angles θstE_flat ϕð Þ and θstE_flat ϕð Þ between ϕ ¼ 0:9 and ϕ ¼ 1:0. In addition, as shown in Fig. 11(b), when the DHM walks on stairs, we update θswE_flat 1ð Þ as θswE_flat 1ð Þ ¼ π=2, which represents a foot orientation parallel to the horizontal plane, to realize the sole-contact on the tread. 4.3.5. Foot collision avoidance in the case of walking on stairs (A11) By executing the previous processes (A7–A10) with next process (A12), the DHM could walk in flat, sloping, and bumpy terrains. However, if the DHM walks on stairs using only these processes, collision of the swing foot with a stair nosing may occur. Thus, to avoid collision, after determining the adapted swing ankle trajectory f 0 A ϕð Þ ¼ x 0 A ϕð Þ; y 0 A � ϕð Þ; z0A ϕð Þ� and swing foot elevation angle θswE ϕð Þ, the swing ankle height z 0 A ϕð Þ should be accordingly modified. As shown in Fig. 12(a), the nosing represents the edge of a tread. As shown in Fig. 12(a), first, the swing foot motion during one-step walking is virtually generated using both f 0 A ϕð Þ and θswE ϕð Þ. Then, as a result, the penetration depths dt1 and dt2 are obtained along with the walking phases ϕ1 and ϕ2, where the toe of the swing foot is located just under the nosing of the stairs. These penetration depths dt1 and d t 2 are estimated as dt1 ¼ distVðpswt ϕ1 � � ; nm;nÞ and dt2 ¼ distVðpswt ϕ2 � � ; nm;nþ1Þ, where pswt ϕð Þ, nm;n, and distVðpswt ϕð Þ; nm;nÞ represent the toe position of the swing foot, a nosing point estimated from the tread boundary Bm;n and the vertical distance between p sw t ϕð Þ and nm;n, respectively. Finally, as shown in Fig. 12(b), the collision-free ankle height z 00 A ϕð Þ is obtained by cubic spline interpolation of z 0 A ϕð Þ, so that it satisfies z 00 A ϕ1 � � ¼ z0A ϕ1 � � þdt1 and z 00 A ϕ2 � � ¼ z0A ϕ2 � � þdt2 at phases ϕ1 and ϕ2, respectively. The positions z 0 A 0ð Þ and z 0 A 1ð Þ and velocities v0 and v1 are also Estimating penetration depths and A nosing A nosing A nk le h ei gh t Fig. 12. Foot collis used as boundary conditions. We set v0 ¼ 0 and v1 ¼ _z0A 1ð Þ, based on our analysis in the preliminary experiment, to measure human walking motion on the stairs described in Section 5.1. As a result of this process (A11), a collision-free ankle position trajectory f 00 A ϕð Þ ¼ x 00 A ϕð Þ; y 00 A ϕð Þ; z 00 A ϕð Þ � � , which has no collisions of the swing foot with the staircase treads, can be completely determined. 4.3.6. Generating walking motion (A12) Finally, as shown in the “walking simulation” section of Fig. 2, all the joint angles of the stance and swing legs are determined based on the results of previous processes (A9–A11). As shown in Fig. 13, first, the pelvis position of the DHM moves to f 0 P ϕð Þ at the walking phase ϕ, where f 0 P ϕð Þ was already adapted to the actual walking environment by the process (A9) mentioned in Section 4.3.3. The swing and stance foot orientations are determined by applying θswE ϕð Þ and θstE ϕð Þ to each foot, which were interpolated by the process (A10) mentioned in Section 4.3.4. All the joint angles of the stance leg are then determined by solving IK analytically to achieve the current contact position pc between the sole and the walking terrain surface. pc is initialized as the heel position of the stance leg at the phase ϕ ¼ 0. Finally, all the joint angles of the swing leg are also determined by solving IK analytically to achieve f 00 A ϕð Þ. In the case of walking on stairs, f 00 A ϕð Þ has been already adapted to the stairs so as to avoid collision of the swing foot with a stair nosing in the process (A11) in Section 4.3.5. Contrastingly, in the case of walking in the other terrains, f 00 A ϕð Þ is directly copied from f 0 A ϕð Þ which has been already adapted to an actual walking environment by the process (A9) mentioned in Section 4.3.3. In this process, both leg motions are determined by solving the inverse kinematics analytically. The details of this inverse kinematics calculation are given below. As shown in Fig. 14, given a pelvis point and an ankle point, we need to compute the hip joint angle (3 DOF) and knee joint angle (1 DOF). To estimate all of these joint angles, we first determine the internal/external rotation angle of the hip joint θinterhip . The internal/external rotation angle of the hip joint obtained from the pre-generated flat walking motion of the DHM is copied to the hip joint of the DHM θinterhip . In cases Interpolating ankle position trajectory ( ) Walking phase 0 1 ion avoidance. DHM at phase A10 , A11 A9 Heel-contact Collision Toe-contact Fig. 13. Generating walking motion. Pelvis Hip joint (3 DOF) Knee joint (1 DOF) Ankle Toe Heel Fig. 14. Joint angle definition. T. Maruyama et al. / Journal of Computational Design and Engineering 3 (2016) 250–265260 where a turning or steering motion is required, the θinterhip of the stance leg is increased gradually in each frame during one-step walking until the rotation angle θrot is reached. θrot represents the angle between the current locomotion vector v and the locomotion vector vpre in the previous one-step walking. Finally, the three joint angles (the flexion/extension rotation angle θflexhip and adduction/abduction rotation angle θ add hip of the hip joint and the flexion/extension rotation angle θflexknee of the knee joint) are computed to achieve the given ankle point by simple geometric computation using trigonometric functions. In addition, as shown in Fig. 13, a collision check between the stance toe position pstt ϕð Þ and a local terrain surface P is performed at every walking step. When a collision occurs, the contact position pc is updated as pc ¼ pstt ϕð Þ to realize the transition from heel to toe contact. The local terrain surface P is estimated by applying the least-square method to the walk surface points Wk locally. 5. Results We validated our proposed modeling and simulation algo- rithm in the two types of complex laser-scanned environments, one is an indoor environment including stairs (around 42 million points were scanned by a terrestrial laser scanner (FARO Focus3D S120)) and the other is an outdoor environ- ment consisting paved slopes (around 41 million points were scanned by a terrestrial laser scanner (RIEGL VZ-1000)). 5.1. Results of 3D environmental modeling Results of the 3D environmental modeling are shown in Fig. 15. Fig. 15(a)–(e) are the modeling results from the laser-scanned point clouds of the indoor environment, and Fig. 15(f)–(h) are those of the outdoor environment. As shown in Fig. 15(b) and (g), walkable surfaces such as floors, slopes, and staircase treads are extracted as the walk surface points. Moreover, as shown in Fig. 15(c), (d), and (h), the free space and its connectivity in the environments could be successfully modeled as the navigation graph. In addition, as shown in Fig. 15(e), the boundary lines of each tread could be reproduced as the tread boundaries. These environment models were automatically generated from the original laser-scanned point clouds. 5.2. Walking simulation results in the 3D environment models Fig. 16 shows walking simulation results in the generated 3D environment models. Fig. 16(a)–(d) show the results of the walking simulation in a corridor with corners, 3-deg slope, 6- deg slope, and stairs, respectively. In the figure, the motion of the DHM was generated based on different three-reference MoCap data selected from the gait DB [29] (female aged 13, male aged 23, and female aged 72). As shown in Fig. 16, the DHM was able to walk in different complex environments automatically, while adapting its strides, turning angles, and footprints to the point clouds- based environment models. Moreover, as shown in Fig. 16 (c) and (d), the stance and swing foot of the DHM could avoid collisions with the point clouds such as slopes and stairs during the walking simulation. In addition, Fig. 16(e) and (f) show the results of the walking simulation in the case of changing stride of the DHM. As shown in the figures, our proposed simulation algorithm could reasonably recreate the oscillation of the pelvis even though the stride was changed, which has been observed as a feature of human walking [31]. This result shows the effec- tiveness of generating human-like walking motion while respecting human walking features. 5.3. Verification of the simulated walking motion of the DHM Fig. 17 shows a comparison of the DHM walking motion with MoCap data of a human subject (male, aged 23). The motion of the DHM was estimated based on our proposed walking simulation algorithm using single reference MoCap data of the subject during flat walking. We then performed a walking simulation on a 6-deg slope, and obtained the walking motion of the DHM on the slope. Finally, its joint angle patterns of the DHM on the 6-deg slope were compared with MoCap data actually measured from the subject on the 6-deg slope. Point clouds of an indoor environment Walk surface points Navigation graph (isometric view) Navigation graph (top view) Tread boundaries Point clouds of an outdoor environment Walk surface points Navigation graph Stair Treads Floor Node Edge Node Edge Tread boundary 3 deg slope 6 deg slope Fig. 15. Results of 3D environmental modeling from laser-scanned point clouds. T. Maruyama et al. / Journal of Computational Design and Engineering 3 (2016) 250–265 261 As shown in Fig. 17, the DHM can generate joint angle patterns similar to those of the human subject on a 6-deg slope. The averaged angle differences between the simulation and reference MoCap data are approximately 51 and 61 in the hip and knee joints, respectively. This shows that our proposed walking simulation algorithm can generate human-like walking motion even in the different environments based only on single reference MoCap data for flat walking for young subject. Regrettably, a comparison with an elderly person's gait on non-flat walking terrain could not be performed because Walking around corners (reference subject: female, age 72) Descending outdoor slope (3 deg) (reference subject: female, aged 13) Ascending outdoor slope (6 deg) (reference subject: male, aged 23) Ascending stairs (reference subject: female, aged 72) Walking with original stride (1.0 m) (reference subject: female, aged 72) Walking with different stride (1.3 m) (reference subject: female, aged 72) Start Goal Pelvis movement Footprints 3 deg slope Stance toe 6 deg slope StartGoal Start Goal Tread boundary Right toe trajectory Left toe trajectory 1.0 m 1.3 m Oscillation Fig. 16. Results of the walking simulation in different environments using different reference MoCap data. T. Maruyama et al. / Journal of Computational Design and Engineering 3 (2016) 250–265262 T. Maruyama et al. / Journal of Computational Design and Engineering 3 (2016) 250–265 263 MoCap data for an elderly subject acquired in non-flat walking terrain is not available in the gait database [29]. As noted in Section 2.3, it is both practically and ethically difficult to persuade elderly persons to join a prolonged experiment on walking motion in various walking environments including flat, 3-deg and 6-deg slopes, and stairs. We leave this comparison to future work. In a previous paper [6], we confirmed that the DHM could generate joint angle patterns similar to those of human subjects of different ages and genders (male aged 22, female aged 13, male aged 73, and female aged 72), at least on flat walking terrain. 5.4. Modeling and simulation performances Table 1 shows the elapsed time of the 3D environmental modeling from the original point clouds and of the walking simulation. The Point Cloud Library [32] was partly used for point cloud processing, and OpenGL was used for rendering. The 3D environmental modeling required approximately 12 s for the indoor environment and 20 s for the outdoor environ- ment, which is significantly faster than manual modeling. The elapsed time for the macro-level simulation, a prepara- tion for beginning the micro-level simulation, was less than a few three-hundredths seconds. In addition, the elapsed time for one-step walking motion simulation with 100-frame interpola- tion in the micro-level simulation required approximately 0.4 s in the indoor environment and 1.2 s in the outdoor Table 1 Elapsed time of the modeling and simulation (CPU: Intel(R) Core(TM) i7 3.30 GH Processing The indoor 42,001,736) Time Pre- process Laser scanning 5 h Registration of point clouds 1 h As-is 3D environmental modeling 12.48 s Post- process Macro-level simulation 0.001–0.02 s Micro-level simulation (generating one-step walking motion with 100-frame interpolation) 0.41 s -60 -40 -20 0 20 40 0 20 40 60 80 100 Stance phase Swing phase Jo in t a ng le ( E xt en si on + ) [d eg ] Gait cycle [%] Left Hip (MoCap) Left Hip (DHM) Left Knee (DHM) Left Knee (MoCap) Ave. difference(Hip): 5deg Ave. difference(Knee): 6deg Fig. 17. Comparison of DHM motion with MoCap data of real subject on a 6-deg slope. environments. In the outdoor environments, the elapsed time of the simulation dropped to a lower value. This is because the number of down-sampled points was greater than that of the indoor environments. This caused a high-computational load for the point clouds rendering and searching in Section 4.3.1. Therefore, our proposed walking simulation has a possibility of simulating real-time walking for one DHM in the one-floor indoor environment. However, the computational time still needs to be improved for real-time walking simulation of the DHM in more large-scale environments. 6. Conclusions In this study, we developed an algorithm for generating human-like walking motion for a DHM in various as-is environments including slopes and stairs. The walking motion of the DHM was generated based on single reference MoCap data for flat walking selected from the gait DB. The DHM could walk autonomously, while adapting its strides, turning angles and footprints to different as-is environment models automatically constructed from laser-scanned point clouds. The simulation results on a 6-deg slope were validated through comparison with MoCap data acquired for the same slope. The differences in joint angles between the DHM and MoCap data were achieved at 51 and 61 for the hip and knee joints on average, respectively. This showed that our proposed walking simulation algorithm could gen- erate human-like walking motion even in the different environments based only on reference MoCap data for flat walking. Additionally, as shown in Fig. 16, it was confirmed that the DHM could walk autonomously in indoor and outdoor environment models based on single reference MoCap data for flat walking acquired from various types of people of different ages and genders. Moreover, an algorithm for 3D as-is environmental modeling from massive laser- scanned point clouds of real environments was also proposed. The 3D environmental modeling required a few seconds, which is significantly faster than manual modeling. Addi- tionally, the elapsed time for one-step walking motion simulation was approximately one second, which is short enough for practical application. z, Memory: 32 GB, GPU: GeForce GTX 560 Ti). environment (original points: (down-sampled: 629,250) The outdoor environment (original points: 41,067,944) (down-sampled: 2,571,165) Time 1.2 h 1 h 20.4 s 0.001–0.03 s 1.15 s T. Maruyama et al. / Journal of Computational Design and Engineering 3 (2016) 250–265264 In a previous paper [6], we confirmed that the DHM could generate joint angle patterns similar to those of human subjects of different ages and genders (male aged 22, female aged 13, male aged 73, and female aged 72) on flat walking terrain. In this study, the simulated walking motion of the DHM on non- flat terrain was compared only with MoCap data for young subjects acquired in the same non-flat environment. However, as mentioned in Section 5.3, we were unable to compare the simulated walking motion of the DHM on non-flat terrain with MoCap data for elderly subjects. This validation will be addressed in future work. Moreover, in this paper, we have reported only the technologies for the accessibility evaluation, and the concrete results of the evaluation have not been shown. In future work, we plan to develop the accessibility evaluation function in terms of physical friendliness for various people of different ages, genders, and body dimensions by making use of the proposed walking simulation. In particular, we will evaluate tripping risks in the environment, based on the toe-clearance of the DHM during walking simulations. In the proposed human behavior simulation, we used a depth-first search algorithm for walking path selection. This is a “brute force” and is therefore an inefficient approach. However, as shown in Table 1, the elapsed time of this process was less than 0.1 s, so it is not a serious problem at this stage. It is possible that the elapsed time will become longer as the walking path structure becomes more complex. Therefore, to achieve efficient and reliable walking path selection in a larger-scale and more complex environment, we plan to introduce a more efficient path selection algorithm, such as an A* algorithm. Acknowledgments This work was supported by JSPS KAKENHI Grant no. 15J01552 and JSPS Grant-in-Aid for Challenging Exploratory Research under Project no. 26560168. Our preliminary experiment was conducted in cooperation with the Digital Human Research Center (DHRC) of the National Institute of Advanced Industrial Science and Tech- nology (AIST). Additionally, the RIEGL JAPAN Corporation provided the outdoor environment point clouds. We would like to express our gratitude to A. Sawatome and Y. Nakamura of the AIST DHRC, and S. Matsuda of the RIEGL JAPAN Corporation, for their support. References [1] ISO21542, Building construction –Accessibility and usability of the built environment, 2011. [2] ISO/IEC Guide 71 second edition, Guide for addressing accessibility in standards, 2014. [3] Thalmann D, Musse RS. Crowd Simulation. London: Springer; 242. [4] Duives CD, Daamen W, Hoogendoorn SP. State-of-the-art crowd motion simulation models. J. Transp. Res Part C: Emerg. Technol. 2013;35:193–209. [5] Kakizaki T, Urii J, Endo M. Post-Tsunami evacuation simulation using 3D kinematic digital human models and experimental verification. J. Comput. Inf. Sci. Eng. 2014;14(2)021010-1–9. [6] T. Maruyama, S. Kanai, H. Date, MoCap-based adaptive human-like walking simulation in laser-scanned large-scale as-built environments, Lecture Notes in Computer Science, vol. 9185, 2015, pp. 193–204. [7] Helbing D, Farkas I, Vicsek T. Simulating dynamical features of escape panic. Nature 2000;407:487–90. [8] F. Tecchia, C. Loscos, R. Conroy, Y. Chrysanthou, Agent behavior simulator (ABS): a platform for urban behavior development, in: Proceedings of the ACM Games Technology Conference, Hong Kong, Jan. 17–20, 2001, pp. 17–21. [9] J. Pettre, J.P. Laumond, D. Thalmann, A navigation graph for real-time crowd animation on multi-layered and uneven terrain, in: Proceedings of the First International Workshop on Crowd Simulation (V-CROWDS’05), Lausanne, Nov. 24-25, 2005, pp. 81–90. [10] Oesau S, Lafarge F, Alliez P. Indoor scene reconstruction using feature sensitive primitive extraction. ISPRS J. Photogramm. Remote Sens. 2014;90:68–82. [11] Nüchter A, Hertzberg J. Towards semantic maps for mobile robots. J. Robot. Auton. Syst. 2008;56(11)915–26. [12] Xiong X, Adan A, Akinci B, Huber D. Automatic creation of semanti- cally rich 3D building models from laser scanned data. J. Autom. Constr. 2013;31:325–37. [13] Rusu RB, Marton ZC, Blodow N, Dolha M, Beetz M. Towards 3D point cloud based object maps for household environments. J. Robot. Auton. Syst. 2008;56(11)927–41. [14] Xiao J, Furukawa Y. Reconstructing the world's museums. Int. J. Comput. Vis. 2014;110(3)243–58. [15] Tang P, Huber D, Akinci B, Lipman R, Lytle A. Automatic reconstruc- tion of as-is building information models from laser-scanned point clouds: a review of related techniques. J. Autom. Constr. 2010;19: 829–43. [16] Troje NF. Decomposing biological motion: a framework for analysis and synthesis of human gait patterns. J. Vis. 2002;2(5)371–87. [17] J. Min, H. Liu, J. Chai, Synthesis and editing of personalized stylistic human motion, in: Proceedings of the 2010 ACM SIGGRAPH sympo- sium on Interactive 3D Graphics and Games, Bethesda, Feb. 19–21, 2010, pp. 39–46. [18] T. Mukai, Motion rings for interactive gait synthesis, in: Proceedings of ACM Symposium on Interactive 3D Graphics and Games, San Francisco, Feb. 18–20, 2011, p.125-132. [19] Grochow K, Martin LS, Hertzmann A, Popovic Z. Style-based inverse kinematics. ACM Trans. Graph. 2004;23(23)522–31. [20] Yin KK, Loken K, Panne M. SIMBICON: Simple Biped Locomotion Control. ACM Trans. Graph. 2007;26(3)105. [21] Coros S, Beaudoin P, Panne M. Generalized biped walking control. ACM Trans. Graph. 2010;29(4)130. [22] Xiang Y, Arora JS, Abdel-Malek K. Physics-based modeling and simulation of human walking: a review of optimization-based and other approaches. J. Struct. Multidiscip. Optim. 2010;42(1)1–23. [23] R.A. Al-Asqhar, T. Komura, M.G. Choi, Relationship descriptors for interactive motion adaptation, in: Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Califor- nia, July 19–21, 2013, pp. 45–53. [24] M.P. Reed, Modeling ascending and descending stairs using the Human Motion Simulation Framework, in: Proceedings of the SAE Digital Human Modeling for Design and Engineering Conference, Göteborg, June 9–11, 2009, 2009-01-2282. [25] Rusu RB. Semantic 3D Object Maps for Everyday Robot Manipulation. Berlin: Springer, Berlin Heidelberg; 225. [26] Ramer U. An iterative procedure for the polygonal approximation of plane curves. J. Comput. Graph. Image Process. 1972;1(3)244–56. [27] VICON, [cited 2015 July 10], Available from: 〈http://www.vicon.com/〉. [28] C-Motion – Visual 3D, [cited 2015 July 10], Available from: 〈http:// www.c-motion.com/products/visual3d/〉. http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref1 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref2 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref2 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref3 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref3 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref3 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref4 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref4 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref5 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref5 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref5 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref6 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref6 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref7 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref7 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref7 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref8 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref8 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref8 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref9 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref9 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref10 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref10 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref10 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref10 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref11 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref11 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref12 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref12 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref13 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref13 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref14 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref14 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref15 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref15 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref15 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref16 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref16 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref17 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref17 http://www.vicon.com/ http://www.c-motion.com/products/visual3d/ http://www.c-motion.com/products/visual3d/ T. Maruyama et al. / Journal of Computational Design and Engineering 3 (2016) 250–265 265 [29] Y. Kobayashi, M. Mochimaru, AIST Gait Database 2013, [cited 2015 July 10], Available from: 〈https://www.dh.aist.go.jp/database/gait2013/〉 2013. [30] H. Pan, X. Hou, C. Gao, Y. Lei, A method of real-time human motion retargeting for 3D terrain adaptation, in Proceedings of the 13th IEEE Joint International Computer Science and Information Technology Conference, Chongqing, Aug. 20–22, 2011, pp. 1–5. [31] Perry J, Burnfield MJ. GAIT ANALYSIS Normal and Pathological Function, Second ed., New Jersey: SLACK Inc.; 551. [32] PCL – Point Cloud Library, [cited 2015 July 10], Available from: 〈http:// pointclouds.org/〉. https://www.dh.aist.go.jp/database/gait2013/ http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref18 http://refhub.elsevier.com/S2288-4300(15)30014-2/sbref18 http://pointclouds.org/ http://pointclouds.org/ Motion-capture-based walking simulation of digital human adapted to laser-scanned 3D as-is environments for accessibility... Introduction Related work Human behavior simulation D as-is environment modeling Digital human modeling for walking simulation 3D environment modeling from laser-scanned point clouds Down-sampling and normal vector estimation Extraction of walk surface points Navigation graph construction Tread boundary estimation MoCap-based adaptive walking simulation in as-is environments Preliminary experiment for measuring human motion in various environments Macro-level simulation Micro-level simulation based on single reference MoCap data Estimating next footprint (A7) Pre-generating virtual flat walking motion (A8) Adapting joint position trajectories (A9) Elevation angle interpolation (A10) Foot collision avoidance in the case of walking on stairs (A11) Generating walking motion (A12) Results Results of 3D environmental modeling Walking simulation results in the 3D environment models Verification of the simulated walking motion of the DHM Modeling and simulation performances Conclusions Acknowledgments References