key: cord-103249-k35o3gxe
authors: Johannsen, Leif; Potwar, Karna; Saveriano, Matteo; Endo, Satoshi; Lee, Dongheui
title: Robotic light touch assists human balance control during maximum forward reaching
date: 2019-11-20
journal: bioRxiv
DOI: 10.1101/848432
sha: 
doc_id: 103249
cord_uid: k35o3gxe

Objective We investigated how light interpersonal touch (IPT) provided by a robotic system supports human individuals performing a challenging balance task compared to IPT provided by a human partner. Background IPT augments the control of body balance in contact receivers without a provision of mechanical body weight support. The nature of the processes governing the social haptic interaction, whether they are predominantly reactive or predictive, is uncertain. Method Ten healthy adult individuals performed maximum forward reaching (MFR) without visual feedback while standing upright. We evaluated their control of reaching behaviour and of body balance during IPT provided by either another human individual or by a robotic system in two alternative control modes (reactive vs predictive). Results Changes in reaching behaviour under the robotic IPT, such as lower speed and straighter direction were linked to reduced body sway. MFR of the contact receiver was influenced by the robotic control mode such as that a predictive mode reduced movement variability and increased postural stability to a greater extend in comparison to human IPT. The effects of the reactive robotic system, however, more closely resembled the effects of IPT provided by human contact provider. Conclusion The robotic IPT system was as supportive as human IPT. Robotic IPT seemed to afford more specific adjustments, such as trading reduced speed for increased accuracy, to meet the intrinsic demands and constraints of the robotic system. Possibly, IPT provided by a human contact provider reflected reactive interpersonal postural coordination more similar to the robotic system’s follower mode. Précis Interpersonal touch support by a robotic system was evaluated against support provided a human partner during maximum forward reaching. Human contact receivers showed comparable benefits in their reaching postural performance between the support conditions. Coordination with the robotic system, nevertheless, afforded specific adaptations in the reaching behaviour.

We investigated how light interpersonal touch (IPT) provided by a robotic system supports human individuals 49 performing a challenging balance task compared to IPT provided by a human partner. 50

Background 51

IPT augments the control of body balance in contact receivers without a provision of mechanical body weight 52 support. The nature of the processes governing the social haptic interaction, whether they are predominantly 53 reactive or predictive, is uncertain. 54

If robotic systems are envisaged as the solution to future shortages in clinical staff and caregivers for the 82 purpose of augmenting of patients' mobility by a provision of balance support, they must show a responsiveness 83 to the social constraints and demands, which govern any routine physical interaction between a patient and a 84 human carer. From a scientific and engineering point of view, therefore, the principles of human-human 85 interactions during physical interactions need to be extracted and evaluated in terms of their transferability to 86 human-robot interactions as exoskeletal approaches may be unsuitable for frail individuals due the weight added 87 to the body. In physical rehabilitation, caregivers and therapists routinely provide physical assistance to balance-88 impaired individuals during postural mobilization and transfer maneuvres. In order to prevent long-term habitual 89 dependency of a patient on external balance aids and other forms of support, a therapist needs to be adopt an 90 optimum level of postural assistance that maximizes a patient's movement autonomy ('assist-as-needed'). One 91 possible approach is the provision of delibrerately light interpersonal touch (IPT) by a caregiver, which can be 92 To explore the interdependencies between CR and CP during IPT in more detail, we evaluated performance in 100 maximum forward reaching (MFR) with and without light IPT applied to the ulnar side of the wrist of 101 blindfolded CR's extended arm intended to provide a social haptic cue and impose social coordinative 102 constraints on both the CR and the CP (Steinl & Johannsen, 2017) . Interestingly, IPT reduced sway more 103 effectively when the CP had the eyes closed and their perception of CR's motion was based on haptic feedback 104 alone. In contrast, IPT with open eyes did not result in reduced sway compared with a condition in which IPT 105 was not provided (Steinl & Johannsen, 2017) . We speculated, therefore, that minimization of the interaction 106 forces and their variability at the contact location during IPT acts as an implicit task constraint and shared goal In the present study, we intended to contrast the effects of human IPT (hIPT) on CR's postural performance 112 against the effects of two different modes of robotic IPT (rIPT) and expected specific costs and benefits on body 113 sway and postural performance due to the robotic response modes. Similar to hIPT, rIPT was applied in a 114 "fingertip touch" fashion to CR's wrist without any mechanical coupling or weight support. The robotic system 115 either followed a participant reactively or predicted a participant's movement trajectory. As the coupling 116 between two humans with IPT in terms of the interaction forces is intrinsically more noisy due to each 117 individual's motion dynamics and response delays, we expected that a predictive mode of the robotic system 118 would result in a less noisy haptic coupling and therefore enhance performance in the MFR task, such as greater 119 reaching distance with less body sway. In addition, the reactive mode of the robot was supposed to be 120 advantageous over hIPT due to the fixed response delay, which would enable participants to extract own 121 movement-related information from the interaction forces for balance control. 122 123

Participants 125 parallel to the reaching direction. CP provided IPT with the right extended arm by lightly contacting the wrist at 140 its ulnar side of the CR. During IPT, CP kept the eyes open to receive visual cues of a CR's motion as would 141 the robotic systems by optical motion tracking. During the robotic IPT conditions, a single KUKA LWR4+ 142 manipulator (Augsburg, Germany) served as CP. The CP kept light contact with CR's ulnar side of the wrist. 143 CR's body sway was determined in terms of the anteroposterior (AP) and mediolateral (ML) components of the 144

Center of Pressure (CoP), as derived from the six components of the ground reaction forces and moments. 145

In in the human-robot interaction conditions, the CR's wrist was tracked by the end effector of the robotic 146 system without any mechanical coupling (Fig. 1b ). The robotic system provided contact via a hemispherical 147 rubber pad attached to a force sensor (OptoForce 3D OMD, OnRobot. Odense, Denmark; 500 Hz) on the end-148 effector, which kept the relative orthogonal distance constant. The force sensor was used to measure force at the 149 contact location. The CR's wrist position, required to control the robotic system, was measured by an 150 optoelectronic motion capture system (OptiTrack, NaturalPoint, Corvallis, OR, USA; 100 Hz). To provide 151 nearly the same feeling for the CR in both touch conditions, the CP was wearing a thin rubber glove to provide 152 similar tactile sensation to the case of rIPT where the end effector of the robot had a rubber surface (Fig. 1b) . 153

Participants' movements of the right hand were tracked with a marker-based optical motion capture system by 154 placing three reflective markers on the right hand (one on the caput ulnae/processus styloideus radii/basis and 155 two on the ossa metacarpi). Tracked hand position was sent to the robot to control the robots' movements but 156 also recorded to calculate reaching distance in the MFR end-state. The robotic control scheme required high 157 control frequencies to avoid unstable behaviors (Siciliano, Sciavicco, Villani, & Oriolo, 2009) . For this reason, 158 the robot was controlled at 500 Hz. Interaction forces were measured at the same frequency of 500 Hz, while the 159 CR's hand was tracked at 100 Hz. Hence, it was necessary to up-sample the motion tracking system to match 160 the robot control frequency. 

The unit of the IoP is bit/s and thus expresses the informational "throughput" of a participant during the 195 movement. trajectory. A constant velocity LKF assumes that the motion is generated by the discrete linear system 209 was generated at 500 Hz 215 and used to control the robotic system. The LKF was exploited to realize two different robotic modes, i.e. the 216 robotic follower and the robotic anticipatory modes. More specifically, in the rIPTfollow mode the robot 217 passively followed the wrist motion while providing a light touch. To implement a passive follower, the position 218

(Position Error: rIPTfollow AP -0.010218m, ML -0.004994 m) (Fig. 3b ) predicted by the LFK at the 219 actual time instant t was used to generate the control command described in the previous section. In this way, 220 the robotic system followed the wrist position with one sample delay (10 ms). In the rIPTanticip mode, the robot 221 predicted the future wrist position to lead the motion while providing a light touch. To realize the leading mode, 222 the LKF was exploited to make a one-step prediction of the wrist position. In particular, the predicted future 223

(Position error: rIPTanticip AP -0.012256, ML -0.007164 m) (Fig. 3a) was 224 used to generate the control command. In this way, the robot was anticipating the human motion by one sample 225 (10 ms), thereby leading the movement execution.

During the MFR task, the robotic system provided a light touch along the contact directions, while predicting 227 and following (or predicting) the participant's right wrist trajectory in the AP direction. 

). The desired contact force Table 1 summarizes all statistical comparisons. The MFR amplitude in the horizontal plane was not affected by 247 the IPT condition. All three IPT conditions resulted in comparable amplitudes (hIPT: mean=35.8 cm, SD 1.5; 248 rIPTanticip: mean=35.4 cm, SD 1.4; rIPTfollow: mean=35.1 cm, SD 1.5). Average (Fig. 4a) and peak planar 249 reaching velocity (Fig. 4b) were slower in both rIPT conditions compared to hIPT. The directional angle of 250 reaching in the horizontal plane was more straight ahead in the rIPTfollow condition (AV angle=-0.83 deg, 251 SEM 1.84) and a tendency of less lateral drift in rIPTanticip (AV angle=-1.34 deg, SD 1.92) compared to hIPT 252 (AV angle=-4.55 deg, SD 2.12). Orthogonal deviation from a straight line, in terms of both the average (Fig. 4c ) 253 and summed deviation (Fig. 4d) as well as the variability, was lower in hIPT than rIPTanticip. Path length was 254 not altered by the IPT conditions but the normalized path length indicated less curvature in rIPTfollow 255 compared to rIPTanticip (Fig. 4e) . Sway variability in either the AP or ML directions was not different between the three IPT conditions in the 260 baseline phase and the MFR end-state. During the reaching phase, however, AP sway variability was reduced in 261 both conditions involving rIPT compared to hIPT (Fig. 5a) and rIPTanticip compared to rIPTfollow. In contrast 262 , only rIPTanticip showed reduced ML sway variability compared to hIPT (Fig. 5b) . 263

The IoD differed between the three conditions in the AP direction., with the lowest scores in hIPT compared to 264 both rIPT conditions. In the ML direction, hIPT had a lower IoD score compared to rIPTanticip only (Fig. 5c) . 265

In contrast, no difference in the informational "throughput" (IoP) was observed between the three conditions 266 (Fig. 5d) . movements in a reactive fashion as well, potentially in follower mode due to visual dominance or as the more 284 optimal strategy due to the inability to stem the computational complexity of predicting CR's trajectory.

In our current study, the provision of IPT by the CP involved visual feedback of CR and his or her 286 movements.As this would be more similar to the optical tracking of CR's motion used by the robotic system. In 287 human pairs, the presence of visual feedback with habitual visual dominance is likely to turn the CP into a 288 follower of CR's movement (Steinl & Johannsen, 2017) . Assessing HHI as well as HRI in a single degree of 289 between two human individuals leader-follower relationships are not necessarily fixed. It seems to be the case, 293 however, that the more adaptive individual, for example the person on whom fewer requirements to fulfill 294 specific movement contraints are imposed, is more likely to take a follower role (Skewes, Skewes, Michael, & 295 Konvalinka, 2015) . 296

Despite impressive advances in the recent decade, current robotics engineering is still distant from developing 297 robotic systems able to assist human individudals socially, especially during postural activities and balance 298 exercises (Sheridan, 2016) . In the both rIPT conditions of the current study, the dynamics of the robotic system 299 were not independent but in one way or another a direct consequence of CR's movements. Despite the lack of 300 any real "social cognitive" capabilities of the robotic system, this fact can nevetheless be interpreted as highly 301 precise responsiveness, which a real human CP could never match. We assume that participants were not able to 302 consciously preceive any difference between the anticipatory and follower rIPT modes, just an absolute timing 303 difference of 20 ms, and therefore would not change their behaviour voluntarily. Possibly due to a shift in 304 participants from less to more reactive, feedback-dependent postural control, CRs reduced their reaching 305 velocity to adjust their movements more precisely to the current position of the robotic end-effector and for the 306 same to stay in contact with their wrist. These concerns could have been even more prominent in the rIPTanticip 307 condition than in rIPTfollow. forms of IPT, it means that IPT provided by a robotic system does not disrupt or distract the human CR. During 314 the reaching phase, the facilitation of stabilization of body sway by rIPT tended to surpass the effect of hIPT, especially in a robotic control mode involving anticipation. This shows that rIPT does not destabilize CR's 316 postural behaviour but can lead to a further reductions in behavioural variability. Nevertheless, human CRs 317 altered their MFR behaviour when IPT was provided not by the human partner but by the robotic system. The 318 most obvious changes were general reductions in the average and peak planar MFR velocity with rIPT. As body 319 sway tended to be reduced in these situations, these adjustments to the robotic CP could reflect a trade-off 320 between speed and accuracy [Fitts, 1954] . According to this interpretation, participants may have effectively 321 controlled sway variability in order to meet any perceived difficulty increase in rIPT resulting from "hardware" 322 constraints imposed by technical limitations of the robotic system and "soft" constraints in terms of fulfilling the Assist-as-needed robotic approaches translate into corrective forces keeping an individual's body or limbs 329 within an initially defined "normal" range. In contrast to such "positive" force feedback, in which a robotic 330 system aims to guide a participant's limb along a specific trajectory by applying a corrective force, our 331 deliberately light interpersonal touch paradigm could be described to act with "negative" force feedback. This 332 means that if participants stray from a reaching trajectory, they will perceive a momentary reduction in touch, 333 which might cue them to perform a subtle correction with the intention to minimize contact force variability. 334

The robotic system in our study was controlled according to this principle, and we believe it imitated CR's 335 behaviour more naturally. At the same time, the reaching trajectory was not prespecified within the robotic 336 system but emerged as a compromise between the CR and the respective CP. In this sense, the CR's movement 337 range remains completely unconstrained. Any constraints result from the "social" context of the HHI or HRI In this context it is remarkable that rIPTfollow led to the straightest forward reaching trajectories with least 359 amount of medial drift. This could mean that a robotic system that emphasizes a reactive follower strategy is a 360 better haptic "communicator" in the sense that it made participants to "listen" more closely to the haptic 361 feedback they received. Possibly, participants interpreted rIPT as more reliable as a relative spatial reference and 362 therefore adjusted their reaching movements more in a feedback-driven manner. In contrast, although 363 rIPTanticip also tended towards a more straight ahead reaching movement, the condition showed the greatest 364 and most variable orthogonal deviation from a straight line connecting the start and end point. The robotic 365 system in leader mode could have actually "misguided" participants in the sense, that it tried to anticipate a 366 participant's next position and so reinforced a participants' tendency to deviate from their current trajectory. Beneficial deliberately light interpersonal touch for balance support during maximum forward reaching is easily 380 provided by a robotic system even when it is mechanically uncoupled to the human contact reveicer. This effect 381 does not rely on the system's capability to predict the future position of the contact receiver's wrist. The effects 382 the uncoupled robotic IPT in reactive following mode were comparable to human IPT on most parameters. As 383 the robotic system itself was not designed for any form of "social" cognition or explicit haptic communication, 384 our study nevertheless demonstrates that robotic IPT can be used to implicitly "nudge" human contact receivers 385 to alter their postural strategy for adapting to the robotic system without any decrements in their postural 386 performance during maximum forward reaching. 387 

Perspectives on human-human sensorimotor interactions for the design of 470 rehabilitation robots

The uncontrolled manifold concept: identifying control variables for a 472 functional task

Human-Robot Interaction: Status and Challenges

Robotics: modelling, planning and control

Synchronised and complementary coordination 478 mechanisms in an asymmetric joint aiming task

Assist-481 as-Needed Robot-Aided Gait Training Improves Walking Function in Individuals Following Stroke

Robotic 484 Assist-As-Needed as an Alternative to Therapist-Assisted Gait Rehabilitation

Interpersonal interactions for haptic guidance during maximum forward 487 reaching

Haptic communication between humans 489 is tuned by the hard or soft mechanics of interaction

We thank Prof. G. Cheng, Prof. M. Buss, Prof. S. Hirche, Dr. K. Ramirez-Amaro, and J. R. Guadarrama Olvera 390 for providing the experimental infrastructure and S. M. Steinl