key: cord-0727976-kzrobm0e
authors: Jeong, Hyoyoung; Lee, Jong Yoon; Lee, KunHyuck; Kang, Youn J.; Kim, Jin-Tae; Avila, Raudel; Tzavelis, Andreas; Kim, Joohee; Ryu, Hanjun; Kwak, Sung Soo; Kim, Jong Uk; Banks, Aaron; Jang, Hokyung; Chang, Jan-Kai; Li, Shupeng; Mummidisetty, Chaithanya K.; Park, Yoonseok; Nappi, Simone; Chun, Keum San; Lee, Young Joong; Kwon, Kyeongha; Ni, Xiaoyue; Chung, Ha Uk; Luan, Haiwen; Kim, Jae-Hwan; Wu, Changsheng; Xu, Shuai; Banks, Anthony; Jayaraman, Arun; Huang, Yonggang; Rogers, John A.
title: Differential cardiopulmonary monitoring system for artifact-canceled physiological tracking of athletes, workers, and COVID-19 patients
date: 2021-05-12
journal: Sci Adv
DOI: 10.1126/sciadv.abg3092
sha: 3b61fe435dd7399b240a2f45a0ddf5a41229cc71
doc_id: 727976
cord_uid: kzrobm0e

Soft, skin-integrated electronic sensors can provide continuous measurements of diverse physiological parameters, with broad relevance to the future of human health care. Motion artifacts can, however, corrupt the recorded signals, particularly those associated with mechanical signatures of cardiopulmonary processes. Design strategies introduced here address this limitation through differential operation of a matched, time-synchronized pair of high-bandwidth accelerometers located on parts of the anatomy that exhibit strong spatial gradients in motion characteristics. When mounted at a location that spans the suprasternal notch and the sternal manubrium, these dual-sensing devices allow measurements of heart rate and sounds, respiratory activities, body temperature, body orientation, and activity level, along with swallowing, coughing, talking, and related processes, without sensitivity to ambient conditions during routine daily activities, vigorous exercises, intense manual labor, and even swimming. Deployments on patients with COVID-19 allow clinical-grade ambulatory monitoring of the key symptoms of the disease even during rehabilitation protocols.

The emergence of wearable technologies capable of multimodal, clinical-grade monitoring of physiological health increases the demand for sensors, systems, and data analytics approaches that enable reliable, continuous operation during natural daily activities. By comparison to traditional devices that loosely couple to the wrist, skin-mounted technologies offer vastly superior measurement capabilities due to their persistent, intimate interfaces to the body (1) (2) (3) (4) (5) . This mode of operation can support a range of clinically standard diagnostic assessments, such as those based on electrocardiography (2, (6) (7) (8) (9) (10) (11) , photoplethysmography (10) (11) (12) (13) (14) (15) , arterial tonometry (16) (17) (18) (19) (20) , and others (21) (22) (23) (24) (25) (26) (27) (28) (29) (30) (31) (32) (33) (34) . A recent set of important capabilities follows from wide-bandwidth measurements of subtle motions and vibrations of the surface of the skin [i.e., mechano-acoustic (MA) responses] (35) (36) (37) that arise from activities of internal organs and accelerations due to global movements of the body (36, 38, 39) . Skininterfaced devices for such purposes use precision, high-bandwidth accelerometers based on microelectromechanical system technologies in layouts that optimize sensitivity to motions of the surface of the skin across a broad range of frequencies, from nearly zero to several thousand hertz. The resulting data reflect not only bulk motions of the body, as with conventional wearable devices, but also features from a broad range of body sounds, as with digital stethoscopes, but impervious to ambient sounds. Additional information appears in a range of frequencies between these limits. For example, when mounted on the neck or the chest, the recordings enable detailed assessments of cardiac activity from motions of the heart and from pulsatile flow of blood through near-surface arteries, of respiratory cycles from chest wall movements, of respiratory sounds from airflow through the lungs and trachea, of swallowing behaviors from laryngeal motions and actions of the esophagus, of vocalization patterns from vocal fold activation, and of movements and changes in orientation of the core body. Distinct features in the temporal and spectral characteristics of these processes yield insights into physical activity and health status via a rich range of conventional [e.g., heart rate (HR)] (36, 40) and unconventional (e.g., coughing frequency) metrics 1 Querrey Simpson Institute for Bioelectronics, Northwestern University, Evanston, IL 60208, USA. 2 Sibel Health, Niles, IL 60714, USA. 3 Department of Materials Science and Engineering, Northwestern University, Evanston, IL 60208, USA. 4 Department of Mechanical Engineering, Northwestern University, Evanston, IL 60208, USA. 5 Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, USA. 6 Department of Biomedical Engineering, Northwestern University, Evanston, IL 60208, USA. (36, 37, (41) (42) (43) , in a seamless manner, without privacy concerns that would follow from use of microphones or other recording devices.

Through these mechanisms, a single device in a sealed, waterproof package that requires only mechanical coupling to the skin can produce a powerful breadth of health-related information. An important caveat is that the diverse range of MA signals contribute to a single stream of time series data in a temporally overlapping fashion. Advanced data filtering and analytics approaches can separate and quantify different characteristic events on the basis of unique temporal and spectral features, but they fail to operate reliably in many scenarios of practical interest. Particular challenges arise when different activities with similar spectral content occur simultaneously. These circumstances render digital signal processing approaches ineffective. For example, respiration rate cannot be determined accurately while running. Related types of motion artifacts are fundamental limitations to both consumer wearables mounted on the wrist and clinical-grade wired monitoring systems.

The work presented here overcomes this limitation through advanced concepts in system designs and optimized choices in anatomical mounting locations, at the hardware level without the need for complex and often ineffective digital signal processing strategies. The approach exploits a pair of time-synchronized, high-bandwidth accelerometers [inertial measurement units (IMUs)] at opposite ends of a skin-interfaced device that locates one of the IMUs at the suprasternal notch (SN) and the other at the sternal manubrium (SM). Differences in movements of the skin associated with cardiac and respiratory activity between these regions lead to differences in signals captured by these IMUs. By contrast, overall movements of the neck and the core of the body produce nearly identical responses. As a consequence, simple differential measurements cleanly eliminate common mode features, thereby separating signals associated with cardiopulmonary and related processes from those due to body movements. An additional benefit of this architecture is that temperature sensors integrated in these IMUs can be used in a similar differential manner to yield estimates of core body temperature, largely independent of the ambient. Here, careful choices in thermal aspects of the device layout, rather than intrinsic anatomical gradients, produce the necessary differential responses.

The following sections present (i) designs of automated devices that incorporate matched pairs of high-bandwidth IMUs with optimized soft mechanics for high measurement sensitivity and accurate time synchronization across the SN and SM; (ii) results of spatiotemporal mapping of movements of the skin at this region of the anatomy during cardiac and respiratory activity; (iii) examples of modeling and design approaches for exploiting these IMUs in dual temperature sensing of core body temperature, with minimal influence of the thermal ambient; (iv) demonstrations of continuous, differential measurements of temperature, HR, and respiratory rate (RR) across a range of vigorous activities and conditions, with benchmarking against the most accurate commercial sensors; and (v) illustrations of the use on patients recovering from COVID-19 infections to track key symptoms of the disease even during intense physical rehabilitation protocols.

The platform exploits a thin, flexible printed circuit board (fPCB) in an open architecture, with an elastomeric encapsulation structure that completely seals the system to physically isolate the electronics from the environment and to facilitate sterilization for reuse. The layouts yield soft mechanical characteristics for comfortable mounting on the skin, even at sensitive regions of the body. A touch-free docking interrogator supports wireless charging and initiates data downloads in an automated fashion to eliminate user burden. Figure 1A shows a device mounted on the base of the neck, positioned to span the SN and SM. This unique anatomical location allows for measurements of a rich range of biophysical information, from activity of the cardiopulmonary system and movements of the core body to a diverse collection of processes across the thoracic cavity, esophagus, pharynx, and oral cavity related to respiration, speech, swallowing, wheezing, coughing, and sneezing. Figure 1B presents an exploded-view schematic illustration of the soft enclosure and the fPCB with passive/active chip-scale components. Top and bottom encapsulating films of a silicone elastomer (thickness, 0.3 mm; Silbione RTV 4420) mechanically isolate the active parts of the systems in a sealed enclosure that allows operation even when submerged in water or exposed to sweat. The design is also compatible with U.S. Centers for Disease Control and Prevention guidelines for cleaning and disinfecting using 70% alcohol solutions ( fig. S1 ) (44) . The fPCB exploits a copper (12 m)-polyimide (PI; 25 m)-copper (12 m) laminate (DuPont, AP7164R) patterned to define conductive traces with widths of 80 and 150 m. The layout ( fig. S2 ) includes separate islands for the circuit components (main body), each of the two IMUs (IMU1 and IMU2), and a wireless charging coil. Serpentine-shaped traces interconnect these islands to mechanically decouple the IMUs from one another, as necessary in precision, differential measurements of MA signals at the surface of the skin. Specifically, two pairs of narrow, filamentary serpentine structures electrically connect the IMUs to the main body in a manner that minimizes mechanical constraints. The fPCB also includes multiple zones to allow for static bending during an assembly process that folds the system into a compact configuration. The image in Fig. 1C shows the overall size relative to a U.S. quarter (diameter, 24.26 mm). The dimensions of the encapsulated device are 46 mm by 22 mm; its thickness is less than 9 mm, and its weight is less than 6.35 g.

Experimental studies (Fig. 1D ) and finite element analysis (FEA) computations ( Fig. 1E ) confirm that the strains in the copper of the serpentine interconnects remain below fracture limits ( = 1%) throughout the assembly process and during operation under different types of external loads: 90° torsional [the left frame in Fig. 1 (D and E) ], 45% tensile [the middle frame in Fig. 1 (D and E) ], and 180° bending [the right frame in Fig. 1 (D and E) ] deformations. This soft, stretchable design can accommodate deformations of the skin at the SN without fatigue or fracture and with minimal irritation and discomfort. Stiffening layers beneath critical regions of the platform reduce the probability for bending/stretchinginduced damage of the solder joints. The result is a mechanically robust platform that enables high sensitivity measurements of both subtle motions of the skin and full body kinematics across a broad range of frequencies. Descriptions of additional design features, including those related to thermal measurements, appear subsequently.

The block diagram in Fig. 1F summarizes the overall system operation. The three main components include the device, a tablet with a customized app as a user interface, and a cloud platform for data storage and analytics. The device uses a BLE SoC (Bluetooth Low Energy System on a Chip) (Nordic Semiconductor, nRF52840), a PMIC (power management integrated circuit) (Texas Instruments, BQ25120), a 4-gigabit NAND flash memory (Micron, MT29F4G01), and two identical IMUs, each with an embedded temperature sensing unit (STMicroelectronics, LSM6DSL). Wireless charging involves voltage and current protection as support for a 75-mA·hour lithium polymer battery. The user interface allows time-synchronized operation of up to 12 devices, simultaneously. Although not explored in the following experiments in this study, this feature supports monitoring of social interactions and/or capture of MA signals at multiple body locations ( fig. S3 ). Figure S4 describes the state diagram of (E) Finite element modeling of the mechanics for the deformations in (D). The contour plots show the maximum principle strains in the metal layer of the serpentine interconnects for twisting (left), stretching (middle), and bending (right). (F) Block diagram of the system operation. A tablet provides an interface for operating the device, wirelessly downloading the data from the device, and transmitting these data to a cloud server through a cellular network. Processing on the cloud platform yields vital signals (HR, respiration, and body temperature) and other metrics of interest (cough count and physical activity). Photo credit: Hyoyoung Jeong, Northwestern University. the system, to illustrate behaviors before and after configuration, followed by deployment on a subject. The diagram also shows operation during data collection, charging, and data transfer.

Full automation of the key operational steps minimizes user burden, of particular importance for use with patients with COVID-19, as described subsequently. The user simply mounts the device during use and places it on the wireless charging platform when removed. The sensor continuously stores data from both accelerometers onto the internal memory module when not on the charging platform; when on this platform, the device charges and simultaneously streams data to a user interface device via Bluetooth protocols. The user interface then passes data to a cloud hub for signal processing to extract various physiological information, including cough count, RR, HR, activity level, body orientation, and calibrated body temperature. The cloud hub is HIPAA (Health Insurance Portability and Accountability Act) compliant, and the interface application uses HTTPS transport layer security (TLS 1.2) with an algorithm for encryption/decryption for the application programming interface and a standard for in-storage encryption (AES-256).

The simplest consequence of the dual-sensing architecture is in temperature measurements that approximate the temperature of the skin (T skin ), largely unperturbed by the ambient (T amb ), following schemes described previously in other contexts (45, 46) . Here, sensors embedded in IMU1 and IMU2, in a configuration illustrated in Fig. 2A , yield temperatures with repeatability of 0.004°C every 5 s (adjustable up to a 52-Hz sampling rate). IMU1 rests directly adjacent to the skin, separated only by the thin bottom encapsulation layer (0.3-mm-thick silicone elastomer). A 6-mm-thick thermally insulating foam (polyurethane mixture) with a metallic film (12-m-thick aluminized polyethylene) minimizes coupling to the environment via convection, conduction, and radiation. The temperature at IMU1 depends strongly on the core body temperature, modulated by the effective thermal properties of the tissues and the ambient conditions (47) . IMU2 resides on the outward-facing side of the device, with only the top encapsulation layer above, to maximize and coupling to the environment (Fig. 2 , B and C). The multiple underlying layers, including the adhesive film, bottom encapsulation, fPCB, battery, and thermal insulating foam limit heat transfer from the skin to IMU2.

Transient heat transfer analysis associated with three-dimensional (3D) thermal conduction and natural convection quantifies these effects. The boundary conditions include a constant temperature at the bottom surface of the tissue layer (T core ) and convective coupling to the ambient air at the free surfaces (T amb ). The parameters include the room temperature, T amb = 18° to 24°C, and the convection coefficient, h = 5 to 30 W m −2 K −1 . Figure 2D highlights temperature distributions across the skin and regions surrounding the device for T core = 37°C, T amb = 22°C, and h = 10 W m −2 K −1 . The temperature profile along the AB cross section shows results inside the device with T core = 37°C (Fig. 2E ) and with T core = 38° to 40°C ( fig. S5 ). Figure 2F summarizes the differences in temperature between the two IMUs (T diff ) for T core = 37°C. As might be expected, reducing T amb and/or increasing h increases T diff . As specific examples, for h = 5 W m −2 K −1 , T diff = 1.72°C when T amb = 24°C and T diff = 2.52°C when T amb = 18°C. In addition, for h = 30 W m −2 K −1 , T diff = 3.80°C when T amb = 24°C, but T diff = 5.55°C when T amb = 18°C. In the same way, fig. S6 shows temperature differences under the condition of T core = 38° to 40°C, with associated temperature distributions in fig. S7 . The analysis also quantifies the effects of changes in the ambient temperature, the core body temperature, the convection coefficient, and other key parameters. Measurements of differential temperature together with subject-specific thermal models yield robust estimates of core body temperature.

A simple demonstration involves a subject wearing a device in an environment with an ambient temperature of 18.2°C, then moving between areas with temperatures of 21.3° and 19.5°C every 3 to 8 min, and lastly remaining in place as the ambient temperature rises from 19.5° to 24.2°C for 7 min. The results for temperatures recorded from IMU1 and IMU2 appear in the top graph in Fig. 2G . The middle graph shows the differential temperature. A subject-specific model converts these temperature measurements into estimates of core body temperature (third row in Fig. 2G ), determined by eq. S17 in the "1-D analytical model for the thermal characteristics" section in Supplementary Materials and Methods, Figure  S9 compares the results to the core temperature estimated from measurements at IMU1 and IMU2. The differences remain less than ~0.5°C across ambient temperatures from 19.5° to 24.2°C. Figure S10 shows Bland-Altman plots of the data. Sensing with only IMU1 (red) yields a mean difference of 3.81°C and an SD of 0.22°C compared to the oral measurement; the dual-sensing approach (blue) yields a mean difference of 0.01°C and an SD of 0.18°C. A 1D heat transfer model for analytics and 3D FEA model of the temperature dynamics (fig. S11) can capture essential aspects of these demonstrations. The analytical and 3D FEA results agree well over the range of different ambient temperature scenarios with relevant heat convection coefficients and the core temperature (36.3°C), similar to the experimental results in Fig. 2G (fig. S11B ).

Dual temperature sensing relies critically on design choices that yield different levels of sensitivity to temperatures of the body and the ambient for IMU1 and IMU2. For dual MA sensing, differential responses arise mainly from spatial gradients in motions across the mounting location, specifically those from the SN, the location of IMU1, and from the SM, the location of IMU2 (2.5 cm below the IMU1). Spatiotemporal maps of motions of this region of the anatomy determined by 3D particle tracking velocimetry (3D-PTV) provide quantitative insights into differential motions associated with respiratory and cardiac activities at these two locations and adjacent regions. 3D-PTV relies on optical techniques to track the Lagrangian paths of fiducial marks on the skin, in 3D using stereoscopic imaging (48, 49) , in a way that recapitulates the point-measurement modality of the IMUs. Here, 3D-PTV can capture the essence of dual sensing from the SN and SM by recording from four time-synchronized, high-speed cameras, each at a frame rate of 200 frames per second (fps) (fig. S12) , and track motions across the neck, including regions of the SN and SM (Fig. 3A) . Displacement and vector contour fields follow from interpolation of fiducials at each frame based on Delaunay triangulation (Fig. 3B) . The results featured here correspond to representative velocities measured near the peak of a cardiac cycle, relative to those between cycles, for a subject at rest. The results reveal notable differences between motions at the SN and SM, as the basis for differential detection. (C to E) shows additional detail, corresponding to z-axis displacement profiles through several cardiac cycles during a breath hold after a brief period of exercise (20 push-ups). The peak displacements at the SN are ~50% larger than those at the SM (Fig. 3C ). By contrast, the displacements associated with body motions are nearly identical (as expected, but not explicitly shown here), thereby allowing for efficient subtraction. The differential result appears in Fig. 3D . Figure 3E shows a color contour plot of z-axis displacements at the peak of the cardiac cycle highlighted by the arrow in Fig. 3D (fig. S12 , C and D, and movie S1).

Similar considerations apply to differential dynamics associated with respiration. Figure 3 (F to H) and fig. S12 (E and F) summarize the displacement distributions for three cycles of breathing while slightly swinging back and forth along the z axis. Figure 3F shows motions at the SN and SM, where responses include contributions from body motions and respiration for each case. The differential result (Fig. 3G ) largely isolates the respiratory signals (fig. S12, E and F). Note that the small periodic features in these data arise from cardiac activity. A color contour plot of z-axis displacements at peak inhalation further highlights the spatial gradients that enable differential detection ( Fig. 3H and movie S2 ). Data captured with the devices show similar trends (fig. S13), and simple 1D analytical models (fig. S14; see the "Analytical modeling of differential accelerometry" section in Supplementary Materials and Methods) can capture essential aspects of these behaviors ( fig. S15 ) that reveal a clear basis for differential detection at the SN and SM.

As verified by 3D-PTV, cardiac and respiratory activities create motions that have different amplitudes at the SN and the SM. Similar amplitudes result from movements of the core body. By consequence, simple subtraction of MA signals measured at these two locations greatly improves the accuracy and reliability of measurements of respiratory and cardiac activity by eliminating large, common-mode features that result from body motions. Figure 4 summarizes results captured using a device platform that incorporates IMUs with capabilities in high-fidelity three-axial accelerometry.

The flow chart in fig. S16A illustrates the approach for calculating the RR [respirations per minute (RPM)] from the differential data. The algorithm selects and performs a weighted average of the five highest energy components (minimum threshold of 50% of the maximum energy) in the frequency spectrum across the range of interest for the RR (6 to 60 RPM) within 1-min time windows (fig. S16F). As in fig. S16 (B and C), the differential signal largely eliminates common-mode "noise" associated with walking. Other signal components such as those due to cardiac activity lie outside the frequency range associated with respiration and/or have power below the threshold ( fig. S16, D and E) .

The flow chart in fig. S17A highlights the corresponding algorithm for HR [beats per minute (BPM)]. As with the RR, the differential data remove features from walking, running, jumping, and related activities. Band-pass filtering of the frequency spectra for 1-min time windows with cutoff frequencies of 45 and 170 BPM eliminates low-frequency signals from slow body processes and high-frequency content from vocalization and related events (fig. S17, B and C). This frequency envelope captures essential features associated with the S1 peaks associated with cardiac sounds, equivalent to those observed in seismocardiograms (SCGs) (fig.   S17D ). The frequency with the maximum energy and those with at least 80% of the maximum energy serve (fig. S17E) as the basis for a weighted average to determine the HR. Figure 4 (B to D) highlights results obtained during sitting, walking, running, and jumping. The first instance involves resting in a chair with a controlled RR of 6, 10, 12, 15, 20, 30, and 35 RPM (0 to 7 min). The subject intentionally controls the exhale/inhale (1:1 ratio) time with a timer while moving. Next, the subject walks (8 to 21 min, 90 steps/min with 50-cm average stride lengths), runs (22 to 29 min, 180 steps/min with 85-cm average stride lengths), and jumps (31 to 36 min, vertical jumps every 2 to 3 s at approximately 40-cm height), all under similar controlled RR. Walking and running generate repetitive sequences of high-amplitude, impulse signals that dominate the data from IMU1 (red) and IMU2 (black). The differential signals, by contrast, feature a clear, periodic response associated with respiration (15 BPM; blue), as shown in the left frame in Fig. 4B (purple dashed region in Fig. 4A ) and the middle frame in Fig. 4B (yellow shaded region in Fig. 4A ). This differential signal also contains information on cardiac activity, as prominent S1 and S2 peaks of an SCG (1.5 s; green dashed region in the middle frame in Fig. 4B ). Figure 4 (C and D) compares RR and HR results extracted on the basis of normal and differential approaches. In the absence of body motions (e.g., sitting), the values are similar [blue shaded region in Fig. 4 (C and D) ]. During ambulatory motions (walking, running, and jumping), results from single-accelerometer data (black dot) are highly variable compared to those from differential data (blue dot) for both HR and RR (yellow, red, and green shaded region in Fig. 4 (C and D) ]. Differential sensing yields accurate results not only for walking and running but also for jumping. Figure S18 S19 ). When comparing single-and dual-sensing performance based on the SD of the extracted RR and HR, the differential signal from dual sensing shows an improvement of 77 and 79% over RR and HR from single-sensing data, respectively.

Athletic competition, fitness training, manual labor, and related activities create daunting challenges for accurate measurements of RR and HR because of fast, dynamic, and highly variable large-amplitude accelerations of the body. The dual-sensor platform offers powerful capabilities in these and other contexts. Figure 5 highlights examples in cycling, playing basketball, and swimming. For exercise on a stationary bike (24 min ; Fig. 5, A and B, and figs. S20 and S21) , both the single-sensing (IMU1) and dual-sensing (differential) data yield HR results that match those obtained with a reference device (General Electronics, Dash3000). The RR values are also similar because of the limited effects of motion artifacts in this scenario. By contrast, for subjects playing basketball (11 min), differential sensing uniquely provides reliable measurements of HR and RR, as might be expected on the basis of controlled studies described previously During ambulatory motions, the single-accelerometer data yield unreliable values of RR. The differential signals (blue dots) yield accurate respiration rates, consistent with ground truth (green triangles). The red arrow indicates the time frame of (B). (D) Single-accelerometer data provide the HR reliably while the subject sits still. During ambulatory motions, the single-accelerometer data (black dot) yield unreliable values compared to those from differential signals (blue dot) and from ground truth (green triangle). Signals associated with tapping between transitions cause aberrant values. The red arrow indicates the time frame of (B).

( Fig. 5, C and D) . Further benefits appear during extreme motions (3.5 to 5 min in Fig. 5D ). The water-tight encapsulation and internal nonvolatile memory allows use in aquatic sports, as illustrated during swimming [5 min in Fig. 5 (E and F) ]. When motion artifacts and the target signal are in a similar frequency range, the differential measurement approach is particularly valuable. During swimming, RR calculations that use the signal from a single IMU are dominated by responses associated with swimming strokes. Our algorithm processes the results as outliers because of the large amplitudes of these accelerations, which are inconsistent with respiration (Fig. 5F ). The differential signal from the dual sensor greatly minimizes signals associated with swimming strokes, thereby yielding clear features associated with cycles of exhalation and inhalation and enabling calculations of the RR. The differential signal yields accurate respiratory 

Inhale Exhale S1 S1 Dual (differential) (E) Image of the dual-sensing device on the SN/SM while swimming. (F) Comparisons of RR and HR determined with the dual-and single-sensing data while swimming for 5 min. (G) Representative z-axis acceleration data acquired from the dual-sensing device during swimming. Accelerations measured from IMU1 (red), IMU2 (black), calculated differential signal (blue), and baseline of the differential signal (light blue). (H) Magnified data associated with the differential signal (blue) and its baseline (light blue) from the area highlighted by the green box (G). Photo credit: Hyoyoung Jeong and Yoonseok Park, Northwestern University. activity, although the patterns of breathing and swimming occur in the same frequency range, as shown in Fig. 5 (G and H) and movie S3. A similar demonstration, highlighted in fig. S22 , involves pushups performed with controlled breathing out of phase with the push-up cycle, such that the RR signal cannot be distinguished from the periodic body motions using data from either IMU1 or IMU2. The differential signal, by contrast, shows clear features associated with exhalation/inspiration, well matched to the periodicity and the amplitude of controlled breathing.

Worker health represents another area of opportunity given the need to continuously monitor key cardiopulmonary parameters in hostile environments (50) . Demanding occupations that involve work in construction, mining, firefighting, and related areas could benefit from noninvasive, high-fidelity monitoring systems to detect fatigue, heat exhaustion, and performance in ways that are seamless and compatible with high motion artifacts and extreme ambient conditions in temperature, sounds, and other. Figure S22 of manual labor including hammering nails, carrying boxes, and shoveling dirt. Data in fig. S23A show that the differential signal exhibits clear features of cardiac activity, otherwise hidden by the strong, impulsive features associated with hammering. Similarly, respiratory features can be easily extracted even during large a n d irregular signatures of body movements in these cases ( fig. S23, B and C) .

An area of urgent interest is in digital monitoring of the key symptoms of patients with COVID-19 to track the progress of recovery and the response to therapies in the hospital and the home. In addition to RR, HR, body movements, and body temperature, measurements also capture the intensity and frequency of coughing, talking, and laughing events. Collectively, these factors are important for symptomatic evaluation of the disease (51) and for indirect assessments of aerosol production. Studies reported here involve a COVID-19positive patient (49 years old; female; height, 170 cm; weight, 107 kg; type 2 diabetes mellitus, obesity, hypertension, and cerebrovascular accident in 2018) provided with the dual-sensing platform and instructions for use in recording over the course of 8 days, as shown in Fig. 6 . The patient captured 171 hours of data from 5 days within this time frame, including periods of dry cough and shortness of breath with oxygen therapy during the recovery phase. Figure 6 (A and B) shows a decreasing trend in cumulative cough count, along with the RR during this same interval, where the blue dots represent 5-min averages and the orange line shows the data after processing with Savitzky-Golay smoothing. Figure 6C summarizes the HR, where the black dots and red line show similar averages and smoothed results, respectively. Figure 6D presents the activity level (green bar) calculated by integrating the spectral power across a frequency range from 1 to 10 Hz (52, 53) and the estimated core body temperature (red line). Daytime corresponds to the time interval between 6 a.m. and 6 p.m., while other times are considered night. Average body temperature recorded from the first day (37.5°C) compared to the eighth recovery day (37.0°C) shows a decrease of 0.5°C. This recovery period includes a regimented and intense set of physical rehabilitation protocols. The ability track vital signs and key symptoms throughout could provide actionable clinical information on recovery and patient readiness to return home. Widescale deployment of this technology could improve patient care, aid in managing the pandemic, and also enhance our understanding of the disease.

This paper presents a low-profile, lightweight, flexible, and wireless sensor that intimately couples to the skin as a dual measurement interface to the SN and SM with modalities for differential sensing of temperature and MA signatures of body processes. The results allow for measurements of a broad range of physiological parameters and activity behaviors that overcome a fundamental challenge in nearly every existing monitoring system: motion artifacts. Specific examples reported here include tracking of cardiac activity, respiratory activity, respiratory sounds, body temperature, and overall activity across a range of controlled settings and natural activities in sports, manual labor, and clinical medicine. These technologies and underlying ideas have many other implications. Examples include rehabilitation for patients with aphasia and/or dysphagia, where measurements of vocal activity and swallowing are possible during daily life, outside hospitals or rehabilitation clinics, of particular relevance to stroke survivors and patients with chronic obstructive pulmonary disease. Capabilities in tracking these processes without privacy concerns associated with microphone recordings and in a manner that is independent of ambient sounds represent key features of the approach. The availability of multiaxial information, including three-axis acceleration measurements, three-axis gyroscope data, and three-axis magnetometer information, suggests additional opportunities for these same platforms. Examples include quantitative measurements of neck movements ( fig. S24 ) for patients recovering from cervical spine surgery by using accurate vector data between IMU1 and IMU2, as well as full-body motion detection followed by full-body motion reconstruction for rehabilitation or early-stage atypical motion diagnosis for cerebral palsy.

Top and bottom molded layers of a low-modulus elastomer (Silbione 4420; each 300 m thick) form a soft encapsulating structure for the electronics. The fabrication process involved placing the electronics onto the bottom layer and then casting a uniform overcoat of a liquid prepolymer to a silicone elastomer (Ecoflex 0030). Mounting the top molded layer with a spacer on each of the short sides of the mold and clamping the assembly together enclosed the system for thermal curing at 70°C in an oven for 20 min. Cooling to room temperature, removing the device, and eliminating excess elastomer from the perimeter using a die cutter completed the process. at the frame rate of 200 fps. The process focused on tracking of 300 fiducial points marked in a grid pattern across the neck covering the SN, the SM, and adjacent areas. The investigation volume was 10 cm by 8 cm by 10 cm illuminated by six arrays for 600 lumen lightemitting diode light bars. Preprocessing, calibration, 3D reconstruction, tracking, and postprocessing used customized 3D-PTV code. Image sequences were preprocessed by subtracting the background noise and enhancing the contrast. 3D calibration exploited the structurefrom-motion technique from multiple views. After removing effects of lens distortion, intrinsic parameters of a single camera were estimated using the checkboard calibration method. Extrinsic parameters of all four cameras, including 3D translation and rotation matrices, were obtained by using a sparse set of points matched across the views. Once all camera parameters were estimated, a dense set of fiducial points across multiviews were detected in a subpixel level and reconstructed in 3D coordinate. 3D reconstructed fiducial points were tracked using the Hungarian algorithm and linked by performing a five-frame gap closing to produce long trajectories. Displacement, velocity, and Lagrangian acceleration were filtered and computed using fourth-order B splines. 3D displacement and vector contour fields were obtained by interpolating scattered fiducial points at each frame based on the Delaunay triangulation. Image sequences during cardiac activities (movie S1) were magnified using the Eulerian video magnification method (54) . More details on 3D-PTV can be found in (55, 56) .

A double-sided medical silicone adhesive (3M, 2477P) secured the sensors to the neck area (aligned IMU1 on the SN and IMU2 on the SM). The authors affirm that all subjects in the study provided written informed consent for study images to be published with faces blurred. All data in this study were captured using IMUs with sensitivity of 0.061 mg (gravitational acceleration), 1666-Hz sampling rates (adjustable up to 6664 Hz), and ± 2g acceleration measurement range (adjustable up to ±16g).

The studies were approved by the Northwestern University Institutional Review Board, Chicago, IL, USA (STU00202449 and STU00212522) and were registered on ClinicalTrials.gov (NCT02865070 and NCT04393558).

All study-related procedures were carried in accordance with the standards listed in the Declaration of Helsinki, 1964. For COVID-19positive patients, double-sided medical silicone adhesive (3M, 2477P) secured the sensor to the neck area (aligned IMU1 on the SN and IMU2 on the SM) for more than 12 hours. For multiple days of use, medical-grade transparent film (Tegaderm, 3M) was applied between the skin and the double-sided adhesive to eliminate irritation from the adhesive. Clinical staff assisted the patient in placing the sensor. After each data measurement session, the device was sterilized with 70% isopropyl alcohol and left to dry at room temperature, and the sterilization process was repeated twice. Figure S25 shows a flow chart of the algorithm. Training data included time series z-axis acceleration data with features associated with tapping, coughing, laughing, and throat clearing. Training of this classifying algorithm used 10 datasets from each class (subjects SP1 to SP4) ( fig. S26 ). Feature extraction used peak detection, spectral information, and spectrograms. The first step identified events associated with tapping, coughing, laughing, and throat clearing using adapted thresholds according to the input signal levels evaluated across sliding windows with widths of 0.5 s. Each extracted event was then aligned to the center of corresponding time frames to maximize the energy of the signal for postprocessing (first row of fig. S25C ) based on continuous wavelet transformations. Resulting images within a 0.12-s window formed the basis for further analysis and classification (third row of fig. S25C ). Specifically, a binary tree architecture using a support vector machine (SVM) classified these extracted features into four activities ( fig. S25D ). First, in the SVM1 classifier, throat clearing activities were removed by negative values of the SVM1 hyperplane. Next, tapping activities were classified from SVM2 with a specific decision boundary (SVM2 result value: 2.5). Last, SVM3 separated the classes (coughing and laughing) with another decision boundary as described in fig. S25E .

Supplementary material for this article is available at http://advances.sciencemag.org/cgi/ content/full/7/20/eabg3092/DC1

Graphene electronic tattoo sensors

Inflammation-free, gas-permeable, lightweight, stretchable on-skin electronics with nanomeshes

Electronic skins for soft, compact, reversible assembly of wirelessly activated fully soft robots

Nanomesh pressure sensor for monitoring finger manipulation without sensory interference

Multifunctional epidermal electronics printed directly onto the skin

Soft microfluidic assemblies of sensors, circuits, and radios for the skin

Highly conductive, stretchable and biocompatible Ag-Au core-sheath nanowire composite for wearable and implantable bioelectronics

A novel wearable EEG and ECG recording system for stress assessment

Modular and reconfigurable wireless E-tattoos for personalized sensing

Ultraflexible organic photonic skin

Fully stretchable optoelectronic sensors based on colloidal quantum dots for sensing photoplethysmographic signals

2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society

Binodal, wireless epidermal electronic systems with in-sensor analytics for neonatal intensive care

Continuous wireless pressure monitoring and mapping with ultrasmall passive sensors for health monitoring and critical care

Flexible piezoresistive sensor patch enabling ultralow power cuffless blood pressure measurement

Soft wearable pressure sensors for beat-to-beat blood pressure monitoring

Clinical assessment of a non-invasive wearable MEMS pressure sensor array for monitoring of arterial pulse waveform, heart rate and detection of atrial fibrillation

Environmentresilient graphene vibrotactile sensitive sensors for machine intelligence

Skin-like pressure and strain sensors based on transparent elastic films of carbon nanotubes

Ballistocardiogram as proximal timing reference for pulse transit time measurement: Potential for cuffless blood pressure monitoring

Fully integrated wearable sensor arrays for multiplexed in situ perspiration analysis

A graphene-based electrochemical device with thermoresponsive microneedles for diabetes monitoring and therapy

Printed multifunctional flexible device with an integrated motion sensor for health care monitoring

Wearable sweat sensors

Wearable contact lens biosensors for continuous glucose monitoring using smartphones

Simultaneous monitoring of sweat and interstitial fluid using a single wearable biosensor platform

Imperceptible electrooculography graphene sensor system for human-robot interface

Gallium-based thin films for wearable human motion sensors

Large-area MRI-compatible epidermal electronic interfaces for prosthetic control and cognitive monitoring

Respiration rate and volume measurements using wearable strain sensors

Wearable sensing and telehealth technology with potential applications in the coronavirus pandemic

Soft, skin-interfaced microfluidic systems with integrated immunoassays, fluorometric sensors, and impedance measurement capabilities

Epidermal mechano-acoustic sensing electronics for cardiovascular diagnostics and human-machine interfaces

Mechano-acoustic sensing of physiological processes and body motions via a soft wireless device placed at the suprasternal notch

Continuous on-body sensing for the COVID-19 pandemic: Gaps and opportunities

Accelerometry: Providing an integrated, practical method for long-term, ambulatory monitoring of human movement

Highresolution seismocardiogram acquisition and analysis system

A chest-laminated ultrathin and stretchable e-tattoo for the measurement of electrocardiogram, seismocardiogram, and cardiac time intervals

Acoustic detection of coronary occlusions before and after stent placement using an electronic stethoscope

Real-time intended knee joint motion prediction by deep-recurrent neural networks

Enabling precision rehabilitation interventions using wearable sensors and machine learning to track motor recovery

Accuracy and precision of a novel non-invasive core thermometer

The accuracy of a disposable noninvasive core thermometer. Can

Theoretical and experimental studies of epidermal heat flux sensors for measurements of core body temperature

Lagrangian description of the unsteady flow induced by a single pulse of a jellyfish

On the dynamics of air bubbles in Rayleigh-Bénard convection

Wearable technology for personalized construction safety monitoring and trending: Review of applicable devices

Symptom screening at illness onset of health care personnel with SARS-CoV-2 infection in King County

The technology of accelerometry-based activity monitors: Current and future

Estimation of free-living energy expenditure by heart rate and movement sensing: A doubly-labelled water study

Eulerian video magnification for revealing subtle changes in the world

Taking a stab at quantifying the energetics of biological puncture

Free fall of homogeneous and heterogeneous cones

led the mechanical and thermal modeling and theoretical studies. J.-T.K. performed and processed 3D-PTV experimental data. S.X. and Aaron Banks provided clinical advice and managed and deployed devices in hospitals. A.J. led the clinical data collection

Differential cardiopulmonary monitoring system for artifact-canceled physiological tracking of athletes, workers, and COVID-19 patients

We thank the staff of Querrey Simpson Institute for Bioelectronics at Northwestern University and Shirley Ryan AbilityLab for assistance in this research. Funding: S.X. and J.A.R. recognize support from contract 75A50119C00043 awarded by the Biomedical