key: cord-0735451-bwksi0ge
authors: Shen, Tony S.; Driscoll, Daniel A.; Islam, Wasif; Bovonratwet, Patawut; Haas, Steven B.; Su, Edwin P.
title: Modern Internet Search Analytics and Total Joint Arthroplasty: What Are Patients Asking and Reading Online?
date: 2020-10-20
journal: J Arthroplasty
DOI: 10.1016/j.arth.2020.10.024
sha: fdbdcbcf88ea117d958e88789cc69e289cd91b5f
doc_id: 735451
cord_uid: bwksi0ge

INTRODUCTION: Patients considering total joint arthroplasty often search for information online regarding surgery, however little is known about the specific topics that patients search for and the nature of the information provided. Google compiles frequently asked questions associated with a search term using machine learning and natural language processing. Links to individual websites are provided to answer each question. Analysis of this data may help improve understanding of patient concerns and inform more effective counseling. METHODS: Search terms were entered into Google for total hip and total knee arthroplasty. Frequently asked questions and associated websites were extracted to a database using customized software. Questions were categorized by topic; websites were categorized by type. JAMA Benchmark Criteria were used to assess website quality. Pearson’s chi-square and Student’s t-tests were performed as appropriate. RESULTS: A total of 620 questions (305 TKA, 315 THA) were extracted with 602 associated websites. The most popular question topics were Specific Activities (23.5%), Indications/Management (15.6%), and Restrictions (13.4%). Questions related to Pain were more common in the TKA group (23.0% vs 2.5%, p<0.001) compared to THA. The most common website types were Academic (31.1%), Commercial (29.2%), and Social Media (17.1%). JAMA scores (0-4) were highest for Government websites (mean 3.92, p=0.005). CONCLUSION: The most frequently asked questions on Google related to total joint arthroplasty are related to arthritis management, rehabilitation, and ability to perform specific tasks. A sizeable proportion of health information provided originate from non-academic, non-government sources (64.4%), with 17.1% from social media websites.

CONCLUSION: The most frequently asked questions on Google related to total joint 22 arthroplasty are related to arthritis management, rehabilitation, and ability to perform specific 23 tasks. A sizeable proportion of health information provided originate from non-academic, non-24 government sources (64.4%), with 17.1% from social media websites. Machine learning, broadly defined, is a field of computer science that uses algorithms to 43 recognize patterns in data. Recent developments in a subset of machine learning, known as deep 44

learning, now allow for pattern recognition in vast quantities of data that were previously too 45 computationally complex to process [1] . In medicine, deep learning has led to advances such as a 46 tool that predicts diabetic retinopathy using retinal fundus photographs and a test that 47 distinguishes COVID-19 from community-acquired pneumonia using chest computed 48 tomography imaging [2] [3] [4] . Machine learning algorithms using patient data from the electronic 49 medical record have been designed to predict acute kidney injury, cancer mortality rate, and 50 prognosis following solid organ transplantation [5] [6] [7] . New applications are sure to emerge as 51 this technology matures [8] . 52

Perhaps the most common way people interact with a sophisticated deep learning algorithm is by 53 using Google Web Search, by far the most widely used search engine in the United States [9] . In 54 2015, Google introduced a machine learning-based system, known as RankBrain, to recognize 55 patterns in individual search queries [10] . By analyzing a large dataset of search queries, this 56 technology allowed Google to predict individual searches and offer suggestions after a search 57 query is entered [10] . In 2018, Google added a natural language processing system into its search 58 analytics platform, a technology called BERT (Bidirectional Encoder Representations from 59 Transformers) [11, 12] . Natural language processing significantly expands the capability of deep 60 learning algorithms to identify search patterns more accurately. The technology behind BERT 61 has been adapted help organize patient data in the electronic health record and recognize health 62 information disseminated on social media [13] [14] [15] . Using RankBrain and BERT, the Google 63 J o u r n a l P r e -p r o o f search results page now provides an extensive list of questions frequently asked along with the 64 original search query [11] . Additionally, links to websites are provided to "answer" each 65 associated question [11] . 66

Internet usage rates among patients considering elective orthopedic procedures have been 67

reported to be as high as 84% [16, 17] . Up to 80% of these patients research their condition 68 online, with one survey indicating that 30% of these patients specifically discuss information 69 found online with their surgeon [16] [17] [18] [19] [20] [21] [22] . Multiple studies have evaluated the quality and 70 readability of online resources for orthopedic procedures [20] . However, these studies provide 71

little insight into what specific information patients are trying to obtain when searching their 72 conditions online. Using the modern search analytics system employed by Google, clusters of 73 frequently asked questions associated with specific orthopedic conditions and procedures can be 74 identified and analyzed. 75

We present an analysis of the questions most frequently associated with total hip arthroplasty 76 (THA) and total knee arthroplasty (TKA) by question type and topic. The websites provided to 77 address each question are also analyzed for source and quality. We hypothesize that there are 78 distinct search patterns for THA compared to TKA. Greater understanding of this data could 79 allow for better understanding of patient concerns as well as inform more effective counseling 80 regarding total joint arthroplasty. 81

Search terms were entered into Google Web Search (www.google.com) using a clean-installed 84

Google Chrome browser for total hip arthroplasty ("hip replacement," "total hip replacement," 85 J o u r n a l P r e -p r o o f "total hip arthroplasty") and total knee arthroplasty ("knee replacement," "total knee 86 replacement," "total knee arthroplasty" Descriptions for these topics may be found in Table 1 . 101

In accordance with previous studies, websites were categorized by source into the following 102 groups: Commercial, Academic, Medical Practice, Single Surgeon Personal, Government, and 103

Social Media (Table 1) A total of 620 questions (305 TKA, 315 THA) and 602 associated websites were extracted and 129 categorized. Interobserver reliability as assessed using Cohen's kappa coefficient was 0.93 for 130 question classification and 0.91 for website classification. The top ten most frequently asked 131 questions for THA and TKA are presented in Table 3 . 132

The majority questions fell into the Fact category using Rothwell's system (Figure 1a , 56.4% 133 TKA, 57.5% THA). In the TKA group, the most popular question topics were Specific Activities 134 (24.3%), Pain (23.0%), Restrictions (15.1%), and Timeline of Recovery (13.1%) ( Table 4 ). In 135 the THA group, the most popular question topics were Specific Activities (22.9%), 136

Indications/Management (19.0%), Restrictions (11.7%), and Technical Details (11.1%) ( Table  137 4). Questions related to Pain were significantly more common in the TKA group (23.0% vs 138 2.5%, p<0.001) compared to THA (Figure 1b ). In the THA group, there were significantly more 139 questions related to Technical Details (11.1% vs 3.9%, p<0.001), Indications/Management 140 (19.0% vs 12.1%, p=0.02), Risks/Complications (9.8% vs 0.3%, p<0.001), and Longevity (3.2% 141 vs 0.0%, p<0.001) compared to TKA (Figure 1b) . 142

The most common website types for both the TKA and THA groups were Academic (31.1%), 143

Commercial (29.2%), and Social Media (17.1%) (Figure 2a ). There was a statistically 144 significantly higher rate of websites in the Other category in the THA group compared to TKA 145 (Table 5, Overall, our findings support our anecdotal experience with common patient questions, which 211 supports validity of our methodology to better understand our patients in a more objective 212 manner. We believe this data is useful for surgeons designing pre-operative classes or other 213 perioperative counseling programs. 214

One important limitation to this study is inherent with the dynamic nature of Google's search 215 analytics system. The questions and websites presented by Google will change as more data is 216 generated by patients researching total joint arthroplasty online. Furthermore, because Google 217 takes individual search patterns into account when presenting search results, questions and 218 websites can differ from person to person. We address this potential variability in the data by 219 using a large sample size of questions and websites. To minimize the effect of individual search 220 history, all data was extracted using a clean-installed web browser. While we do not believe that 221 these issues significantly affect the validity of our findings, the inherent qualities of Google's 222

proprietary system still represent a limitation. Our categorization system is also an important 223 limitation of this study. Although the categories were made in accordance with previous studies 224 and tested for interobserver reliability, there remains inherent overlap between some of the 225 categories. 226 227

Although the academic sources were the most common type of website in this study, a sizeable 229 proportion of websites analyzed originated from non-medical, non-government sources (47.6%). 230

Interestingly, the average JAMA score was significantly higher in non-medical, non-government 231 sources (3.09 vs 2.06, p<0.001). There was also no statistically significant difference in JAMA 232 scores between academic websites and social media (2.06 vs 2.18, p=0.84). These relatively low 233 scores should be cautiously interpreted. As an instrument, the JAMA Benchmark Criteria is 234 limited in its ability assess the accuracy and appropriateness of online information. A perfect 235 score can be obtained simply by the presence of an author, date, disclosures, and list of 236 references. As such, the JAMA Benchmark Criteria is a better measure of website transparency 237 than information quality. This limitation is acknowledged by Silberg in the article that first 238 described the JAMA score [27] . Additionally, modern website designs may distribute the 239 components of the score across multiple pages, thus decreasing the score for an individual page. 240

Despite these limitations, the JAMA Benchmark Criteria remains as one of the most well-241 established instruments available for assessing online health information [20] . Instruments that 242 assess the information quality are difficult to standardize across multiple fields of study. As such, 243 many studies examining the quality of online literature on specific orthopedic topics utilize 244 original instruments designed for that subject [20, 45, 46] . Certain principles of the JAMA score, 245 such as authorship and citations, may help an otherwise high-quality website appear more 246 transparent to readers, which may indirectly affect the subjective credibility of the website. 247

However, for academic institutions and surgical practices, we ultimately believe that it is more 248 valuable to provide high-quality and up-to-date information rather than focus on the specific 249 components of the JAMA score. 250

We considered the website source itself as an indirect indicator of information quality. Websites 251 maintained by academic centers, established practices, and professional societies almost certainly 252 provide more accurate and up-to-date information compared to sources such as social media. 253

Non-medical websites can still play a certain role for patients. For example, our study found that 254 social media websites were more commonly linked to questions about performing specific tasks, 255 a topic that may be discussed between patients in groups online. Internet communities that 256 facilitate patient communication can be useful as a supplemental source of information and 257 support [47] . Still, the best information available regarding total joint arthroplasty likely comes 258 from sources that are directly or indirectly maintained by orthopedic surgeons. We believe that 259 the arthroplasty community should take the lead in providing the most up-to-date and accessible 260 information for patients on the internet. There is opportunity to optimize modern orthopedic 261 websites to better integrate with Google's changing search algorithms to reach more patients. 262

Search engine optimization techniques that have been widely employed by businesses are now 263 also being used by medical practices to increase viewership [22, 48, 49] . As advances in 264 information technology continue to shape our world, the field of orthopedics will need to adapt 265 appropriately to best take care of our patients. sections for which p<0.05 are designated with "*"; p<0.01 with "**"; p<0.001 with "***". Can I stay alone after total knee replacement? How long will I need pain medication after total knee replacement? What should I avoid after knee replacement? Why is a knee replacement so painful? How do you sit on a toilet after knee surgery? How long does it take to bend your knee after surgery? Why does knee replacement hurt more at night? Can I vacuum after knee replacement? Can you damage a knee replacement? Can you wait too long for a knee replacement?

J o u r n a l P r e -p r o o f J o u r n a l P r e -p r o o f *JAMA Score reported as mean (standard deviation) **Websites sources in this category were law firm (4), financial firm (3), and non-medical device manufacturer (1) J o u r n a l P r e -p r o o f Note: Sections for which p < 0.05 are designated with "*"; p < 0.01 with "**"; p < 0.001 with "***". 

Artificial intelligence in healthcare

Development and Validation of a Deep Learning Algorithm for Detection of Diabetic 272

Retinopathy in Retinal Fundus Photographs

Artificial 278 intelligence to codify lung CT in Covid-19 patients

Learning Approaches to Predict 6-Month Mortality Among Patients With Cancer

A machine learning-based approach to prognostic 286 analysis of thoracic transplantations

Internet use in 310 an orthopaedic outpatient population

Internet use amongst an elective spinal surgery outpatient population

The Internet and the Physician-Patient Relationship

Orthopaedic Patient Information on the World Wide Web

Pew Internet & American Life 320 Project

The future of social media in 322 orthopedic surgery

Answers to Health Questions: Internet 325

Search Results Versus Online Health Community Responses

Mixed Company: Communicating in Small Groups and Teams

The quality of internet sites providing information 330 relating to oral cancer

Content of Internet-Based Information for Ten Common Orthopaedic Sports Medicine 334

Assessing, Controlling, and Assuring the Quality of Medical Information on 336 the Internet

Quality of Online Video Resources 338

Concerning Patient Education for the Meniscus: A YouTube-Based Quality-Control 339

The quality of internet sites providing information relating to 341 hip resurfacing

Surfing for Hip 343 Replacements: Has the "Internet Tidal Wave" Led to Better Quality Information

The Measurement of Observer Agreement for Categorical Data

The Shift 348 to Same-Day Outpatient Joint Arthroplasty: A Systematic Review

Cost-Effective Mobile-Based Healthcare 351 System for Managing Total Joint Arthroplasty Follow-Up

Effectiveness of an Automated Digital Remote 354

Guidance and Telemonitoring Platform on Costs, Readmissions, and Complications After

Effectiveness of Telemedical Applications in Postoperative 358

After Total Joint Arthroplasty

Patient-361 controlled analgesia in a postoperative total joint arthroplasty population

Lower limbs function and pain relationships after unilateral total knee 364 arthroplasty

The Effect of a New Multimodal Perioperative 366

Anesthetic Regimen on Postoperative Pain, Side Effects, Rehabilitation, and Length of 367

Guidelines Can Decrease the Amount of Opioids Prescribed After Total Joint 374

Trends and 376 predictors of opioid use after total knee and total hip arthroplasty

Opioid Use Increase the Risk of Early Revision Total Hip Arthroplasty?

Multimodal Pain Management in Total Knee 384

The quality of web-based information on hip 386 resurfacing arthroplasty: a cross-sectional survey

An Evaluation of Information Available on the 389

Internet Regarding Minimally Invasive Hip Arthroplasty

Perceived healthcare provider reactions to patient and caregiver use of online health 393