key: cord-1044582-sc7awj94 authors: Warren, Christopher J.; Wisener, John; Ward, Brittany; Behbahani, Sara; Shah, Tejash; Fano, Adam; Paskhover, Boris; Sadeghi-Nejad, Hossein title: YouTube as a Patient Education Resource for Male Hypogonadism and Testosterone Therapy date: 2021-03-19 journal: Sex Med DOI: 10.1016/j.esxm.2021.100324 sha: e93ac82b59aed84964e7e5e4e1818e5b5e73c7b4 doc_id: 1044582 cord_uid: sc7awj94 INTRODUCTION: YouTube is an unregulated platform that patients are using to learn about treatment options. AIM: To assess the reliability of YouTube videos (YTVs) related to male hypogonadism and testosterone therapy. METHODS: Searching on YouTube by relevance and view count, we analyzed the top 10 videos (80 videos total) for the following search terms: low testosterone, testosterone replacement therapy, AndroGel, and hypogonadism. MAIN OUTCOME MEASURE: We recorded the number of views for each video, evaluated videos using the DISCERN score (DS) criterion, and compared the DS for videos including board-certified physicians and videos without. A second comparison was made between videos with board-certified physicians in urology, endocrinology, other MD, and those without any physician. RESULTS: The YTVs analyzed received a total of 38,549,090 views, a median of 25,201 and 17.30 views/day. Videos that featured physicians had significantly fewer views/day than videos that did not (39.48 CI 9,72 vs 1,731 CI 330, 3,132; P = .019). Most YTVs studied were unreliable. The median DS across all videos was 2. However, most videos created by physicians were found to be reliable with a median DS of 4. In addition, YTVs that did not feature a physician were found to be significantly less reliable than videos that featured a physician (3.22 CI 3.06, 4.09 vs 1.87 CI 1.56, 2.18; P < .001). There was no significant difference in the reliability or viewership of YTVs stratified by physician type. CONCLUSION: Most YTVs related to male hypogonadism/testosterone therapy were unreliable, but there are reliable YTVs available. Reliable videos usually feature a physician and receive fewer views than unreliable YTVs. Physicians and academic societies should work to provide verified videos to provide patients with reliable information about male hypogonadism and testosterone therapy. CJ Warren, J Wisener, B Ward, et al. YouTube as a Patient Education Resource for Male Hypogonadism and Testosterone Therapy. Sex Med 2021;9:100324. Male hypogonadism, a clinical condition that affects up to 4 million men in the United States, is characterized by low serum testosterone (T) in combination with clinical symptoms including decreased libido, decreased energy, decreased muscle mass, increased body fat, and depressed mood. 1, 2 Male hypogonadism is a broad term that describes primary hypogonadism, secondary hypogonadism, and adult-onset hypogonadism (AOH) aka late-onset hypogonadism. 3 Testosterone therapy (TT) as a treatment for male hypogonadism has been a controversial topic, particularly in regard to AOH, which is likely the most prevalent form. 3 Between 2000 and 2010, prescriptions for TT increased yearly. 4, 5 In 2015, prescriptions for TT sharply declined in the USA after a Food & Drug Administration warning label about cardiac risk was mandated. 6 This has led to both over and undertreatment of AOH by clinicians. According to the American Urological Association, there is a significant portion of men being treated without indication. 7 Conversely, there may be a large portion of men who require TT but do not receive it because clinicians are concerned about possible adverse events. Most Americans use the internet to read about medical conditions and men who experience sexual symptoms may be more likely to turn to the internet before seeing a clinician out of embarrassment. 8, 9 YouTube is now the most prevalent form of social media used by adults in America and there is a growing body of literature evaluating the reliability of the medical content available to patients on YouTube. 10 However, there is currently no information available about the reliability or patient utilization of YouTube videos (YTVs) related to male hypogonadism and TT. We sought to perform the first qualitative assessment of YTVs using common search terms related to male hypogonadism and TT. We hypothesized that there will be significant interest in male hypogonadism and TT on YouTube and that the reliability of YTVs will be higher if a board-certified physician is featured. This study was determined to be exempt from review by the Rutgers Institutional Review Board. YouTube was searched on June 17, 2019, using Mozilla Firefox in "private browser" mode for the following terms: low testosterone, testosterone replacement therapy, AndroGel (Abbvie, North Chicago, IL), and hypogonadism. Searching in "private browser" mode prevents You-Tube from displaying search results based on the users' past interactions and search history. When several terms could be used to search for videos on a similar subject, Google Trends was used to determine the term with the highest average search volume over the past year on YouTube. When multiple terms are queried using this tool, the results are displayed as a line graph depicting the relative interest over time. This allows the user to easily visualize which search term was used most often. For example, AndroGel, Testim (DPT Laboratories, San Antonio, TX), and Fortesta (Pharbil Waltrop, Waltrop, Germany) were compared and AndroGel had the most traffic on YouTube and was therefore chosen for analysis. We did not choose to search all terms because AndroGel was clearly the most popular search term used and we wanted to investigate the average YouTube users' experience. A list was compiled of the top 10 videos using 2 different search filters; relevance (YouTube's default) where YouTube's proprietary algorithm attempts to direct the user to videos most similar to the queried term and view count where the results are displayed from the most viewed videos to the least viewed videos related to the queried term. Non-English language videos were excluded from the analysis. The top 10 videos were chosen for statistical analysis to evaluate the videos which are most likely to be viewed by patients for each term. Duplicate videos across search terms and filter settings were eliminated and only evaluated once. This led to the evaluation of 72 unique videos. We determined the American Board of Medical Specialties board-certification status and specialty of any featured physician using certificationmatters.org and evaluated each video using the DISCERN score (DS) criterion. 11 The DS is a reliable and validated tool designed to assess the quality of health care literature. It has been used in several studies to evaluate the reliability of YouTube videos related to various health conditions and we used a previously accepted cutoff of 3 or above to determine if videos were of acceptable quality as a patient education resource. 9, 12, 13 The videos were divided for evaluation between 2 3rd-year medical students (C.W. (male) and S.B. (female)), and a MD/ urologist (T.S., male). All evaluators were trained to use the DS and independently reviewed their assigned videos. We calculated the DSs and bias scores for each video rated. Consistency among raters was confirmed by calculating an intraclass correlation coefficient. Intraclass correlation coefficient for bias was 0.838 and for reliability was 0.836 indicating strong interrater reliability. In addition, the number of views, views per day, and author of the video was recorded. We then compared videos featuring a U.S. board-certified physician compared with videos that did not. We then stratified physician videos into 3 categories (urologists, endocrinologists, and other MD) and compared those videos along with videos without a physician. Statistical analysis was conducted using IBM Statistical Package for the Social Sciences (SPSS Statistics) V20.0 (SPSS Inc, Chicago, IL) standard statistical techniques including independent sample t tests and ANOVA. If data showed variance among groups (Levene's test), a Welch ANOVA was performed in lieu of a standard ANOVA. All ANOVA analysis (standard and Welch) was supplemented with a Games-Howell post hoc analysis. All analyses were two-tailed and a P-value of <0.05 was considered statistically significant. After the elimination of duplicate and non-English videos, we analyzed 72 unique YTVs in which 27 featured a physician; 10 featured urologists, 8 featured endocrinologists, and 9 featured various physicians in other specialties or physicians whose specialty could not be identified. In aggregate, the YTVs included received a total of 38,549,090 views, with a median of 25,201 views and 17.30 views/day. As can be seen in Table 1 , videos that featured non-physicians had 17 times more views than videos that did (827,150 vs 49,161, P ¼ .004). This difference was consistent with views per day showing non-physician videos receiving 44 times more views per day than physician featured videos (1,731 vs 39, P ¼ .019). Although physician videos were viewed significantly less, physician featured videos were more reliable than non-physician featured videos (DS 3.22 vs 1.87, P < .001) and exhibited less bias (bias score 3.52 vs 1.87, P < .001). When subgrouping physician videos, there was no significant difference in views/day between urologists, endocrinologists, and other MDs ( Table 2) . Most YTVs studied were unreliable. The median DS across all videos was 2 and the average DS was 2.38 CI (2.07, 2.68). However, most videos created by physicians were found to be reliable with a median DS of 4 and an average DS of 3.22 CI (3.06, 4.09). In addition, YTVs that did not feature a physician were found to be significantly less reliable than videos that did feature a physician (3.22 CI 3.06, 4.09 vs 1.87 CI 1.56, 2.18; P < .001). The mean DS stratified by physician type can be seen in Table 2 . There was no significant difference in reliability when stratified by physician type. In addition to containing unreliable content, most videos displayed strong bias with a median bias score of 2 and an average bias score of 2.49 CI (2.14, 2.83). However, most videos that featured a physician were unbiased with a median bias score of 4 and an average bias score of 3.52 CI (2.80, 3.82). These YTVs were significantly less biased than videos that did not feature a physician (3.52 CI 2.80, 3.82 vs 1.87 CI 1.50, 2.24; P < .001). There was no significant difference in bias score when stratified by physician type. The present study examines the reliability and bias of YTVs related to male hypogonadism and TT. Most videos were found to be unreliable and unacceptable as a patient education resource using a previously accepted standard. In addition, most videos contained significant bias. However, YTVs that featured a physician were significantly more reliable and less biased than videos that did not feature a physician. Moreover, most videos featuring a physician were of acceptable quality to be used as a patient education resource. With over 38 million views, there are a substantial number of patients turning to YouTube to learn about male hypogonadism and TT. Our findings are consistent with previous studies evaluating sexual health information on YouTube. There are several studies in the literature evaluating the reliability of YTVs related to men's health conditions such as male infertility, premature ejaculation, erectile dysfunction, benign prostatic hyperplasia treatment, and prostate cancer that also found that most YTVs related to the evaluated subject are unacceptable as a patient education resource. 12,14e16 For example, Loeb et al found that there is widespread dissemination of misinformation about prostate cancer on YouTube and that less reliable videos had increased user engagement. 12 However, the present study is one of the only studies to show that there is reliable information on YouTube and that it could be acceptable to use as a patient education resource if patients are directed to videos featuring board-certified physicians. There is evidence that YTVs are valuable as a patient education resource. In one study, Lauckner et al found that these videos are a more effective learning tool for patients than other forms of media in both retention time and patient attitude toward the message. 17 They randomly assigned patients to view the same cancer risk reduction message through Twitter, Facebook, Blogs, or YouTube and found that YouTube resulted in the most favorable patient attitude toward receiving the message as well as the greatest retention time. In addition, Gul & Diri found that most information related to premature ejaculation on YT is reliable based on the expert opinion of 2 urologists with an excellent interrater agreement. 9 This is further evidence that there are both reliable and unreliable YTVs available to patients. Unfortunately, reliable videos are either being viewed at the same rate as, or, as in the present study, significantly less than unreliable videos. It would benefit patients if physicians, academic hospitals, or physician societies created more patient education videos and uploaded them to YouTube. The higher quality physician featured videos in our study were heavily outnumbered (27 vs 45) by videos that were created by other authors (health coaches, patients, YouTube celebrities). Moreover, they received significantly fewer views even though they were more reliable and less biased. Based on the findings in our study as well as previous literature there is evidence that patients are having difficulty finding reliable information on YouTube. Urological organizations such as the American Urological Association can benefit patients by increasing the content of their own YouTube channels. In the wake of the COVID-19 pandemic, and with the success of the virtual American Urological Association Live experience many urologists have accepted the merit of virtual learning and video creation. With the video resources now already in place, it may not be difficult to upload patient education content on these channels. This can provide patients and clinicians with a free educational resource that is easily accessible on YouTube. In the interim, independent physicians that wish to create their own content for patients can follow the guidelines published by the YouTube Creator Academies, a YouTube channel created by the company to help content creators increase the success of their YouTube channels. 18 They recommend taking the following steps: 1) create a custom thumbnail for each video that is at least 1,280 Â 720 pixels to ensure that it is clear across all devices, 2) use a descriptive video title to ensure that the users who click on the video truly want to watch it as an increased average watch time will increase the videos rank in search, and 3) to use descriptive tags that can assist the YouTube search algorithm in displaying your video when users search for similar content. This study has several limitations. First, we used a physician specializing in urology and medical students to evaluate the videos. Their above-average medical knowledge may bias the results of the DSs. Second, we only watched the top 10 videos for each search term/filter setting and patients may scroll below the top 10. Finally, the YouTube algorithm may personalize results for different queries depending on the history of the videos that the user has interacted with. There was an attempt to eliminate this bias by searching on "private browser mode" which eliminates the users browsing history. YouTube is used by patients as a source of health care information, but most views are generated by biased, unreliable videos, especially if the video does not involve a U.S. boardcertified physician. Moreover, videos created by physicians are receiving significantly fewer views even though they contain more reliable content. In the age of digital media, it is important that physicians direct patients toward high-quality videos. This can be done by creating high-quality videos or finding videos that were already made and providing patients with a direct link. In the future, there needs to be a collaboration between physicians, patients, and digital media experts to create high-quality, easily accessible patient education videos. This can be accomplished by urological societies increasing the number of patient education videos on their YouTube channels and following YouTube Creator Academy recommendations for increasing viewership. design, Data analysis and interpretation, Critical revision of the manuscript, Supervision, Approval of the final manuscript. Hossein Sadeghi-Nejad: Research conception and design, Data analysis and interpretation, Drafting of the manuscript, Critical revision of the manuscript, Supervision, Approval of the final manuscript. Risks of testosterone-replacement therapy and recommendations for Monitoring Testosterone therapy in men with Androgen Deficiency Syndromes: an endocrine society clinical Practice guideline Adult-onset hypogonadism T" as in "Template": How to Sell DiseaseComment on "Promoting 'low T'"Low "T" as in "Template"Opinion Promoting "low T": a medical Writer's Perspecti-vePromoting "low T"Promoting "low T Testosterone Prescribing in the United States Evaluation and Management of testosterone Deficiency: AUA guideline Majority of adults Look online for health information YouTube as a source of information about premature ejaculation treatment Share of U.S. adults using social media including Facebook, is mostly unchanged sincee 2018 DISCERN: an instrument for judging the quality of written consumer health information on treatment choices Dissemination of misinformative and biased information about prostate cancer on YouTube The quality of YouTube videos as an educational resource for Attention-Deficit/Hyperactivity Disorder A systematic evaluation of youtube as an information source for male infertility Quality of information in YouTube videos on erectile dysfunction Information on surgical treatment of benign prostatic hyperplasia on YouTube is highly biased and misleading The Differential Effects of social media Sites for Promoting cancer risk reduction California: YouTube Funding: None.