key: cord-0974637-4xjtiy98
authors: Bothun, Luke S.; Feeder, Scott E.; Poland, Gregory A.
title: Readability of Participant Informed Consent Forms and Informational Documents From Phase III COVID-19 Vaccine Clinical Trials in the United States
date: 2021-06-03
journal: Mayo Clin Proc
DOI: 10.1016/j.mayocp.2021.05.025
sha: 87da22851f6dc83581f78adeed841c983faf53a2
doc_id: 974637
cord_uid: 4xjtiy98

Objective To assess the readability of the informed consent forms from the phase III COVID-19 vaccine trials conducted in the United States. Patients English consent forms utilized for patients in phase III COVID-19 vaccine clinical trials. Consent forms were obtained in October 2020. Methods Using Microsoft Word tools, we analyzed the readability (i.e., the ease of reading) of written consent forms and informational documents from phase III COVID-19 vaccine clinical trials in the United States from the following manufacturers: AstraZeneca, Moderna, Pfizer, Johnson & Johnson, and Novavax. Results Due to low readability and several format factors, this study determined that none of the consent forms or informational documents from the recent phase III COVID-19 vaccine clinical trials conducted in the United States met readability standards at the recommended 7th grade readability level for the average vaccine research volunteer in any readability category. The average English-speaking vaccine-trial volunteer would have great difficulty comprehending the information provided in the consent forms and informational documents. To ensure that study subjects receive and fully comprehend information regarding a clinical study and can provide reliable consent, greater attention should be given to the development and use of simplified consent forms, multimedia formatting, personal discussion, and comprehension assessments.

CoV-2 and is responsible for the current global pandemic. 1 In response, pharmaceutical companies implemented phase III clinical trials in the United States and other countries to assess the efficacy and safety of COVID-19 vaccine candidates. Prior to enrollment in these vaccine trials, informational documents and consent forms are provided to potential study subjects. These documents provide subjects with basic study information, including necessary precautions, privacy measures, and potential vaccine benefits, risks, and side effects. By providing their signatures on the consent forms, study subjects acknowledge that they understand the potential consequences of participating in the study and are willing to accept the risks associated with the study. In this paper, we analyze the readability of these informational documents and consent forms.

During the informed-consent process, United States law requires volunteer study subjects to be informed and educated about the risks, benefits, and alternatives of the experimental medical intervention and procedures. 2 For informed consent to be valid, the patient or study volunteer must adequately understand the risks of participating in the clinical trial and voluntarily provide consent. 3 In fact, any intervention without clear patient consent could be considered assault. 4 Several factors may impede informed consent, including the subject's education level, inadequate or complicated consent information, or ineffective communication between subjects and research professionals. Because study information J o u r n a l P r e -p r o o f packets and consent forms are the primary method by which study information and procedures are communicated regarding the clinical trial, the readability and comprehension of these materials are critical. If a study volunteer is unable to read and comprehend the information provided in the consent form, the provision of consent is misguided and invalid. Thus, informed consent documents must be written in a manner that accommodates trial participants in regard to their reading ability and comprehension.

Assessing consent-form readability is a critical step toward ensuring that the text is at an appropriate level for readers who have varying proficiency. 5 Data collected by the National Work Group on Literacy and Health indicate that approximately a quarter of U.S. citizens have rudimentary reading skills, and are likely unable to read and understand even medication instructions, a bus schedule, or directions on cleaning products. 6 The average UK citizen reads at a lower level than the average U.S. citizen; 7 however, this does not mean that these Other consent form factors such as font size, document layout, charts, pictures, vocabulary, and word length can also greatly affect readability. 10 Many experts recommend using plain vocabulary, culture-and-gender-neutral wording, short sentences, active voice, J o u r n a l P r e -p r o o f bulleted lists, and present tense in documents intended for the public. 11 The purpose of this study was to analyze and compare the readability of the consent forms and informational documents from multiple phase III COVID-19 clinical vaccine trials being conducted in the United States to determine if they meet the guidelines for adequate public readability.

We acquired electronic consent forms and informational documents from AstraZeneca, Johnson & Johnson, Moderna, Novavax, and Pfizer for the phase III COVID-19 vaccine trials recently conducted in the United States. All five companies provided an informed-consent document written for adult study subjects. Two companies also provided informational documents given to study subjects and one company provided a consent and assent form for their vaccine study involving adolescents.

Using the readability tools available in Microsoft Word, we evaluated formatting metrics and readability statistics in each consent form and informational document. Two authors independently analyzed formatting and readability to reduce bias. Assent form F was utilized in a vaccine trial on adolescents. Because of this, it has significantly higher (i.e., easier) readability scores in all of the categories analyzed and therefore we did not include the readability scores for the adolescent assent form in our report outlining the mean metrics of the adult consent forms.

We used three readability formulas to determine the difficulty of a text. The Flesch

Reading Ease score is a scale that predicts reading ease on a scale from 1 to 100, with 70 being an appropriate level for the average reader in the United States. 11 Higher scores indicate easier J o u r n a l P r e -p r o o f readability. This formula, which is the most tested and reliable readability formula, is widely used (See Table 1 ).

The Flesch-Kincaid Grade Level score is a readability score developed by the U.S. Navy. It predicts the reading ease of a passage on a scale from 1 to 12, which indicates the educationgrade level necessary to read the passage. 11 A 7 th grade level is recommended for the average reader in the United States. 12 Although the Flesch Reading Ease score and Flesch-Kincaid Grade Level score are both based on word length and sentence length, they have different weighting factors. We recognize a difference in grade-level assessment between the mean Flesch-Kincaid Grade Level and mean Gunning-Fog Index. The Flesch-Kincaid Grade Level truncates readability at the 12 th -grade level and may in fact underestimate the actual reading level required for complex text. 12 Table 1 compares the Flesch Reading Ease score to the Flesch-Kincaid Grade Level score.

Lastly, The Gunning-Fog index, which is similar to the Flesch-Kincaid Grade Level, is a readability test that scores text from 1-20. Scores are based on the number of complex words per sentence in a ~100-word passage. A complete score is obtained by averaging the Gunning-Fog index score for three random 100-word passages throughout the consent forms. Texts requiring near-universal understanding generally require an index of 7 (See Table 2 ). 13

We obtained and reviewed two informational documents and seven consent forms, including the adolescent assent and adult consent forms for an adolescent study, used in the Table 3 .

The percent of passive sentences ranged from 0% to 42.1% (mean 33%). The Flesch

Reading Ease Score ranged from 43.3-68.3 (mean 49.6). Flesch-Kincaid Grade Level ranged from 8-11.8 (mean 11). The Gunning-Fog Score ranged from 9.9-15.6 (mean 13.3). See Table 4 .

To our knowledge, this study is the first systematic analysis of informed consent readability for the phase III COVID-19 vaccine trials conducted in the United States. We found J o u r n a l P r e -p r o o f that none of the consent forms or informational documents had readability levels that met recommended standards for average English-speaking vaccine volunteers in any category.

Additionally, very few met the recommended criteria in page number, sentence length, font style, and percentage of passive-voice sentences. While assent form F for the adolescent study did not reach desired readability metrics, it scored notably better in readability metrics, demonstrating that improved readability is possible.

Study subjects are less likely to fully comprehend informational documents that are excessively detailed and lengthy. 14 Thus, the number of pages in informational materials should be fewer than 15. 15 The informed-consent documents we reviewed ranged from 8-36 pages (mean 24.6). This average exceeds the recommended number of 15 by nearly 10 pages.

High word density can also lead to incomprehensibility because it raises the difficulty level of a text. Too many words in a sentence or on a page can hinder or prevent comprehension of key content. For this reason, it is recommended that sentences be between 12 and 17 words. 16 The mean sentence length ranged from 16.9 to 20.4 words (mean 19.1), which is several words over the recommended 12-17 word limit.

Other factors, such as font style and size, can also play a role in assessing the readability of a text. Sans serif fonts, such as Arial or Calibri, are preferred over serif fonts such as Times New Roman. In addition to this, larger size fonts, especially size 14, are easier to read than smaller fonts. 17 The font size in all of the consent forms and readability documents was too small; however, some did use appropriate font style (e.g., Arial). Although studies demonstrate that 14-point font is preferred, 17 all of the consent forms and informational documents in this J o u r n a l P r e -p r o o f study varied between 10 and 12-point font. Documents D, E, F, G, and Z used the favorable Sans Serif (e.g., Arial or Calibri) fonts, while documents A, B, C, and Y used Serif (Times New Roman) font. 18 A high percentage of passive-voice sentences, or sentences in which the subject receives the action of the verb, also negatively impacts readability. 11 The percentage of passive-voice sentences in the consent forms ranged from 0% to 42.1% (mean 33%). Only consent form G had 0% sentences in passive voice. Assent form F was marginally successful in this category, with only 18.3% written in passive voice.

It is critical that research subjects are able to comprehend informed-consent material.

Many studies indicate that research volunteers often do not understand critical aspects of the research in which they are participating, suggesting that "informed" consent is imperfectly realized. 19 This is largely due to the increasing length and complexity of consent forms.

Providing too much information can have similar consequences to not providing enough information. Consequently, the informed-consent process may not accomplish the goal of adequately informing prospective participants about the nature of a study and its potential risks and benefits. Since almost half of American adults read at or below the 8 th -grade level, 12 this study demonstrates that the consent form information used in the phase III clinical trials in the US is too difficult to ensure comprehension for the majority of adult readers. Assessing the study participants' knowledge about the study at the end of the consent process could help improve their consent-form comprehension. If subjects fail to correctly answer questions critical to indicating an understanding of the study, it would indicate a lack of understanding the J o u r n a l P r e -p r o o f consent documents. Engaging with study subjects may also help investigators learn about limitations of their consent forms.

Aside from readability that is too complex and text that is too long, the consent forms analyzed in this study also lacked instructional graphics and multimedia, including pictures, charts, videos, or slideshows, which increase both appeal and comprehension levels. 20 Verbal discussion between volunteers and study educators also appears to be an effective way of improving research participants' understanding. 21 Further research is necessary to determine the most effective methods to educate patients on study information.

Consent-form readability could be significantly improved by using standard readability formulas, applying rewriting techniques, and being aware of subjects' reading-comprehension levels. 22 The biopharmaceutical industry, FDA, National Cancer Institute, and National Institutes of Health often advise developing consent forms appropriate for a 6 th -8 th grade reading level. 8 Using readability formulas like the Flesch Reading Ease Score, Flesch-Kincaid Grade Level, and the Gunning-Fog Index would enable pre-analysis of readability and allow development of reliable and understandable consent forms. Since many factors influence text readability, 10 using several of these formulas as well as other physical formatting aspects of the text can assist in providing highly readable consent forms.

The limitations of our study include analyzing readability statistics primarily based on the population of the United States; therefore, these reading scores only pertain to analysis of such documents written in the English language. Studies in other nations may have different population education or language-readability statistics. This study does not consider that J o u r n a l P r e -p r o o f readability scores may be further affected based on specific institutional review board specifications in a given region or country. This study also does not evaluate whether some individuals desire more information than presented in the consent forms. Studies show that while some individuals believe that vaccine pamphlets provide too much information, others may not think the information received is adequate, and at least one study has evaluated provision of vaccine-related material based on the subject choosing the grade-level complexity they prefer. 23 The strengths of our study include comparing consent forms for all five phase III COVID-19 vaccines conducted in the United States.

The results of our study indicate that an appreciable percent of the study participants 

World Health Organization. Coronavirus disease (COVID-19) pandemic

Audio-visual presentation of information for informed consent for participation in clinical trials

Informed consent procedures with cognitively impaired patients: A review of ethics and best practices

The effectiveness of health literacy interventions on the informed consent process of health care users: a systematic review protocol

Word Frequency and Readability: Predicting the Text-Level Readability with a Lexical-Level Attribute

Communicating with patients who have limited literacy skills: Report of the national work group on literacy and health

Building skills for all: a review of England. Policy insights from the survey of adult skills

Consent Forms, Lower Reading Levels, and Using Flesch-Kincaid Readability Software

On Improving Text Readability by Creating a Personal Writing Style

The Principles of Readibility

Readability Standards for Informed-Consent Forms as Compared with Actual Readability

Oxford Guide to Effective Writing and Speaking: How to Communicate Clearly

Department of Health and Human Services Office for Human Research Protections. Meeting new challenges in informed consent

Readability and comprehensibility of informed consent forms for clinical trials

Sentence Length in Education Research Articles: A Comparison between Anglophone and Turkish Authors

The effects of font type and size on the legibility and reading time of online text by older adults. CHI '01 Extended Abstracts on Human Factors in Computing Systems

Evaluating patient choice of typeface style and font size for written health information in an outpatient setting

Comprehension and informed consent: assessing the effect of a short consent form

A polio immunization pamphlet with increased appeal and simplified language does not improve comprehension to an acceptable level

Interventions to improve research participants' understanding in informed consent for research: a systematic review

Consent forms: how to determine and improve their readability

A parent-driven, computer-based vaccine information system: addressing variability in information needs for the varicella vaccine

We thank the scientists, companies, and tens of thousands of clinical-trial coordinators and volunteers around the world that have given their time in order to advance the research producing safe and effective vaccines against COVID-19 for the world.J o u r n a l P r e -p r o o f

significant work needs to be done to improve the readability and comprehension of informed consent documents.