key: cord-0215452-uoa41g2f
authors: Sharma, Shakshi; Sharma, Rajesh; Datta, Anwitaman
title: Misleading the Covid-19 vaccination discourse on Twitter: An exploratory study of infodemic around the pandemic
date: 2021-08-16
journal: nan
DOI: nan
sha: 64858473bab587bb93db1263cd4a28ac6208bbb4
doc_id: 215452
cord_uid: uoa41g2f

In this work, we collect a moderate-sized representative corpus of tweets (200,000 approx.) pertaining Covid-19 vaccination spanning over a period of seven months (September 2020 - March 2021). Following a Transfer Learning approach, we utilize the pre-trained Transformer-based XLNet model to classify tweets as Misleading or Non-Misleading and validate against a random subset of results manually. We build on this to study and contrast the characteristics of tweets in the corpus that are misleading in nature against non-misleading ones. This exploratory analysis enables us to design features (such as sentiments, hashtags, nouns, pronouns, etc) that can, in turn, be exploited for classifying tweets as (Non-)Misleading using various ML models in an explainable manner. Specifically, several ML models are employed for prediction, with up to 90% accuracy, and the importance of each feature is explained using SHAP Explainable AI (XAI) tool. While the thrust of this work is principally exploratory analysis in order to obtain insights on the online discourse on Covid-19 vaccination, we conclude the paper by outlining how these insights provide the foundations for a more actionable approach to mitigate misinformation. The curated dataset and code is made available (Github repository) so that the research community at large can reproduce, compare against, or build upon this work.

"We live in an era of unprecedented scientific breakthroughs and expertise. But we're also stymied by the forces of misinformation that undermine the true knowledge that is out there." -Dr. Laolu Fayanju [Bosman et al. (2021)] A recent study Machingaidze and Wiysonge (2021) compares vaccine hesitancy in several low and middle income countries (LMIC) with vaccine hesitancy in the US and Russia, which were some of the countries at the forefront of Covid-19 vaccine research. The average vaccine acceptance rate in LMICs was reported to be 80.3%, compared to 64.6% in the United States and 30.4% in Russia. A complex set of reasons exist for the tremendous vaccination hesitancythis includes lack of adequate knowledge and certainty about Copyright © 2020, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. the virus itself, as well as the surprisingly rapid development and very short span of testing within which the vaccines had to be deployed given the magnitude of the global pandemic.

The fears, uncertainties, and doubts (FUD) around Covid-19 and the currently available vaccines are amplified by the ongoing discourse in social media -leading to an infodemic (a portmanteau of information and epidemic, referring to the spread of possibly accurate and inaccurate information about a disease, itself spreading like an epidemic). The infodemic is preyed on and further fuelled by prolific misinformation spreaders (for instance, the 'disinformation dozen' The Center for Countering Digital Hate (2021)), who may have special interests and agendas in doing so.

Given such relentless assault of misinformation and the immense impact it has on the society at large, it is imperative to foremost characterize the nature of such misinformation as well as be able to identify instances of such misinformation at scale, that is, in an automated manner. That is the thrust of our current work, specifically focused on the discourse on Twitter. Such understanding and detection mechanisms are vital to devising countermeasures, for example, given a specific piece of misinformation, quickly identify it as such and determine what might be appropriate factual information best poised to counter said claim; or identify the most prevalent misinformation and the particular FUDs they prey upon so that policy makers can determine and direct the resources for public messaging accordingly.

Overall, the issue at hand is of vital importance and pressing in nature. As such, it is being studied -both within academia (across multiple disciplines: medical scientists and epidemiologists, social and political scientists, as well as information and computer scientists), as well as by many other stakeholders, particularly among governance, public policy making entities, and traditional journalists and pollsters, beyond academia. The current work based on the study of openly accessible Covid-19 vaccine-related tweets fits within and adds to this broader effort. In particular, we would like to emphasize the strengths but also the shortcomings of our approach in comparison to traditional surveys. Traditional survey driven analysis, for example, Bosman et al. (2021) are able to collect and analyze details, particularly demographic breakdowns such as age, race, gender, education, income, geography, etc. that a study like ours is incapable of carrying out because such information is scarce if ever available. In contrast, a foremost benefit of using information from social media is its sheer scale. Furthermore, once the analysis methodology and the necessary data pipeline to do so is put in place, the exercise can be repeated or carried out continuously, and compare the evolution of the discourse and gauge the situation over (real-)time 1 . Given that most of the discourse is online, ironically even more so given that physical interactions are attenuated because of the pandemic, and online social media (OSM) platforms are where misinformation thrives -targeting a study of the social media content allows to identify not only the emerging misinformation but also the prolific agents responsible for their spread. Likewise, based on the data already collected, a robust model to classify tweets as being potentially Misleading can be realized. This direct and pinpointed access to the well of misinformation provides opportunities for real-time intervention with actionable intelligence that can be derived from the analysis we carry out in this work (we outline such possibilities in our concluding remarks).

The key highlights and contributions of the current work are as follows:

1. Data (following FAIR principle, i.e., findable, accessible, interoperable, and reusable): From an initial set of over 200,000 tweets which we originally collected, we denoised and curated a representative collection of 114,635 tweets related to Covid-19 vaccination, along with two mutually exclusive subsets of 1246 and 1000 tweets manually labeled in terms of whether they are Misleading or Non-Misleading (Section 3). Adhering with Twitter's content redistribution policy 2 , we provide the Tweet IDs (and our labels, when applicable) for these collections. We also provide the source code used in this work for data collection, processing, and analysis. These can be found at https://github.com/shakshi12/CovidVaccination.

We analyze the dataset across three dimensions: (i) Language Exploration (Section 4) utilizing syntactic structure and the principal themes involved in both Misleading and Non-Misleading tweets, (ii) Opinion Study (Section 5) leverage the sentiments and emotions of both types of tweets, (iii) Effect of Visibility (Section 6) involves analyzing meta-data of the tweets. These dimensions based on the tweets provide us insights in order to categorize the tweets into Misleading and Non-Misleading tweets.

3. Classification & explainability: Aforementioned analysis aided the identification of potential features which could be explicitly leveraged to classify tweets to determine whether they are Misleading or not. This explicit approach was compared against a black-box model (XL-Net), and the mutual consistency of these approaches, aided with the explainability dimension of our approach, reinforces the credibility of the classifiers (Section 7). The efficacy, as well as marginal contributions of a subset of features, were explored empirically for further validation.

Beyond the immediate value from the understanding it provides, this work lays the foundation for building tools for intervention to mitigate the spread of misinformation.

Online social media (OSM) witnesses active discussions related to vaccinations for various diseases, old and new, including measles Cossard et al. (2020) Yuan, Schuchard, and Crooks (2019) , Ebola virus, human papillomavirus (HPV), and the flu Raghupathi, Ren, and Raghupathi (2020) . In particular, researchers have studied the spread of misinformation Cossard et al. (2020) and the discourse among pro and anti-vaccination groups Yuan, Schuchard, and Crooks (2019) , as well as the role of bots on these platforms in steering the discussions driven by vested interests Yuan, Schuchard, and Crooks (2019) .

These studies have spanned diverse OSMs, including Facebook Ma and Stahl (2017) , Mejova and Kalimeri (2020) , Twitter Mitra, Counts, and Pennebaker (2016) , Germani and Biller-Andorno (2021), Cossard et al. (2020) , and e-Commerce platforms Juneja and Mitra (2021) , which have been criticized for tackling vaccine misinformation Wardle and Singerman (2021) . In Evanega et al. (2020) , authors study COVID-related misinformation that emerged in traditional media. Studies have also analyzed the financial influence and impact of anti-vaccination groups across multiple dimensions, including the revenues generated across various OSM platforms such as Facebook, YouTube, and Instagram Burki (2020) , in particular through advertisements promoted on Facebook involving misinformation narratives by using conspiracy theories to unverifiable claims Mejova and Kalimeri (2020) , which in part explains the deep entrenchment of such false narratives in the public discourse. It is common to see the involvement of politicians and influencers in anti-vaccination campaigns, who cheer-lead and exploit misinformation Cossard et al. (2020) , and are, in turn, often used by anti-vaccination proponents as reference points Germani and Biller-Andorno (2021) . In addition, a longitudinal study has shown that conspiracy theories and government distrust are often used by long-standing antivaccination proponents to recruit new individuals among their cohort Mitra, Counts, and Pennebaker (2016) .

A few AI based studies have also been carried out with respect to vaccine misinformation. For example, in Mønsted and Lehmann (2019) , authors train a deep neural network for predicting tweet vaccine sentiments, and in Sear et al. (2020) , machine learning was used to quantify covid-19 researchers' content in the online health opinion. In Ma and Stahl (2017) , a study employed a multimodal critical discourse analysis approach to analyze the textual and graphic information within a public anti-vaccine Facebook group. In Juneja and Mitra (2021) , authors conduct two sets of algorithmic audits for vaccine misinformation on the search and recommendation algorithms of Amazon-the world's leading retailer.

One of the closest works to that of ours is Potthast et al. (2017) , where authors analyzed 1,627 articles from highly partisan (left and right) using stylometric techniques. In this work, we try to explore the Misleading and Non-Misleading tweets into three dimensions, namely, syntax exploration, opinions, and utilizing meta-data of the tweets. In addition, we also try to use these features and predict the tweets with explainability.

The data collection process is described in this section, followed by the processing steps.

The first news of the world's COVID-19 vaccine registration 345 in August 2020, as well as Trump's order to carry out vaccination even before it had been thoroughly tested and approved 67 , signaled a vaccine rush among various countries around the world. Naturally, this amplified manifold the discussions around Covid-19 vaccines both offline as well as online. We examine what type of discussions about COVID-19 vaccination spread over Twitter since it provides an easy to mine source of information representing all the important narratives. We collected tweets related to COVID-19 vaccination for the period from September 2020 to March 2021. Most countries had begun vaccine rollouts 8910 as of March 2021, and gradually people are less hesitant to get vaccinated 11 even though a significant part of the population remains so. Thus, the period leading up to large-scale vaccine rollout is critical for studying and understanding the nature of misinformation and its spread. Given the restrictions of Twitter API, collecting an exhaustive dataset is not feasible, and as such, we aimed for a representative sample instead. We queried the Twitter Streaming API with a wide variety of relevant keywords, for example, -vaccine, anti-vax, anti-vaccination, antivaxxer, antivaccine, CovidVaccine, COVID19, Chinesevirus, covax, COVIDVaccine, COVIDVaccination, COVAX, etc. in order to collect tweets related to COVID-19 vaccination. This resulted in over 200,000 tweets. After filtering the tweets in English, the final dataset used in this study has 114,635 tweets.

In order to perform the analysis, we further process and label the tweets as Misleading and Non-Misleading.

Data Cleaning: To clean the tweets, we apply standard NLP techniques such as removal of white space, nonalphanumeric characters, lowercase, stop words, URL links, apostrophe replacement, tokenization, and Porter stemming. However, we did not remove the stop words while analyzing the Syntactic dimension in Section 4.

Annotation Task: Next, we discuss the annotation process of the tweets. We designate a tweet to be Misleading when the content of a tweet deviates from the evidence shared by news media or reputable sources such as the WHO (World Health Organization), and even if it uses facts in parts, it might add connotations to it that encourages vaccine hesitancy. Otherwise, we consider the tweet as Non-Misleading. For instance, the tweet, "I wonder if the type of covid vaccine we get will become the new stereotype or derogatory term people use. Imagine someone talkin shit bout you just cuz you got Pfizer lmaooo" spins a narrative to increase people's distrust. Thus, in this work we consider it as a Misleading tweet. Another example of a Misleading tweet is, "[username] 12 I know its so bloody weird that Gates funded every single vaccine moderne Pfizer Oxford etc WTF. Something un my gut says dont have the vaccine. I dk why -it just a feeling I can share." Parts of these tweets might be true, but it was written in such a way that it tries to alter the facts thereby, encourages vaccine apprehension and fear. As such, we deem them misleading and analyze the nature of such tweets.

The reason for such Misleading tweets could be the result of the widespread COVID-19 vaccination half-truths and myths 13 . Examples of Misleading tweets from our dataset includes -"The COVID-19 vaccine is not safe because it was rapidly developed and tested", "I already had COVID-19 and I have recovered, so I don't need to get a COVID-19 vaccine when its available", "COVID-19 vaccines will alter DNA", "The COVID-19 vaccine was developed to control the general population either through microchip tracking or "nanotransducers" in our brains". These narratives are designed to prompt vaccine hesitancy Cossard et al. (2020) and eventually becomes the root cause of further Misleading information.

Since data annotation is a costly and labor-intensive operation, we explore the application of Transfer Learning Pan and Yang (2009) to annotate the tweets as Misleading or Non-Misleading. To that end, we took a random sample of 1246 tweets which is a representative of the complete dataset, and manually annotate these tweets as either Misleading or Non-Misleading tweets. We maintained a balanced dataset. After labeling these 1246 tweets, we fine-tune the XLNet 14 language model, which is an extension of the Transformer-XL model, and learns the bidirectional contexts. BERT and RoBERTa were also tested. XLNet, however, outperforms all other models for our dataset. Table 1 , row 1 shows the evaluation metrics calculated on the XLNet model's validation set. Precisely, 389 out of 1246 tweets are validation set, and the rest are used as a training set. Finally, we use our fine-tuned XLNet model to annotate the rest of the tweets.

In order to validate the efficiency of the annotated tweets, we manually verify 1000 random tweets. This sample is also balanced in nature. Table 1 , row 2 shows the evaluation metrics of the manual validation process. The accuracy of the sample is 0.98, indicating that our model has been well-trained, and we can rely on these labels to get further insights. The dataset 15 along with obtained labels as well as the 1246 tweets with their manual labels is available at repository 16 .

Next, we explore our dataset using three dimensions: Language Exploration (Section 4) covering Syntactic and Topic Analysis, Opinion Study (Section 5) comprising Sentiments and Emotions, and Effect of Visibility (Section 6) involving exploration of tweets' meta-data. 

After assigning Misleading and Non-Misleading labels for each tweet (as discussed in Section 3), we study ten Syntactic aspects (attributes) to distinguish the structural patterns of both types of tweets. First, we visualize the Syntactic distributions of both Misleading and Non-Misleading tweets. Next, to validate that the difference in both the distributions are indeed significant, we use Kolmogorov Smirnov Test 17 . First, we look at the Nouns, the main building blocks of any sentence. We observe from Figure 1 (a) that visually there is a slight variation in both distributions. To determine whether this difference in the distributions is statistically significant, we calculated the p-value of Nouns (see Table 2 , row 1), which is much lower than the significance level, which implies that the two distributions are in fact dissimilar. Second, Pronouns are the substitute for Nouns. Figure 1 (b) shows that Pronouns are more used in Non-Misleading tweets than Misleading tweets. Third, Type-Token Ratio (TTR) measures the lexical diversity (quality) 14 https://huggingface.co/transformers/model doc/xlnet.html 15 The dataset is shared following the Twitter Policy rules 16 https://github.com/shakshi12/CovidVaccination 17 https://www.itl.nist.gov/div898/handbook/eda/section3/eda35g.htm of the text. Specifically, it is the ratio between the total number of unique words (types) in the text and the total number of words in the text. The higher the value of this ratio, the higher the lexical diversity of the text.

We notice in Figure 1 (c) that the distributions are rightskewed, indicating that the text is of good quality in terms of lexical diversity. However, we observe some differences between the distributions. In contrast to Non-Misleading, the mean of the Misleading distribution is below 90. In addition, when TTR is near to 100, the density of the Misleading distribution is lower in comparison to its mean. Whereas, Non-Misleading distribution has a similar density with respect to its mean. This implies that Misleading tweets are less lexically diverse in contrast to Non-Misleading tweets.

In addition, the p-value is present in Table 2 , row 3, which is lower, indicating the distributions are different. Fourth, Stop words such as a, an, the, is, be are often part of the text. It is clear from Figure 1 (d) that the mean of the Misleading and Non-Misleading distribution is close to 22 and 18, respectively. Furthermore, Misleading distribution has a flatter peak. This implies that majority of the Misleading tweets use more Stop words. Fifth, Verbs are used in a text to describe any action, occurrence or state of being. It is visible from Figure 1 (e) that the spread of the Misleading distribution is skewed towards right side of the graph than the Non-Misleading distribution. Sixth, Conjunctions are the words that are used to connect the words (sentences). We observe from Figure 1(f) that there is a slight variation in both the distributions. Seventh, Adverbs qualifies or modifies the Verbs, Adjectives, or other Adverbs. It is clear from Figure 1 (g) that the distribution of Misleading tweets is more spread than the Non-Misleading tweets.

We investigate other Syntactic aspects namely, Determiners (Figure 1 (h)), Adjectives (Figure 1 (i)), and WH-words (Figure 1(j) ). Their p-values present in Table 2 , row 8, 9, and 10. Although these values are near but less than significance level, we consider them distinguishable attributes in finding Misleading and Non-Misleading tweets. 

Next, using topic modeling, we inspect the top five most talked-about topics among Misleading and Non-Misleading tweets (see Table 3 ). 2) Shots & Real Side-Effects: These two themes are about informing individual's actual experience after getting the vac-cine shot. We provide an example of a tweet which is related to both the themes -"I had the PFIZER shot(s), 30 days apart, and both times it didn't hurt. My arm was sore later on for a day or two. 24 hours after the second shot I got a fever and chills and promptly went to sleep. Woke up 10 hours later feeling great. No other side effects. GET THIS VACCINE!" 3) Vaccine Efficacy: Both Misleading and Non-Misleading have a shared theme. This indicates that they are both discussing the vaccine's efficacy. Misleading tweets, on the other hand, aim to cause uncertainty regarding vaccine efficacy, while Non-Misleading tweets emphasize the positive aspects of vaccine efficacy, such as -"#JabMe: A single shot of either the Pfizer or Oxford vaccine provides about 80 percent protection against being treated in a hospital, according to the latest data from the UK vaccination program." 4) Data & Facts: Non-Misleading tweets are more concerned with presenting accurate information, such as -"BBC: Around 5 million Europeans have already received the AstraZeneca vaccine. Of this figure, about 30 cases had reported "thromboembolic events" -or developing blood clots. European medicines regulator said there was no indication the jab was causing the blood clots."

The top five themes indicate that the different subjects explored in both types of tweets. Precisely, Misleading tweets mostly misleads the reader using political dimension or raise fear among the people for vaccination. In comparison, Non-Misleading tweets discuss the real side-effects of the vaccination and try to bring the facts with evidence.

Previously, we explored the Syntactic dimension. We now look into the second dimension, the role of Opinion in relation to Misleading and Non-Misleading tweets.

We first explore the impact of sentiments on Misleading and Non-Misleading Covid-19 vaccination tweets, considering three broad categories of sentiments: Positive, Negative, and Neutral. The sentiments are calculated using VADER API 21 . Figure 2 shows that Negative sentiments are more prevalent in Misleading tweets followed by Positive and Neutral sentiments. An example of a Misleading tweet with Positive sentiment is "A little Angel in my dreams today told that our bodies will be developing antibodies on its own within a few days without vaccination". While a reading of it indicates vaccine prejudice and skepticism, the sentiment analysis tool latches upon the positive sounding phrases in there.

We then identify the topics which are being discussed with respect to sentiments. Apart from 'Vaccine Efficacy' which confirms the above mentioned tweet's theme, the related topics 'Operation Warp Speed' and 'Trials' are also identified within the Positive sentiments of Misleading tweets. (See Table 4 Figure 2 : Sentiment Analysis. The x-axis and y-axis denote the labels of the tweets and percentages, respectively.

with Positive sentiments inject the negativity by sugarcoating the tweets with positive words to easily trick people into either believing in their positive hypothetical situations or providing a new dimension to the topic.

Positive sentiments on the other hand dominate in Non-Misleading tweets, though a substantial portion of them again have Negative or Neutral sentiments. An instance of a Non-Misleading tweet with Negative sentiment is "Dr Kathrin Jansen, Pfizer's head of vaccine development: We were never part of the Warp Speed ... We have never taken any money from the U.S. government, or from anyone. Trump is a liar". In this instance, the Negative sentiment of the Non-Misleading tweet is due to it counteracting the Misleading information. Many facts and news related to the pandemic have naturally Negative sentiments. Similar to this tweet argument, topics that are discovered in the Negative sentiments of Non-Misleading tweets are Operation Warp Speed and Vaccine Efficacy, in addition to, Trials and Data & Facts (See Table 4 ). These Non-Misleading tweets with Negative sentiments indicate that they are either attempting to clarify claims against Covid Vaccination's Development Companies or myths against the vaccination process with their choice of negative words. Overall, Negative sentiments are more common in Misleading tweets, whereas, Non-Misleading tweets have more Positive sentiments. We go through five different emotions in detail in the next Section. 

The sentiments serve as the foundation for analyzing the tweets. As a result, we dig deeper into the impact of emotions on tweets. Figure 3 displays the five different emotions -Anger, Fear, Happiness, Sadness, and Surprise. In Misleading tweets, the most common emotion is Fear, followed by Surprise, Sadness, Happiness, and, finally, Anger. Whereas, Non-Misleading tweets have a tie for the first place with Fear and Surprise, followed by Happiness, Sadness, and at last, Anger. We observe that Fear and Surprise are the two most popular emotions in both types of tweets, which is understandable given that 45% of unvaccinated people are afraid to get the vaccine because they are worried about the adverse sideeffects 22 .

To confirm this, we look into the topics around the Fear and Surprise emotions. The topics which are similar to the above statement are Trials and Vaccine Efficacy in both Misleading and Non-Misleading categories (See Table 5 ).

However, the emotion Fear is higher in Misleading tweets in contrast to Non-Misleading tweets, which is attributable to the fact that most Misleading tweets reference fake and fabricated vaccine side-effects which misleads the users with false stories of Operation Warp Speed (Table 5 ).

In contrast, emotion Surprise is higher in Non-Misleading tweets than Misleading tweets. Upon closer look at the data, we found that a significant part of the Non-Misleading tweets with emotion Surprise discuss the governments' fast response towards vaccination, fitting into the Data & Facts topic.

Furthermore, emotions, Anger, and Sadness are higher in Misleading tweets. One of the possible reasons could be that these tweets often involve a political dimension and accusing the government of not making the right decisions.

The matching topics under both emotions are Politics, Vaccine Availability, and Operation Warp Speed in the Misleading category.

Non-Misleading tweets that have emotion Happiness discuss their experience about receiving the shot and facing no bogus side-effects spreading across the Internet. A related topic is Real side-effects in the Non-Misleading category.

To summarize, the majority of the Misleading tweets have Fear emotions more than Non-Misleading tweets.

Next, we explore the emotions defined by the NRC-(VAD) lexicon 23 : Valence, Arousal, and Dominance. These emotions assist in comprehending the words that are more conducive to specific emotions.

The top 30 contributing words for emotion Valence are shown in Figure 4a in the form of a word shift plot, quantifying which words contribute to a difference between the two groups, and how they contribute. Contributing words in Misleading tweets include speed, money, kill, die, danger, stop, while Non-Misleading tweets contain words like shot, ill, reaction, profit. In addition, Misleading tweets have more words for emotion Valence in the top 30 than Non-Misleading tweets. This analysis also confirms that the sentiments of Misleading tweets are more Negative, and Non-Misleading tweets are more Positive. Figure 4b represents the contributing words for emotion Arousal. We notice that contributing words in Misleading tweets are die, kill, danger. Whereas frequent words such as speed, shot, money, reaction are found in Non-Misleading tweets. Also, Misleading tweets contain more Arousal words in the top 30 than Non-Misleading tweets. Figure 4c represents emotion Dominance. In Misleading tweets, contributing words are trump, effect, kill. In Non-Misleading tweets, contributing words are speed, money, chief, hope. One thing to note is that the words such as money, speed occurs in both types of tweets. The possible reason could be that these words are used heavily in context of Trump's involvement in Operation Warp Speed. However, it conveys different meanings in both types of tweets. Non-Misleading tweets contain more Dominant words in the 23 http://saifmohammad.com/WebPages/nrc-vad.html top 30 than Misleading tweets. We infer that Misleading tweets have more Valence and Arousal words, whereas Non-Misleading tweets have more Dominant words.

So far, our analysis was confined to the content of the tweets themselves. The focus of the third dimension, looking at information and meta-data in the tweets that influence their visibility, e.g., words used, hashtags, likes, etc. to study whether there are distinctive characteristics across Misleading and Non-Misleading tweets.

Certain words are used more frequently in the tweets than others. In Figures 5a and 5b , we use Word Clouds to summarize this for both Non-Misleading and Misleading tweets visually. The relative frequency of the words is reflected in the size of the words. The Figures show that the most frequent words in both Word Clouds are completely different, indicating that the choice of the words in both types of tweets significantly varies. Shot, report, jab, ill, sore, and fact are all recurring words in the Non-Misleading Word Cloud, whereas, wait, death, risk, effect, trump, and die are all frequent words in the Misleading Word Cloud.

To study this difference quantitatively, the top 50 words from both the Misleading and Non-Misleading classes are then extracted along with their relative ranking information. Only seven words were found to be common in both classes. We also computed the Kendall Tau correlation coefficient 24 on the union of the top 50 Misleading and Non-Misleading words. A score of -0.81 was observed, showing disagreement between the word groups (the Kendall Tau range is [-1, 1], with -1 indicating strong disagreement and 1 indicating strong agreement). This clearly suggests that frequently recurring words are unrelated, implying that the word choices in both classes differ. We also plot the top 30 most frequent words present in both classes for comparison purposes. Figure 6 is the Shannon Entropy Word Shift 25 that finds the surprising words by ranking the difference between the entropies. It can be observed that words used in the Non-Misleading class are very distinctive from the Misleading class. Non-Misleading words -Warp, Speed, shot, sore mentions about the real side of the story such as Operation Warp Speed (initiated by the 25 https://shifterator.readthedocs.io/en/latest/shifts.html US government to facilitate development and distribution of the Covid vaccines), people expressing the actual sideeffects they face after getting jabbed. On the other hand, trump, damage, trust, kill, stop are the Misleading words used to describe the false stories of vaccination.

Extensive usage of hashtags is a popular way to enhance the targeted exposure of the tweets. We investigate the hashtags from two perspectives. Unique Hashtags: We explore such hashtags that are relatively unique to Misleading versus Non-Misleading tweets. Table 6 lists some of the popular ones. Note that the hashtags mentioned in the Table are chosen depending on how many times they appear in the tweets. In Misleading tweets, the #untestedvaccine clearly indicates that the tweet refers to one of the vaccine myths. In contrast, the #vaccinatedandproud represents that tweet is in support of the vaccination process. Thus, the choice of the hashtags can provide a clue about the Misleading tweets. Co-hashtags: We also consider the combination of hashtags that frequently occurred together in a tweet, i.e., co-hashtags. In Non-Misleading tweets, we find 280 cohashtags, while 86 co-hashtags are found in Misleading tweets. After filtering those co-hashtags that occurred more than once, we found that co-hashtags repeatedly occurred only in Non-Misleading tweets. There is no pattern (consistency) concerning co-hashtags in Misleading tweets, making their hashtags more random.

The number of Retweets, Replies, and Likes count are all essential visibility attributes. Figure 7 depicts the mean values of the counts of Retweets, Replies, and Likes for both regardless of the type of information it contains. However, there is a variation in the Retweets count and Likes count. Relatively, the Misleading tweets get fewer Retweets and Likes than Non-Misleading tweets.

The names of the Covid-19 vaccinations are frequently mentioned in tweets. In this regard, we attempt to assess the influence of vaccine names in both Misleading and Non-Misleading tweets. From the dataset, we discovered five popular vaccines: Pfizer, Moderna, AstraZeneca, Covaxin, and Johnson & Johnson. These names are used either in an individual or combined manner. Figure 8 shows that the proportion of Misleading tweets is lower than the proportion of Non-Misleading tweets until the number of vaccine names is fewer than or equal to three. When the count reaches four or five, the number of Misleading tweets begins to rise. This means that when the number of vaccine names in a tweet grows more than three, the likelihood of a Misleading tweet also grows. Figure 8 : Count of Vaccine Names used in a Tweet. X-axis and y-axis in Figure 8 (a) represents the count of the vaccines' names and percentages, respectively. Value 0 on the x-axis corresponds to no mention of the vaccine name in the tweet, Value 1 denotes mention of one vaccine, and so on. Figure 8(b) shows the percentage of the tweets with respect to the count of the vaccine name.

We also look at how the tweets' creation time influences

Misleading information. Figure 9 shows the generation of tweets month by month from September 2020 to March 2021.

9/20(1.2) 10/20(8.5) 11/20 (15) Figure 9 : Tweet Creation Time (Monthly). X-axis and y-axis represent the year-month and percentages, respectively. The numbers inside the brackets denote absolute numbers of the tweets in thousands.

Non-Misleading tweets peaked in November 2020, whereas Misleading tweets peaked in January 2021. This could be due to the fact that several data reports and guidelines 26 , such as California Reports Allergic Reactions to Moderna Vaccine, Who can take the Pfizer-BioNTech COVID-19 vaccine?, New variant identified in Japan, On the use of COVID-19 mRNA vaccines in pregnancy, were released in January 2021. As a result, it appears that when data and guidelines were published, false information based on misinterpretations rose on Twitter.

From the analysis presented so far, we conclude that the writing styles, as well as meta-information carry distinctive characteristics that may help segregate the Misleading and Non-Misleading tweets. Practically, presence of such signals and explicitly understanding their behavior aides improve

Our analysis demonstrated that the writing styles of the Misleading and Non-Misleading tweets are clearly distinct. Now, we use these writing patterns as features in machine learning models to predict whether tweets are Misleading or not. The features, and not the actual content of the tweets is considered in the prediction task. The primary aim is to determine whether or not these writing styles are sufficient to categorize the tweets into Misleading and Non-Misleading tweets. In addition, we scrutinize the features in terms of their contribution to the prediction, in an effort to enhance understandability of the obtained results.

By providing the tweets as an input to the pre-trained XLNet model, the focus was to obtain the labels for the tweets. These labels are treated as 'ground-truth' for the prediction task using the features. In this section, instead of using the tweets themselves, we use the descriptive features described in the previous sections 4-6 as an input to the various machine learning models to classify Misleading and Non-Misleading tweets. The purpose is to check if these descriptive features can distinguish Misleading tweets (it does). This helps us understand more explicitly the differentiating characteristics across non/misleading tweet. Furthermore, such feature based classification can potentially be re-applied, as in transfer learning, to other domains beyond Covid-19. Specifically, the features constitute -Stop words, Pronouns, Nouns, Adjectives, Average length, WH-word, Adverbs, Conjunctions, Verbs, Determiners, TTR, Sentiments, Emotions, and Hashtags. The dataset was divided into balanced train and test set in an 80:20 ratio.

We apply five-fold cross-validation on the train set, which accounts for 80% of the entire data. Each fold further divides the train set into fold-train and fold-test sets to train and evaluate the model. Finally, we assess the performance of the trained model on the unseen test set, which is 20% of the entire data. Table 7 shows the evaluation metrics for the test set, ordered in descending order of accuracy. As it can be seen, the ensemble-based model, that is, Random Forest performs best in our case with an accuracy of 0.90, followed by Extra Trees, Decision Tree, and so on. This demonstrates that the writing styles can effectively segregate the Misleading and Non-Misleading tweets. Furthermore, other measures from all models, such as Precision, Recall, F1 Score, and AUC (Area Under The Curve) ROC (Receiver Operating Characteristics) Score demonstrate that our results are consistent throughout, implying that the trained models are generalizable. 

Next, to assess the contributions of each feature in the classification task, we use the SHAP 27 Explainable AI tool. This tool assists us in determining the significant features in the prediction by computing the average marginal contributions of each feature. The importance of the features (or simply SHAP ranking) is shown in Figure 10 in descending order. The most significant contributor, Sentiments, has a negative impact on prediction, implying that a lower value of Sentiments predicts Misleading class and vice versa. This makes sense because higher Sentiments values imply positive sentiments, and lower values suggest negative sentiments, which is consistent with the findings from Section 5 that Misleading tweets contain more negative sentiments. Furthermore, Nouns, Emotions, and Conjunctions features have negative impact on Misleading tweets. This implies that Misleading tweets contain less number of Nouns and Conjunctions compared to Non-Misleading tweets. This might be due to the fact that focus of the Misleading tweets is to use fancy words or catchy phrases to attract the readers rather than presenting proper facts using Nouns and Conjunctions. The remaining features have positive impact; for example, unlike Nouns, Misleading tweets have a higher number of Pronouns than Non-Misleading tweets.

Feature Ablation Study After evaluating the importance of features, we try to see if there is a decline in accuracy if particular features are not included. Essentially, we try a few different settings by removing some of the features based on their importance as determined by SHAP ranking in Figure  10 and then rerunning all the models in the same environment. Note that we only show results of the best-performing model, Random Forest, due to space constraints. Please note that the best accuracy attained with all features is 90%. We start by removing Emotions and the rest of the features listed below as per the SHAP plot. Figure 10 : Feature Importance using SHAP tool. The x-axis and y-axis denote the SHAP values and features' names, respectively. Each data point refers to an instance of the dataset. The red color indicates a higher value for the feature than its average value, whereas the blue color denotes a lower value. Red values on the right side of the x-axis indicate a positive impact on the prediction and vice versa.

Features are sorted in descending order (best seen in color).

dataset, the values of the evaluation metrics decline, indicating that they are truly relevant. Next, we remove features that are less important than Emotions, such as Type token ratio and the remainder of the features (shown in Table 8 , row 2). We continue to run experiments and discover that even the least significant feature, Hashtags, contributes to the model's improvement. These results indicate that all of the features we discussed are both valuable and necessary for detecting Misleading tweets. Correlation and the SHAP Ranking Is there any association between the features' correlation values and the SHAP ranking? The hypothesis is that the highly correlated features should be close in the SHAP ranking. The correlation between each feature pair are shown in Figure 11 . The dark color denotes a strong correlation between the two features based on the absolute value and vice versa. Please note that the correlation between the features does not surpass a certain threshold. This is why we use all of the features in the classification task. The numbers in the brackets next to the feature names correspond to the feature's SHAP ranking. One thing to note is that the highly correlated features are always positive. Furthermore, it can be observed that highly correlated features are also close in SHAP ranking. For instance, Stop words are highly correlated with Verbs and score near to each other in the SHAP ranking compared to the less correlated features. Sentiments and Determiners is another example. Sentiments are least correlated with Determiners and, thus, farther from each other in SHAP ranking, demonstrating that our hypothesis is indeed true.

In this paper, we carried out an exploratory analysis of the content and meta-information associated with tweets pertaining to Covid-19 vaccines to determine the characteristics of both Misleading and Non-Misleading tweets. The topic detection aspect of our study helped establish the main themes of discourse across these categories, as well as identify potentially distinguishing characteristics. The latter were explored as features to carry out a classification task, where the observed outcomes support explainability. We observe that this explainability property coupled with the aforementioned identification of the topic of tweets, actionable intelligence can be generated, which determines a principal thrust of our future work. In particular, the mechanisms studied in this current work can be used to preliminar-ily shortlist potentially problematic tweets at an early stage and use that, in turn, to even identify accounts with prolific contribution is spreading misinformation, and accordingly (ii) put in targeted mechanisms to reduce the virulence of their spread until vetted for authenticity, and additionally or alternatively (ii) device or promote counter-messaging to mitigate and dispel such misinformation. Moreover, such counter-messages can be readily identified by using the same mechanism of topic detection and classification of Non-Misleading tweets. Furthermore, we want to explore whether the approach laid out can be generalized to identify Misleading tweets on other topics beyond Covid-19 vaccination.

Beyond the extension of the work to the aforementioned application, there is also an opportunity to refine the techniques by carrying out an analysis that is fine-grained in geographic, temporal, and linguistic dimensions: for example, which Misleading tweets are more prominent and specific to certain regions, which of them persist over what span of time, and doing so in languages beyond English.

Who are the unvaccinated in america? there's no one answer

The online anti-vaccine movement in the age of covid-19

Falling into the echo chamber: The italian vaccination debate on twitter

Coronavirus misinformation: quantifying sources and themes in the covid-19 'infodemic

The antivaccination infodemic on social media: A behavioral analysis

Auditing e-commerce platforms for algorithmically curated vaccine misinformation

A multimodal critical discourse analysis of anti-vaccination information on facebook. Library & Information

Understanding covid-19 vaccine hesitancy

Advertisers jump on coronavirus bandwagon: Politics, news, and business

Understanding anti-vaccination attitudes in social media

Algorithmic detection and analysis of vaccine-denialist sentiment clusters in social networks

A survey on transfer learning

A stylometric inquiry into hyperpartisan and fake news

Studying public perception about vaccination: A sentiment analysis of tweets

Quantifying covid-19 content in the online health opinion war using machine learning

The Center for Countering Digital Hate. 2021. The disinformation dozen

Too little, too late: social media companies' failure to tackle vaccine misinformation poses a real threat

Examining emergent communities and social bots within the polarized online vaccination debate in twitter