Disinformation on Twitter Increased During Periods of Turmoil

Research

ZEW Study Analyses Almost Two Million Tweets on the War in Ukraine

A study by the ZEW, Erasmus University Rotterdam and the University of Mannheim shows that, during the transition of the short-message service Twitter to X, the volume of serious disinformation regarding the war in Ukraine rose sharply.

During the transition of the short-message service Twitter to X, the volume of severe disinformation about the war in Ukraine rose sharply. These are the findings of a study conducted by ZEW Mannheim together with Erasmus University Rotterdam and the University of Mannheim, in which around 1.9 million English-language tweets and 6.5 million retweets about the war in Ukraine from late March to mid-May 2023 are analysed. A tiny fraction – 0.08 per cent – of all accounts produced nearly one third of the moderate to severe disinformation circulating in the sample. Certain narratives, such as the delegitimisation of Ukraine or the inversion of aggressor and victim, were spread particularly frequently by these accounts.

“In October 2022, Elon Musk acquired the short-message service Twitter, which he renamed X in July 2023. In April 2023, the restrictions on accounts belonging to state-funded media outlets were lifted and a clear increase in disinformation accounts and related posts occurred. We observe that many accounts which later spread large amounts of false information were created precisely during periods of major turmoil on the platform. This might indicate broader strategic preparation,” says Dr. Raphaela Andres, co-author of the study and researcher at ZEW’s “Digital Economy” Research Unit.

General documents

Download ZEW Analysis

Disinformation in the context of platform rules

Bar chart showing the breakdown of the narratives and the number of original tweets for each.

The study identifies a striking correlation between platform-rule decisions made by X and the intensity of disinformation. In the period from 26 March to 18 May 2023, two significant peaks in the spread of false information were observed. The first and strongest wave coincided with the period from 6 to 8 April 2023, when X lifted visibility restrictions on accounts affiliated with state media and changed its labelling of sources.

More disinformation accounts created during periods of turmoil at X

A timeline from January 2022 to June 2023 showing the daily number of newly created accounts spreading disinformation.

A large proportion of the accounts spreading this disinformation were created during periods of major turmoil at X. The sharpest increase in newly created disinformation accounts occurred on 28 October 2022, when Elon Musk officially took over Twitter. On that day alone, more than 2,400 of such accounts were created in our sample – an almost 60-fold increase compared with the daily average. Further peaks occurred during waves of redundancies at X.

The network analysis reveals a very high concentration of activity among these accounts: Just 0.08 per cent of all accounts in the sample were responsible for 30 per cent of all posts containing moderate to severe disinformation that have been reshared in the network. These accounts operate in closely interconnected clusters that are clearly distinct from the rest of the network and efficiently amplify false information.

“Our findings suggest that there is a connection between the expansion of disinformation infrastructure and the regulatory gaps during the transition from Twitter to X,” says Dr. Raphaela Andres.

Risk of disinformation entering AI training data

The increased volume of disinformation on X may also have implications for AI systems. Large language models (LLMs) are trained on vast amounts of user-generated content from the internet and disinformation narratives may thus enter the training data of AI systems. This could result in AI reproducing or unintentionally amplifying false information. 

“If this false information becomes part of the datasets used to train AI models, this could distort the content generated by future AI systems, which increasingly inform individual choices. Without effective countermeasures, there is a risk of a vicious circle in which AI systems further automate the disinformation on which they are trained,” says co-author Dr. Olga Slivko from Erasmus University Rotterdam.

1.9 million tweets analysed

The study is based on a comprehensive analysis of around 1.9 million English-language original tweets and 6.5 million retweets collected for the period from March to May 2023 using specific keywords relating to the war in Ukraine. To process the vast amount of data, the researchers developed an automated classification system based on a locally hosted LLM. The model assessed each tweet according to the severity of its disinformation and assigned it to one of twelve pro-Russian narratives, including “elites vs. the people” and “delegitimisation of Ukraine”. 

The reliability of the AI-assisted analysis was validated by comparing its results with human assessments carried out by three independent experts. The findings confirmed the suitability of LLMs for the large-scale monitoring of disinformation. In addition, a network analysis was conducted to determine the structure of the dissemination channels and the centrality of individual accounts. 

Additional Information