One of her first research projects was entitled “Detection of Toxicity and Aggression in Postings and Comments on the Net” – in short DeTox. It focused on developing and applying text analysis and filters for hate speech with the help of artificial intelligence. It was also back then that she started working with the Hessian Ministry of the Interior, which set up the “HessenGegenHetze” (“HesseAgainstHate”) reporting office in 2020, which citizens can contact if they are themselves victims of hate speech or want to report such incidents.

Since December 2023, the Ministry of the Interior in Wiesbaden has again been funding cybersecurity projects that contribute to “developing innovative solutions that can also serve to protect our digital infrastructures and our security in practice,” explains Stefan Sauer, State Secretary of the Interior. The h_da research project “BoTox – Bot and Context Detection in the Environment of Hate Speech” is one of them. Professor Siegel and her team have €292,000 at their disposal up until 2025. “We are focusing,” she says, “on recognising the relevance of hate speech in terms of criminal law.”

The researchers, who include a research associate and four student assistants, want to develop a system that automatically detects when texts and conversations fulfil possible criteria such as insults, sedition, or incitement to violence. “We have identified twelve different criminal offences,” says Melanie Siegel. With the help of automatic learning and artificial intelligence, the new application will not only detect, analyse and pre-classify such content but also forward it directly to the “HessenGegenHetze” reporting office. The aim is to support the office’s work as well as make it easier and more efficient. “So that action can be taken more quickly if there is a threat,” she says. To do this, the research team needs, of course, to be familiar with the law and with basic legal principles. This expertise is provided by Professor Dirk Labudde from Hochschule Fresenius in Idstein, with whom Melanie Siegel has collaborated on previous projects. Labudde is a legal counsellor and professor of digital forensics.

All these aspects and training the software to automatically detect hate speech call for a vast amount of data. An important element here is that the researchers need a sufficiently large number of relevant examples of hate speech for the system to learn to differentiate in the first place, but also a lot of irrelevant comments. In addition, the data must reflect reality. She reports that men post hate speech far more often than women and that, according to the findings, ultra-right groups are more active than ultra-left ones. The datasets must reflect this, too. The basis for the analysis is also important. Three of her student assistants read the same comments and then rate them. Is it an extreme opinion, is it insulting, or does it even incite sedition? Do the students rate them in the same way? “A Sisyphean task,” says Siegel.

In previous research projects, Siegel and her team could make use of the messenger service Twitter. Thanks to a research licence, they were able, for example, to collect around one million Twitter data from comments on political TV shows for the DeTox project. Today, however, Twitter (or rather X, as it is now called) is no longer accessible “because research licences have become so expensive that no one can afford them anymore,” she reports. That is why the team now has to resort to other platforms such as Facebook, Telegram or YouTube, “and access is not easy there either.” However, the data from the “HessenGegenHetze” reporting office, which is meanwhile better known, are helpful. While users initially reported around 2,000 comments per month, the office now receives the same number per week.

However, the team also continues to use the existing training data. Although not all topics have been mapped – COVID-19 was still unknown and the war in Ukraine had not yet erupted – “we are working on making the data transferable,” explains Melanie Siegel. In some cases, this is easy: “There was a similar choice of words for Chancellor Merkel, for example, as there is today for the current government.” Many topics have also stayed the same: anti-Semitism, Holocaust denial, xenophobia, migration, racism, or discrimination against minorities.

The project will also incorporate new scientific findings. For example, Mina Schütz, a doctoral student at h_da, has examined the choice of words used in fake news and found that much more emotional language is used online than in scientific or journalistic texts. Personal pronouns are also used more frequently, as are exclamation marks, emojis and headlines in big letters. Jian Xi, who is also a doctoral candidate, has studied the interplay between images and text in fake news.

For Siegel, it is important to involve her students and incorporate research findings into her current teaching. However, she also wants to act as an adviser and raise awareness. In the near future, she will hold training courses and workshops at a police academy: “The police officers and lawyers there need to be on the alert.”