This research is conducted within the framework of the project “Civil Society’s Active Involvement Against the Disinformation”, implemented by Kristina Hristova, project manager of Factcheck.bg, on a grant provided by the Bulgarian-American Fulbright Commission for educational exchange. The research is implemented in the period September-November 2022.
The project’s main goal is to advance a strategy for Factcheck.bg for the development of a community of active citizens, capable of detecting and responding to disinformation. The strategy will be developed in a separate document in Bulgarian and will be presented to organizations interested in supporting its implementation.
In this paper the author presents only the best practices in the field and the conclusions regarding the successful strategies for engaging citizens in support of the fact-checking organisations.
The main project activity is research of the practices of IFCN code of principles signatories across the world. The good practices for citizen involvement were investigated along the lines of the following four topics:
- How are citizens engaged to tip off fact-checkers about dubious information on traditional and social media, and in political statements?
- How could citizens be engaged in the actual process of fact-checking?
- What are the successful strategies for dissemination of the verified content with the support of civil society?
- How could civil society support the sustainability of a fact-checking project in the long term?
According to The Duke Reporters’ Lab annual fact-checking census there were 391 fact-checking projects that were active in 2021 . Of those, 378 are still operating now. But the current research involved only those organisations that are verified signatories under the IFCN Code of principles – currently 94. The research is based on a total of 24 interviews conducted both online and in person. The interviewees were representatives of the International Fact-Checking Network in St Petersburg, Florida, and representatives of pre-selected members of the network. The preselection was made based on their active and recent projects involving citizens in the fact-checking process. Out of the 20 interviewed fact-checking organizations, the author selected 10 projects (one of them is outside the International Fact-Checking Network because it represents a unique example of a citizens’ self-organization). The main criterion for their selection was that their activities are part of a long-term sustainable strategy for engaging citizens..
Context:
Current state of fact-checking in the world
Fact-checking has been traditionally used in newspapers as an internal procedure for verification of all the facts before an article is published. In the last decades fact-checking became mostly known as a weapon against disinformation conducted with the purpose of manipulating society. That is why fact-checking now mostly refers to the verification process of political statements and information in traditional and social media by a third party. The current research is based on this understanding of fact-checking.
According to the Oxford Research Encyclopedia of Communication, organizations specializing in such fact-checking first appeared in the United States in the early 2000s. In Europe the first regular source of political fact-checking appears to have been a blog launched by the United Kingdom’s Channel 4 News in 2005, to cover a parliamentary election. (The Rise of Fact-Checking Sites in Europe – Reuters Institute, 2016). The latest census in 2021 by the Duke Reporters’ Lab identified 341 active fact-checking projects in 102 countries.
The rise of fact-checking in its modern form is related also to the growth and influence of social media platforms in citizens’ daily lives. This allowed various malevolent actors to spread misinformation and propaganda through these new channels of communication.
The rise of social media has transformed the journalistic profession into a race to be the first to break the news. This has also undermined the accuracy and the attention to verified facts.
All these changes in the media world are part of the reason why fact-checking became an external activity, operated by specialized organisations. Still, a lot of big media have their own fact-checking departments (AP, AFP, DPA, Le Monde, CNN). There are also fact-checking platforms created by universities (like South Africa’s Africa Check), but in the last decades fact-checking has predominantly been developed by specialized civil society organizations (PolitiFact.com, Factcheck.org, Factual.ro, Zastone.ba, Factcheck.bg).
Is fact-checking an effective instrument against disinformation?
“The effects of fact-checking on beliefs are quite weak” and even “negligible” in many real-world scenarios. …though fact-checking can be used to strengthen pre-existing convictions, its credentials as a method to correct misinformation (i.e. counter-attitudinal fact-checking) are significantly limited.”
These are the findings of a meta-analysis (based on data from multiple prior studies) by the communications scholars Nathan Walter, Jonathan Cohen, Lance Holbert, and Yasmin Morag (published October 2019).
Fact-checking might even have a backfire effect occasionally among a hard core of strong partisans, according to the study “The Elusive Backfire Effect: Mass Attitudes’ Steadfast Factual Adherence”.
The main challenge for the fact-checkers is that they don’t reach the audience they need to reach, i.e. the people that are already persuaded in conspiracy or propaganda theories. The fact-checkers are outspent by disinformation campaigns as these campaigns use paid trolls and aggregator websites and invest significant amounts of money to reach a large audience. In Bulgaria 65% of the top 25 media that spread Russian propaganda are actually aggregator sites that only copy texts from other places and are not staffed by real people, much less journalists. The remaining 35% publish content that pages bots reprint. („The Russian propaganda in Bulgarian online media“, research by the Human and Social Studies Foundation- Sofia, March-July 2022)
According to the study “The Spread of True and False News Online”, made by three MIT scholars “.. false news stories are 70 percent more likely to be retweeted than true stories are. It also takes true stories about six times as long to reach 1,500 people as it does for false stories to reach the same number of people. When it comes to Twitter’s “cascades,” or unbroken retweet chains, falsehoods reach a cascade depth of 10 about 20 times faster than facts. And falsehoods are retweeted by unique users more broadly than true statements at every depth of cascade.”
Why it is important to involve the citizens in support of the fact- checking organizations
The starting point of my research is that nowadays conspiracy and propaganda are spread not only by paid “trolls” but equally by ordinary citizens who want to be outside the system or because of their political beliefs. The army of propaganda consists of professional “trolls” and propaganda experts supported by convinced ordinary citizens.
We could not expect that fact-checkers alone could fight this army with their investigations. They will need the support of ordinary citizens who understand the danger misinformation puts us in.
We must now reverse the direction and incite society to start fighting disinformation by developing critical thinking and fact-checking as a natural behaviour when being online.
In pursuit of those goals, fact-checking organizations cannot limit themselves to publishing fact-checks and continue treating citizens simply as consumers of information the way traditional media do. Fact-checking organizations should involve, educate, and activate citizens to resist and fight disinformation on a daily basis.
What are the best practices for engaging citizens in support of fact-checking organizations?
In the framework of my research, I identified several approaches fact-checking organizations are taking to get the support of citizens against disinformation. My research covers specifically the members of the International Fact-Checking Network with one exception – the so-called Lithuanian “elves” – and three examples of crowdsourced fact-checking projects.
Citizen engagement is more common for those fact-checking platforms that are managed by non-governmental organizations. In Eastern Europe it is a common phenomenon that those organizations were involved in the democratic reforms in society and have the instinct to look for citizen support and to apply fact-checking as an educational tool for citizens.
‘We see fact-checking as part of the project of the civil reform movement,’ said FactCheck Ukraine project head Igor Korkhovyi for the research “The Rise of Fact-Checking Sites in Europe”. ‘The main idea of our fact-checking is to involve average people into the process of accountability of officials, and monitoring their rhetoric and combating populism.’
In my research I identified 5 main strategies to involve citizens in the fight against disinformation:
- Bring together our expertise and skills.
The idea behind this strategy is to organise those citizens who support the fight against disinformation and put their expertise to use according to the current needs. This strategy counts on the people willing to be useful for society and on their pride of being experts in a given field. Those citizens could be involved in different activities around the main fact-checking process such as the translation of fact checks (FactcheckEU) or their distribution, as in the case of the online campaign organized by FullFact.org for distributing fact checks related to the UK referendum on EU membership. The same strategy is used by the coordinators of the Lithuanian “elves” – ready to use fact checks are sent to the “elves” who then distribute them within their own networks.
There are two very successful examples using this strategy – Science Feedback with its communities of scientific reviewers and Maldita.es with its community of Malditos with superpowers. Science Feedback rely on their reviewers’ desire for public recognition and apply strict rules for participation in their scientific database. Science Feedback has two specialized scientific communities: one on the climate and the other on health-related issues. The approach of Maldita.es is much more oriented to individuals willing to be part of a special community with a social cause. Maldita.es has a database of diverse “superpowers” held by its supporters, such as knowledge of foreign languages, culture or the educationl system. Those malditos are invited to special meetings and parties with the journalists.
- Educate and activate.
This strategy consists mainly in providing citizens with fact-checking and media literacy training. The training sessions are focused on students, civil society activists, teachers, and pupils. Some organisations senior citizens. There is an example of targeting very specific groups such as farm working women in India (Fact Crescendo). The idea behind this strategy is to educate citizens in critical thinking as prevention against propaganda. One result from these training sessions is that the participants can get to know the fact-checking organizations and gain trust in them. Most of those participants continue to cooperate after the training by sending claims for fact-checking and by spreading the fact checks within their networks.
A successful example of such cooperation between citizens and fact-checkers is the training organized by DebunkEU.org for the so-called “elves” – ordinary citizens fighting disinformation and the “trolls” on social media. Another project that is still in its beginning and cannot be evaluated yet is the work of Corrective.org to involve citizens trained in the fact-checking process, providing them with support from automated fact-checking instruments and professional fact-checkers. The project just started in March 2022.
- Support citizens in their battle for democratic reforms.
This strategy consists in active support for civil society’s efforts to make democracy work. Most organizations that implement this strategy apply fact-checking as part of a bigger mission to educate society on how to hold politicians and institutions accountable. This model of fact-checking is typically used by organizations in Eastern Europe like Factual.ro and Zastone.ba but other good examples include FullFact.org in UK, Chequeado.com in Argentina and AfricaCheck.org, which operates in several African countries.
These fact-checking organizations are very active during important political events like elections or protests. During periods of social or political disturbance they have proved themselves to be a reliable source of information (Factual.ro). Specific element of their approach to citizens is to deal with issues important for the local communities. They prefer the bottom-up approach and offer citizens additional instruments enabling them to participate more effectively in the democratic process and interact with local and national institutions.
- Sharing the responsibility for fact-checking operations
This strategy is based on people’s disposition to contribute to society by supporting the causes they consider important. Several organizations rely on citizen funding and offer different options for support. An important part of this strategy is to involve citizens in the process of the fact-checking organizations’ work. This activity has two bonus effects – it educates society on how fact-checking works and makes the process transparent and accountable. Normally, this strategy offers different ways to involve donors – special newsletters with reports on how the organization distributes its budget, what it did in the past month; special meetings with fact-checkers, opportunities to test new instruments, participate in training and events/parties.
Another way to involve citizens in providing financial support to the organization is through crowd funding at specific public events. These could include major public protests (Factual.ro) or approaching elections (PolitiFact). In those cases, the message is: support us and we will give citizens an objective insight into events so they can make correct decisions.
- Involving citizens in crowdsourcing
There have also been several attempts to perform crowdsourced fact-checking relying on citizen engagement. Some of these examples use pre-vetted lists of experts who meet the standard required to support fact-checking on certain topics, as in the case of Science Feedback.
However, crowdsourcing is very vulnerable to manipulation as the users could deliberately feed journalists with false information. Therefore, journalists need to make additional efforts and perform rigorous fact-checking that could result in a time-consuming activity. Two projects have in the past come to this conclusion: TruthSquad and FactcheckEU. Both of them are not functioning any more.
Currently Twitter is trying out a new model of crowdsourced fact-checking involving an algorithm that rates the notes attached to tweets by people who tend to have different political opinions. Their notes are visible only after they are rated as valuable by other Twitter users involved in the pilot phase. This project is still to be evaluated, as Twitter started to show the notes to the US public only in October 2022.
So far crowdsourced fact-checking has shown positive results mainly in developing media literacy skills in participating citizens.
Fabrice Florin, executive director, and founder of NewsTrust which runs TruthSquad, said the project discovered that “amateurs learned valuable fact-checking skills by interacting with professionals”, and later said that a primary focus of crowdsourced fact-checking was to teach skills rather than produce usable output.
Best practices around the world.
Fact-checking organizations that involve citizens in their work
Maldita.es
https://maldita.es
Representative of the organization: Beatriz Lara, engagement and community editor
Maldita.es is a nonprofit fact-checking platform based in Madrid. Maldita Hemeroteca was started as a Twitter account in 2014, later followed by Maldito Bulo (2017) and Maldita Ciencia (2018). Maldita.es launched a crowdfunding campaign in 2018 to become a foundation. In 2019 Maldita started a project to unite the superpowers of its supporters – whom they call ‘malditos’ – in a searchable database. Using funding from the Engaged Journalism Accelerator, Maldita’s team built a CRM to organize its community members’ expertise.
The project started as a survey in 2018 to gather data on what users thought about Maldita.es, how they found out about it, which projects they knew about and the skills they were prepared to contribute to the organization. The team called these skills “superpowers”.
The CRM allows the team to find and reach qualified individuals who can help them in fact-checking through specific knowledge or skills. According to Beatriz Lara, the community editor of Maldita.es, the community now has Malditos such as scientists, lawyers, teachers, historians, experts in religions and specific cultures like for example Japanese. All specific skills that can contribute to the fact-checking process are stored and easily searched through the CRM.
The community editor of Maldita.es conducts a background check on all experts and reaches out to them to certify who they say they are. She is also asking them for documentation to prove that they really have the skills they subscribed with in the database.
According to Beatriz Lara now the database comprises 2800 people with “superpowers”. Recently Maditas.es launched a campaign to engage more women with superpowers, as the percentage of women in the database is 30% compared to70% for the men.
“Malditos” are fully engaged in the fact-checking process. They report misinformation to the Maldita.es team to fact-check. The team then sends the results back to the community in formats that can be easily spread and forwarded on mobile devices. Malditos share and distribute the fact checks through their personal channels. The aim is to make fact-checked information as available and more viral and popular than misinformation.
Maldita.es organizes regular fact-checking training for its community members and journalists.
Sciencefeedback.co
https://sciencefeedback.co
Representative of the organisation: Katie Valentine, Science editor, Climate
Science Feedback is a not-for-profit organization registered in France. The organisation verifies the credibility of influential claims and media coverage presented ase scientific and provides feedback to editors and journalists about the credibility of information published by their outlets.
Science Feedback is one of the best examples of how citizens can be engaged in fighting disinformation. The organization gathers scientists who can respond to influential information by verifying facts and science theories.
In addition to its core team the organization works with voluntary scientific reviewers who contribute to the fact-checking process. Science Feedback uses the principle of crowdsourcing scientific expertise. The scientists evaluate articles and provide a credibility score based on accuracy, logic, objectivity, and factual precision.
The organization’s work principles:
“Science editors identify trending news and claims according to their influence on social media and the quantity or degree of scientific evidence within the reporting. Once an article or claim is selected for review, one editor leads the fact-checking effort by contacting relevant experts, reading and summarizing the scientific literature on the topic, and writing a draft of the review. At least one other editor reviews the draft, challenges the claims, and pinpoints where additional clarity and references are needed. Challenging questions are decided by the director.” (https://sciencefeedback.co/team-advisors-contributors/)
Currently the organisation has two communities of reviewers: Climatefeedback and Healthfeeback.
https://climatefeedback.org/community/
https://healthfeedback.org/community/
How the reviewers are selected
The application process requires the contributors to have credentials in relevant scientific areas and to adhere to high-level ethical standards in their scientific activities and public positions.
According to Katie Valentine, one of the Science Feedback editors, the organisation does not accept scientists as reviewers if they work for a lobbyist group or organisation as this could distort their objectivity. The candidates should also have publications in highly rated scientific journals.
How they work : “When analyzing a media article, reviewers must practise scientific inquiry by:
- ensuring that facts are correct (fact-checking)
- drawing attention to all relevant evidence (providing context, giving due weight, not cherry picking)
- using robust logical reasoning
- limiting authoritative comments to their area of expertise; when commenting outside their direct field of expertise, contributors must back their claims using strongly supported scientific theories and observations and cite reliable sources.
Currently there are 400 reviewers in Climate Feedback and 500 in Health Feedback communities from around the world. They are working voluntarily, and each article has one or two reviewers at least.
Identified strengths for a successful strategy to engage citizens in support of fact-checking:
Opportunity to apply one’s expertise for a social cause: The huge number of contributors working voluntarily for the organization marks a success story in the field of fact-checking. Most of the scientists are there because of their personnal belief they should fight disinformation with the expertise they provide.
Opportunity to promote one’s expertise: Another reason for their voluntary work is probably also the opportunity to be quoted is a publicationby a highly respected media, according to Katie Valentine. Most scientists want to discuss their research with the media so this is an opportunity for them to speak about it publicly.
Created by one of us: The founder of the organization is also a scientist so this could be identified as anothery key to the initiatives’ success, i.e. the trust in the community.
What next:
Currently, Climate Feedback are starting a project that offers access to their database of scientists to other IFCN signatories. For more information:
https://climatefeedback.org/science-desk/
Factual.ro
https://www.factual.ro/
Representative of the organization: Elena Calistru, “Funky citizens” founder
Factual.ro is the first political fact-checking site in Romania. Factual was formally launched in May 2014 by an enthusiastic group of NGO people, corporate workers, entrepreneurs and political consultants interested in good governance. Since August 2014, the project has been implemented by “Funky Citizens”.
The project relied only on citizens’ support and crowdfunding until 2021 when it received its first grant from the Swiss government through the Civic Innovation Fund. Factual.ro won the citizens support and trust during the protests in 2017, when it provided factual and objective analyses of the impact of the legislative changes that brought people out on the streets. In 2017 it received 24,000 euro via crowdfunding in only two days.
According to Elena Calistru, the founder of “Funky citizens” this support was due to the timely and correct information they offered to the public on trending topics. Factual.ro also performs adequate and reliable fact-checking during electionl debates. In principle, the financial support by ordinary citizens increases during specific public events when organizationsrespond quickly and provide the needed fact-checked information.
The platform currently reaches around 7,5 million people and 44.000 followers on Facebook.
Factual.ro relies on a large network of supporters and partnerships. It organizes fact-checking training for various civil society organizations, such as volunteers at environmental or local-level organizations. The platform also offers fact-checking training to teenagers.
An important part of the work to engagecitizens is the training and partnerships with local journalists and the development of fact-checking on a local level. This activity involves local citizens through focusing on information they feel closer to and recognize as relevant to their daily lives.
An keyit part of the strategy for reaching a young audience is the partnership with debating or youth associations. Factual.ro teaches young citizens how to use fact-checking in their arguments and debate on the basis of fact-checked information.
Factual.ro also involves scientists in its activities on the project Countering misinformation surrounding the transition to a green economy, which focuses on disinformation related to climate change. The fact-checking platform works in collaboration with two organizations in the field of climate change – one NGO active in local communities (Reper21) and the academics and experts from Infoclima.ro. Currently, the platform is inviting more non-governmental organizations and think thanks to join the pre-bunking and debunking with their expertise related to climate change.
Factual.ro’s outreach strategy to reach people naturally involves all social networks popular in Romania, like Facebook and Instagram. They present fact checks through videos and graphics which are more suitable for social media users. The organization also offers general fact-checking education by providing short videos with explanations on how to do basic fact checks.
Fact Crescendo
https://factcrescendo.com
Representative of the organization: Mayur Deokar, Fact-checker
Fact Crescendo is an independent digital journalism initiative and is a part of Crescendo Transcription Private Limited (CTPL), a private limited company registered in India. (https://www.crescendotranscriptions.com/). Fact Crescendo works in 15 languages in six countries. Fact Crescendo started its operations in Sri Lanka in 2019 and now covers Myanmar, Afghanistan, Cambodia and Bangladesh.
The organization is very active on social media networks and presents fact checks in varied formats like video, image, story/status, and WhatsApp broadcasts. Its WhatsApp FactLine, Facebook groups and social media accounts receive a large number of fact-checking requests every day.
Fact Crescendo implemented the media literacy initiative FacTree in an effort to reach the population in rural areas. FacTree primarily focuses on women (housewives and farm workers) in rural and semi-urban areas. Through this program, Fact Crescendo educated people how to verify online information, how to be safe online, and how to report misinformation.
The organization is planning to run this project in three villages in each of the following states: Maharashtra, Gujrat, Uttar Pradesh, Odia, Assam, Kerala, Tamil Nadu, and West Bengal throughout the next year.
Why is the project is focus on women in rural areas?
According to Mayur Deokar, around 70% of the population of India lives in rural areas. India has 336.6 million rural internet connections, 20% more internet users than in urban areas. 81% of users aged 50 and above access the internet daily.
The population in rural parts has lower literacy and a lower media literacy rate as compared to urban areas. That is why Fact Crescendo provide training in the regional language and even set up a WhatsApp Tipline number. By encouraging critical thinking, the team hope to bring some behaviour changes to the mindset of the participants.
The fact-checking group explained for the purposes of this research that female active internet users have grown by a whopping 61% since 2019 in India (male users grew by 24%). One in every three female internet users in rural areas is actively using the internet. A UN Women study on the Empowerment of women notes that women were attacked with false information on social media more often than men. Especially the 18-24 age group. Hence, it is necessary to address the challenges faced by women. A women-centric approach to media literacy is the need of the hour and Fact Crescendo is doing just that.
The team usually go to villages and gather up women with the help of the heads of the village. Their objective is to spread awareness among as many women as possible.
The women’s initiative’s objective is to teach them how to use the internet safely, how to report or block unwanted messages or people, and why not to believe WhatsApp forwards. The project focuses on training in social media best practices. The trainers also encourage the women to send suspicious content on WhatsApp Tipline.
CORRECTIV
https://correctiv.org
Representative of CORRECTIV.Faktenforum : Caroline Lindekamp, Head of project
CORRECTIV is the first non-profit investigative newsroom in Germany (When it was created? ). (https://correctiv.org/en/about-us/) The organisation is implementing a series of projects aimed at educating and its readers in the fight against misinformation through media literacy and journalism workshops.
In March 2022 CORRECTIV started the project CORRECTIV.Faktenforum. Behind CORRECTIV.Faktenforum is an interdisciplinary team of scientists, programmers, journalists and media lawyers. The project is implemented in partnership with the Ruhr University Bochum (the Cognitive Signal Processing working group is led by Prof. Dr. Dorothea Kolossa and the working group Digital Forensic Linguistics is led by Prof. Dr. Tatjana Scheffler).
From the Dortmund Institute for Journalism, a team is led by Prof. Dr. Tobias Gostomzyk with expertise in media and internet law.
Fact checks by citizens
In cooperation with CORRECTIV, the scientists are developing AI-supported assistance systems that help, for example, to identify potentially incorrect information, to categorize it, and to analyze text and images. Seminars with professional fact checkers from CORRECTIV strengthen the media skills of the laypeople.
“CORRECTIV.Faktenforum is intended to be a platform for citizens who want to take action against disinformation on the internet. In the forum, they should be able to discuss, check and refute the false news that is currently circulating.” (https://correctiv.org/projekte/faktenforum/)
What are the principles of the forum: “Check facts as a community”
The citizens that want to join the platform are not required to have experience in fact-checking, journalism or any specific technical experience. The participants will receive basic fact-checking knowledge and they will be supported by automated research tools in fact-checking. The citizens will work in cooperation with experienced fact-checkers.
The project is still in its afterbirth phase and could not present results of its work. Two staff members are currently doing community building and management. The project looks for participants through its partnership network – organizations that stick to similar values but have diverse expertise and interests to contribute according to the topic. They will learn how to stick to journalistic standards when they work on topics that are close to their personal beliefs, to be objective and not to follow their organization’s agenda.
Currently the project is looking into people’s opinion through an online survey – what they expect from the forum and how they are willing to support it.
Dorothea Kolossa, Ruhr University Bochum: “In this project, we want to enable fact-checking by citizens using machine learning methods and artificial intelligence. We plan to support the crowdworkers on our platform with automatic analyses and quick access to relevant information and to encourage their collaboration. However, we always leave the decision on fact or disinformation with the human user. It is particularly important that our AI methods are transparent and that their decisions are easy to understand.” (https://correctiv.org/in-eigener-sache/2022/03/08/mit-menschlicher-und-kuenstlicher-intelligenz-gegen-desinformation/)
Reporterfabrik und Bürgerakademie (Reporter factory and Citizens’ Academy)
Correctiv has already a long-term experience in developing citizen journalism. It runs its own educational program with an online academy consisting of the Reporterfabrik (reporter factory) and the Bürgerakademie (citizens’ academy). The training is both live and online, consisting of more than 100 scheduled workshops and 1100 tutorials in the form of learning videos and exercises. The organization works actively with school children and their teachers.
Zasto ne
https://zastone.ba
Representative of the organization: Amra Hasanović, project coordinator
“Zasto ne” is one of the best examples of how to involve citizens in sustainable support for fact-checking activities and create a critical thinking environment.
Citizens’ Association (CA) “Zašto ne (Why not)” is an organization that operates in Bosnia and Herzegovina. The organization combines fact-checking (https://www.istinomjer.ba, (https://www.raskrinkavanje.ba) with projects for citizen participation (https://www.javnarasprava.ba) facilitating public debate by citizens and parliamentarians on laws being currently proposed; https://www.glasometar.ba – an internet application created to provide support to potential voters in their election-related decisions.
It also runs several projects for monitoring government and political processes, research and advocacy; promoting the use of technology; civic education for elections and projects in support of the NGO sector in BiH.
These diverse but at the same time interrelated activities help the organization spread the facts in the society and at the same time educate it to be critical.
“Zasto.ne” focuses a lot of efforts in developing media literacy through webinars and workshops for students and journalists. The training always involves examples from fact-checking on the platform. The team noticed that after such training the participants started to monitor and send information to be checked. The result was increased capability of the platform to cover more information.
The organization has a strong relationship with the academic community, involving it in different investigations and projects. At the same time, it also involves students in its core activities through internships and partnership with student councils.
The organization is also constantly developing its network of journalists by holding frequent training fact-checking sessions for journalists from independent media.
The Demagog Association
https://demagog.org.pl
Representative of the organization: Piotr Litwin, analyst
The Demagog Association is the first fact-checking organization created in Poland in 2014. The organization is an example of a multilateral approach to engage its audience in support of fact-checking activities. This is also an example of how media literacy training is working in support of fact-checking organizations.
Demagog newsletter has two side purposes: to give an update onr ongoing fact checks and to educate, i.e. raise awareneww on how fact-checking works. It also outlines how the money from the crowdfunding was spent. One of the success stories for Demagog is that on several occasions the citizens themselves organised crowdfunding campaigns in support of the organization.
Demagog conducts workshops and implements educational projects aimed at young people, such as the Fact-Checking Academy. It also organizes media literacy workshops in centers for senior citizens. According to Piotr Litwin, the seniors are really willing to learn and get involved in the training. Most of the trained participants start to send claims to be fact-checked after being involved in the organization’s activities.
The organization runs educational platforms and conducts educational campaigns in fact-checking and media literacy.
Demagog tries to involve the scientific community in its work through promotional videos that show scientific fact checks presented by the scientists themselves.
Chequeado
https://chequeado.com
Representatives of the organization: Pablo M. Fernández, Executive director; Olivia Sohr, Director of Impact and New Initiatives
Chequeado, Argentina, has been online since October 2010 and is the first site in Latin America dedicated to fact-checking. Chequeado is the main project of La Voz Pública, a non-partisan and non-profit foundation based in the City of Buenos Aires.
The organization has several ways of engaging citizens with its activities and the most common are the suggestions for fact-checking and volunteering in event organization.
“We believe we need to build different “rings” around Chequeado of engaged citizens, according to how much time or energy they can dedicate. For the wider group we give more general and easy tasks, and for those who want to be more engaged, we offer more complex and intensive ways to contribute. We do believe that it is very important to have this engagement with our community (donors and non-donors) so that we have the opportunity to share with them our work and to have their support”, said Olivia Sohr for this research.
Chequado has the so called “Checkadist program”. Checkadists financially support the organization and they have special privileges, such as meetings and discussions with the journalists’ team; beta testing of new products; access to special training in checking and verification.
Chequado organizes special events for the “checkadists” where new initiatives are presented. The team discusse with their supporters the organisation’s current work and ask their opinion about future initiatives. They are also invited to try new tools like the Healthy Internet Project in which they are trying out a navigator extension.
Chequado has a Media program for education, which seeks to develop the critical capacity of young people and train journalists in the use of data and evidence. El Checkador is an online educational platform created and designed by Chequeado’s Education and Innovation teams to teach fact-checking and data journalism to everyone who wants to learn.
Another interesting project is the Collective Check: It is an open platform to verify content at the request of the community.
Africa Check
https://africacheck.org
Representative of the organization: Laura Kapelari, Head of digital communication
Africa Check is the first independent fact-checking organization in Africa set up in 2012 to promote accuracy in public debate and the media in Africa.
Africa Check involves 4 African countries. It was established in South Africa in 2012. Africa Check’s francophone office was established in Senegal in 2015. The offices in Nigeria, set up in 2016, and Kenya, opened in 2017.
Africa Check is one of the best examples of a multilevel and comprehensive strategy to educate a new generation of critical thinking citizens. Although the fact-checking was at the beginning of the project, now it seems just a part of a bigger program.
Africa Check has extensive and regular media literacy and fact-checking training programs. They involve different formats and levels of complexity.
There are ready to use tips for fact-checking and spotting disinformation presented in short texts with graphics, so everybody can share them online. The organization hosts a colossal collection of videos for online training focused both on the ordinary citizens and more complex for the journalists.
Africa Check also organizes a lot of life training. According to the information on its website, 4,500 journalists were trained on verification best practices.
The organization has developed a multilevel approach to its partnership strategy (https://africacheck.org/get-involved). It offers the opportunity for the citizens to support the organization through donations, spreading the fact checks and reporting claims. On a higher level, Africa check also seeks long term financial, expert and communication support from institutional partners.
One of the good examples Africa Check could give to the fact-checking world is the involvement of the local radio in spreading the fact checks. The approach is highly creative and involves Interactive Radio Drama, media literacy content, discussions with experts and interviews. The idea behind that is that fact checks are often written in the official languages and in long format, which leaves millions of people in Africa outside the reach of the fact-checked information because of the lack of internet access and the variety of local languages.
The project focused on the topic of vaccination against COVID.
“To curb dangerous misinformation on vaccines and Covid-19 , we launched an innovative participatory radio initiative in April 2021. The initiative is in collaboration with Theatre for a Change and is supported by the Google News Initiative. It is the first of its kind and uses Interactive Radio Drama, media literacy and experts to change attitudes and behavior in Nigeria and Senegal”, explained Laura Kapelari for this research.
The shows aired weekly on Radio One in Nigeria and Radio Oxyjeunes in Senegal. The shows are titled “On Top Di Matter” in Nigeria and “Diisoo Ngir Aaru” in Senegal. They are in two widely spoken local languages, Pidgin in Nigeria and Wolof in Senegal. By using local languages Africa Check could reach audiences who do not necessarily speak English or French, in a deeper and more meaningful way.
The programmes used a participatory approach, expert interviews and media literacy content to empower people against health misinformation. Each one-hour magazine-style programme started with a pre-recorded radio drama that highlights Covid-19 vaccine misinformation. Listeners were then given a chance to call in and replace a character in the drama to show what they would do differently. Many called in weekly to engage with the cast and say how they would fight misinformation. By playing a role in the drama, listeners can practise fighting misinformation and educate each other. (https://africacheck.org/fact-checks/medialiteracy/harnessing-power-radio-africa-fight-covid-19-misinformation)
Another project oriented to reach the community is the “Fact Ambassadors’’ project . The Fact Ambassadors are community leaders who are involved in spreading content to their community. The content was shared on social media platforms in local languages. These include English and Zulu in South Africa, Kiswahili in Kenya, Wolof and French in Senegal, and Pidgin and Hausa in Nigeria. The participants were trusted leaders in their community.
To recruit Fact Ambassadors, Africa Check put out an open call for people who stand up for accuracy. Key to the nomination process was a passion for the importance of accurate information. After a careful selection process, 100 Fact Ambassadors from Kenya, Nigeria and South Africa have joined the programme. The Fact Ambassadors are from a diverse set of backgrounds: public health, journalism, engineering, statistics and development.
They are involved on a voluntary basis and did not receive payment for spreading the fact-checked information. The participants in the program received basic training in fact-checking and social media.
Africa Check teamed up with comedians, satirists and content creators to tackle misinformation and educate the general public about its dangers as well as what they can do to stop its spread under the hashtag #StandUp4Facts.
Full Fact
https://fullfact.org
The information about Full Fact is gathered both trough an online interview with Ross Haig, Head of communication and from its website.
Full Fact, was founded in 2010, is a registered charity with a board of trustees that includes journalists as well as members of the country’s major political parties
The organization sees its mission beyond publishing fact-checked information online. It actively seeks corrections from the politicians and the media. Full Fact is one of the organisations that identified themselves as second generation fact-checkers (together with Africa Check and Chequeado). They present themselves as organisations seeking change through fact-checking.
In the publication “Fact checking doesn’t work (the way you think it does)” – a kind of joint statement made by the three organisations – they identified the second-generation fact-checker as a power for social change by actively seeking corrections and pressing people not to make the same mistake, by providing society with understanding where the misleading claims come from.
Full Fact says it works for system change. “Using the evidence from the fact checks we identify patterns and common causes, points where we can intervene to significantly reduce particular kinds or sources of information. The pattern might be who’s publishing something, where it’s published, a particular subject that there’s a lot of false information about, or something else. The interventions can range from educating children or adults to advocating for policy changes.”
Full Fact involves citizens in its activities by inciting them to suggest fact checks and to train the fact-checking AI.
During the 2015 election Full Fact had 120 volunteers helping the media monitoring operation. The volunteers looked through the entire media output every day and extracted the claims being made. Then the journalists proceeded with the professional fact-checking.
In the UK EU membership referendum in 2016 thousands of members of 38 Degrees (38 Degrees is an online campaigning organisation involving more than 2 million people) took part in a fact distribution network. They would take the results of fact-checks and repost them on forums, social media groups, share them with friends and family. This gave Full Fact coverage into communities they wouldn’t have otherwise had.
When citizens create their own network for fact checking
The following information is gathered through an interview with one of the “elves” (elves are anonymous), previous articles on the topic and information from the page “About the elves” at https://www.debunkeu.org/about-elves.
Lithuanian elves are a unique phenomenon representing the self-organizing power of ordinary citizens in the face of propaganda. Elves are volunteers organized in coordinated activities against Russian trolls and disinformation.
The movement was born in Lithuania in 2014, after the Maidan Uprising in Ukraine, when clashes broke out between protesters and the pro-government forces backed by the pro-Russian Ukrainian President Viktor Yanukovych. Shortly afterwards, Russia invaded Ukraine and occupied parts of the country. The conflict also spread to the online space, as the events in Ukraine were followed by disinformation attacks and a constant flood of propaganda.
This provoked a group of friends to start gathering volunteers to organize the “defence” against troll attacks. The elf army was composed of about 40 individuals who were confronting toxic pro-Kremlin trolls on online media, particularly in the comments sections, and on social networks. Their first victory was to turn the score in the comments in favour of the democratic and pro-European information.
Now the army of the elves consist of thousands of professionals in different fields: foreign politics, cybersecurity, IT, economics, environmental protection and other. They already have branches in other countries like Estonia, Latvia, Czechia, Finland, Slovakia, and Ukraine.
In the beginning the elves relied solely on their own skills and competences. Now they are working with fact-checking organizations like DebunkEU.org.
Not only the elves provide professional fact-checkers with misinformation to be debunked, they also contribute with their specific knowledge in different fields.
The group also changed their tactic, switching from defence to attack. In 2017 a group of elves started a campaign against the Russian TV Sputnik. They visited Sputnik’s international Facebook page, spread memes and rated their page. The campaign was supported by Ukrainian elves who joined forces to downgrade the rating of the page. In six days Sputnik’s Facebook page rating went from 4.3 to 2.6 and on the last day of the month they closed the page rating system.
One of the main activities of the movement now is to hunt after companies that in any way support the Russian aggression in Ukraine and the promotion of the Soviet Union. For example, they organized a very successful online campaign forcing Adidas to retire clothes with Soviet Union symbols. (#stopadidas: https://www.boredpanda.com/adidas-ussr-themed-sports-collection/)
Now the fact-checking platform Debunkeu.org is trying to support the process of engaging more citizens in the elf army by organizing training events and workshops. Those events provide elves with all the skills and tools necessary for fighting disinformation.
The wisdom of crowds or crowdsourced fact-checking
The following information is collected from different online sources related to the specific projects.
There were several attempts to use the wisdom of crowds in the fight against disinformation. According to a research study made by the Massachusetts Institute of Technology, crowdsourced accuracy judgements from groups of normal readers can be as effective as the work of professional fact-checkers.
The study “Scaling up Fact-Checking Using the Wisdom of Crowds” was published in September 2021.
The researchers used 207 news articles that an internal Facebook algorithm identified as needing fact-checking, either because there was reason to believe they were problematic or simply because they were being widely shared or were about important topics like health. The experiment deployed 1,128 U.S. residents using Amazon’s Mechanical Turk platform.
At the same time, three professional fact-checkers were given all 207 stories and asked to evaluate them after researching them. The average ratings of small, politically balanced crowds of laypeople correlated with the average fact-checker ratings as well as the fact-checkers’ ratings correlated with each other.
Birdwatch at Twitter
Twitter is experimenting with this approach in its Birdwatch project where pilot contributors can write notes on any tweet. If enough other contributors rate that note as helpful, highly ranked Birdwatch notes may be publicly shown on a tweet. (https://help.twitter.com/en/using-twitter/birdwatch). “Birdwatch notes do not represent Twitter’s viewpoint and cannot be edited or modified by its teams. A tweet with a Birdwatch note will not be labelled, removed, or addressed by Twitter unless it is found to be violating Twitter rules, terms of service, or our Privacy Policy. Failure to abide by the rules can result in one’s removal by the Birdwatch pilot, and/or other remediations.” (https://help.twitter.com/en/using-twitter/birdwatch).
Contributors are lay people, who sign up to write and rate notes about potentially misleading tweets.
Since October 2022 Twitter users in the US can see some Tweets accompanied by a note containing relevant information that’s been rated “Helpful” by Birdwatch contributors. Most notes contain additional sources that can be clicked for an even deeper dive into a subject or conversation.
In order to be shown on a tweet, Birdwatch notes need to be found helpful by people who have tended to disagree in their past ratings. This means the algorithm takes into account not only how many contributors rated a note as helpful or not helpful, but also whether people who rated it seem to come from different perspectives, according to the Birdwatch guide.
The Birdwatch algorithm is publicly available on GitHub, along with all the data that powers it, so anyone can audit, analyze, or suggest improvements.
According to the results of four surveys run by Twitter at different times between August 2021 and August 2022, a person who sees a Birdwatch note is, on average, 20-40% less likely to agree with the substance of a potentially misleading tweet than someone who sees the tweet alone. Survey participation ranged from 3,000 to more than 19,000 participants and the results were consistent throughout the course of the year, even as news and tweet topics changed.
TruthSquad and FactcheckEU
TruthSquad, a pilot to crowdsource fact-checking run in partnership with Factcheck.org, started in 2010. The primary goal of the project was to promote news literacy and public engagement, rather than to devise a scalable approach to fact-checking, according to an article by Mevan Babakar at Full Fact .
Readers could rate claims and then with the aid of a moderating journalist fact-checks were produced. According to the project lead Fabrice Florin: “Our editorial team did much of the hard work to research factual evidence. (Two thirds of story reviews and most links were posted by our staff.) Each quote represented up to two days of work by our editors, from start to finish. So this project turned out to be more labor-intensive than we thought[…]””
FactcheckEU was run as a fact-checking endeavour across Europe by Alexios Mantzarlis, previous director of the International Fact-Checking Network and currently the manager of the product policy team for Google Search. Alexios Mantzarlis, while commenting on the project results, concluded that trying to get a lot of people involved and at the same time keeping the quality of the fact checks high required a lot of effort by the professional fact-checkers. However, he also stated that “most of the success we got harnessing the crowd was actually in the translations. […] People were very happy to translate fact checks into other languages.”
For more information related to this research, please contact Kristina Hristova at [email protected].
