Keeping Misinformation, Hate, and Violence from Going Viral

Omidyar Network commits $10 million toward making private messaging more trustworthy

Private messaging platforms like Messenger, Signal, Telegram, WeChat, and WhatsApp have, in the last decade, grown at a staggering rate to become nearly ubiquitous components of our digital existence. Two out of five people use the platforms to collectively send hundreds of billions of private messages each day.

By assuring private communication on a global scale, these innovations can play a critical role in empowering ideas and social movements by facilitating trust and solidarity among users. The platforms’ current design choices, however, also contribute to complex problems across society. Specifically, the features that encourage scale, virality, and monetization have transformed private messaging platforms into conduits for the large-scale spread of dangerous and distorted content.

Today, Omidyar Network is releasing a new vision and strategy to help the platforms, their users, governments, and nonprofit advocates make private messaging safer by curbing the flow of misinformation, hate, violence, and extremism.

In “Beyond Encryption: Our Vision for Trustworthy Messaging in a Viral World,” we call on a range of stakeholders to pursue industry-wide changes in product design, policy, and business decision-making to fundamentally reduce the ways private messaging platforms can be exploited without compromising users’ right to private communication.

“Problems online hardly ever start there — and they rarely end there,” said Wafa Ben-Hassine, a human rights lawyer and principal at Omidyar Network. “It’s our imperative to make sure private messaging platforms stop enabling their spread. We believe the most effective solutions lie in confronting the suboptimal design choices of platforms themselves, rather than trying to censor problematic content and objectionable behavior, which can quickly result in many questions and concerns about users’ rights to privacy and freedom of speech.”

Under the current paradigm, platform companies are incentivized to target and acquire massive user bases, engineer for convenience and minimal friction, and maximize engagement and virality. Some go as far as harvesting reams of data about users that can then be monetized. They know, however, that their group sizes and dynamics, scale, and convenience and speed also enable the spread of mis- and disinformation. In fact, modern propaganda campaigns are specifically engineered and optimized for this type of technology.

In our latest point-of-view on responsible technology, we layout two priorities for private messaging platforms:

  1. Uphold encryption to empower users, communities, and movements
  2. Establish new incentives that promote safer design choices, transparency, and accountability. In our report, we highlight several specific design choices that would significantly inhibit virality while also building trust and credibility.

We also articulate three, urgent actions needed to build trust between technologists, users, governments, nonprofits, researchers, and shareholders:

  1. Establish disclosure and transparency commitments
  2. Create a distributed governance and accountability system
  3. Encourage collaborative research and design

Omidyar Network has committed $10 million toward this multi-year agenda and has to date awarded nearly $3 million in grants or contracts to individuals and organizations working to make private messaging platforms more trustworthy (and as a result, safer). Early to notice the risks, our partners have been conducting research and educating tech companies and governments on how to preserve privacy protections while making the necessary product and policy changes to stop the viral spread of conspiracy theories, destabilizing hoaxes, false claims, hate speech, and organized disinformation campaigns.

One of our longest-running partnerships related to this work is with Meedan, a technology nonprofit that builds software and initiatives to strengthen global journalism, digital literacy, and accessibility of information for the world. Today, we are announcing a new grant of $500,000 to support the scaling of Meedan’s Crypto Chat project and Digital Health Lab misinformation response, both of which we supported in earlier stages. At the start of the COVID-19 pandemic, Meedan brought together a team of scientists to form Health Desk, a team tasked with supporting fact-checking organizations and journalists combatting health misinformation about COVID-19. More than 1.5 million people visited health-desk.org in 2021.

We’re also celebrating our newest partnership with the Institute for Strategic Dialogue UK (ISD). With a new grant of $537,360 ISD will conduct research on closed and encrypted messaging platforms that will tell stakeholders more about how hateful and extremist groups behave online. Their investigations and educational efforts will help develop practical, technical solutions as well as inform government policy and accountability for the technology companies whose designs are currently enabling harmful and illegal activities.

These two dynamic organizations join other leaders — Africa Check, Atlantic Council (Digital Forensic Research Lab), Carnegie Endowment for International Peace (Encryption Working Group), National Conference on Citizenship (Junkipedia), Shorenstein Center at the Harvard Kennedy School, Stanford Internet Observatory, The Fletcher School at Tufts University, University of North Carolina at Chapel Hill, University of Texas at Austin (Center for Media Engagement) — in combatting mis- and disinformation in private spaces online in partnership with Omidyar Network. Going forward, our goal is to work with more stakeholders who are helping to ensure any technological fixes, policies, governance structures, and learning networks dedicated to private spaces online remain flexible and dynamic to respond to the evolving technology and an ever-changing legal landscape.

Our drive toward private and trustworthy messaging is reinforced by our broader $60 million commitment toward building a responsible technology system and demanding stronger accountability and performance from companies. Our programs support activities related to comprehensive data protection and privacy laws; antitrust action and pro-competition regulation; corporate governance overhaul; alternative business models; new data paradigms; and other essential facets of a better tech system. It also means helping to curb pressures from equity capital markets that reinforce short-termism and benefit speculative actors at the expense of society and reorienting the rules that govern markets to better balance the interest of all stakeholders.

To learn more, read “Beyond Encryption: Our Vision for Trustworthy Messaging in a Viral World” and watch “Information Disorder,” a news program hosted by Washington Post Live and sponsored by Omidyar Network. Stream it live on January 11 at 11 a.m. ET/ 4 p.m. GMT or catch the recording at wapo.st/informationdisorder.