February 17 2022 GM

Glitter Meetups What are Glitter Meetups Notes from Past Glitter Meetups Virtual Events Calendar Join the IFF Mattermost

Glitter Meetup is the weekly town hall of the Internet Freedom community at the IFF Square on the IFF Mattermost, at 9am EST / 2pm UTC. Do you need an invite? Learn how to get one here.

The Facebook Papers: thoughts from technologists and activists

A large set of internal Facebook documents were leaked to Congress and the press over the last year. These documents describe internal decision-making processes and warnings about troubling activity on one of the most widely-used communications platforms in the world. We'll talk about what is in these documents, what they mean for people on and off of Facebook, and consider useful ways that we can respond as a global community to these revelations. These documents confirmed many of the issues raised by activists and the rest of civil society over the last decade, in particular impacted communities and people outside of the US and EU.

Dia Kayyali (they/them) is Associate Director for Advocacy at Mnemonic, the umbrella organization for Syrian Archive, Yemeni Archive, and Sudanese Archive. In their role, Dia focuses on the real-life impact of policy decisions made by lawmakers and technology companies about content moderation and related topics. Previously, Dia worked at WITNESS and the Electronic Frontier Foundation. Dia served as co-chair for the Advisory Network to the Christchurch Call for two years and is on the Advisory Board for OnlineCensorship.org.

dkg (he/they) is part of the small group of independent monitors aiding journalists and media sites establish guidelines for an accountable review of the Facebook Papers prior to publication. He is a technologist working at the intersection of technology, law, and civil rights and civil liberties. A bike-riding New Yorker, dkg develops free software, tries to understand and improve Internet protocols, and collaborates with activists, lawyers, nerds, and other advocates for justice to make our digital infrastructure more compatible with the rights of people and communities to thrive and flourish under their own control. dkg is one of the folks in the small group of independent monitors aiding journalists who are going to publish the papers.

• Date: Thursday, February 17th

• Time: 9am EST / 2pm UTC

• Who: Dia and dkg

• Where: As a guest of the Glitter Meetup on IFF Mattermost Square Channel.

👤 Don't have an account to the IFF Mattermost? you can request one following the directions here.

Notes

For those who are not familiar with the FB papers, can you tell us more?

• FB papers are a dump of internal facebook documents by an internal whistleblower (Frances Haugen) to a consortium of news organizations (and to US congress).
• The majority of the papers have not been published publicly.
• News stories have tended to focus on US and European impacts.
• Major themes:
  • Failures in content moderation (both directions).
  • FB as a channel for coordination of serious harms.
  • FB has global impact with minimal international engagement (one of the huge things we saw was FB staff internally pleading for more resources for conflict areas and some regions)
  • FB internal understanding differs from their public comments.
• Each document is basically a collection of pictures that Haugen snapped with her phone of internal FB documents.

Can you tell us how you both first got involved with the review of the dumped documents?

• It was really frustrating that it was "western" news orgs getting access to the documents. After seeing a story about Christchurch, Dia reached out to a news outlet and asked for access.
• dkg got involved when a researcher working with Gizmodo reached out to him to ask him to help them establish redaction policies so that they can publish as many of the documents as possible.

Why was it such a shock/ major news story? A lot of the stuff that came out (like about content moderation for example), wasn't completely new information

• 100% agree. The problem is that when people with lived experience say it, they are ignored. When the Wall street journal published an article, then it became important.
• It's also evidence that Facebook knows that this stuff is going on on their platform. Their public messaging has tended toward being oblivious, but this was the receipts -- showing that they were more aware than they let on.

Which findings have surprised you most in the papers so far?

• Very little has been surprising. ONLY surprised at how many facebook staff have made the same arguments internally that civil society makes all the time.
• There are very interesting technical details. For instance, the rules for automated post flagging are basically made of chewing gum and string. Or how machine learning classifiers are being created by Arabic speakers for dialects they don't understand. Facebook talks about how they have people working with 70 or 100 languages, but there are way more languages than that in active use on Facebook.

How do the documents you've seen or heard about relate to the work you're doing?

• The documents reinforce all the arguments we've been making for the last 8 years of advocacy, and they give us statistics that really drive the points home. Also help us understand the internal dynamics, again just reiterating what we already thought.
• It showed how hard it would be to "make a difference from the inside" at facebook, but also how much staff at the company use our arguments internally.

How do you feel about the implications of the FB papers so far? Is what’s happening "enough"? Are you surprised about any of the effects it had?

• It was impressive how quickly the media lost interest, and also how much Frances Haugen ignored the existing work on these issues.
• It was impressive and frustrating all the incredibly important information about impacts outside of the US and EU that didn't get nearly enough media attention, and that still need non US/EU eyes still.
• We hope that the release will encourage more document dumps from people inside these otherwise-unaccountable giants. We worry that the quiet reception might make potential whistleblowers feel like it would be futile.

What are the specific things you might expect to see in the documents that you think journalists might not have caught?

• Everyone who has ever had a conversation with Facebook has been fed some level of PR. These documents blow those claims out of the water over and over again, and there are very few journalists who have the background to understand.
• These documents touch on very specific issues. For example, this issue of garbage data that goes into machine learning.
• Or the issue of specific problems that we asked about and FB fed us a PR line...the contrast between the reality and their statements is so important.
• Also, obviously, "local context" is lost on US/EU journalists.
• One of the reasons dkg agreed to help Gizmodo figure out what could be published (and how to redact them) is that he wants the papers to be available to folks outside the current consortium, so that activists and reporters outside the global north can review them and write their own stories.
• But it's still weird to be a person from the global north sitting in that position. The redaction team that dkg is on is itself basically global north, and they themselves don't have the language expertise needed to ensure that everything in the documents is safe to release.
• dkg has had a few offers of help from different language speakers (e.g. to look for personal information about people in the documents that is in non-english), but if anyone else wants to offer skills in a particular language, you can reach him by e-mail: [email protected]
• Dia has focused largely on documents relating to Arabic language moderation and conflict zones. Facebook KNOWS about the problems and simply isn't fixing them. Their own engineers and policy people repeat things civil society has said word for word.
• The documents really throw into relief the contrast between resourcing for both English language in particular and US/EU in general and the "rest of the world." Arabic is either the 3rd or 5th most spoken language on the platform, with widely differing dialects, yet they simply don't have dialect expertise....yet the majority of countries in our region are "at risk" for mass violence.

Is there something written publicly about the revelations of the FB Papers relating to Arabic language moderation and conflict zones that you can share with us? What are the main NGOs in Arabic-speaking countries doing this work along with mnemonic?

• There have been a few articles that have touched on this, but nothing in depth.

What do activists need from this data to improve advocacy?

• Every activist should see every document pertinent to claims Facebook has made about how it works in their country/language/specific community (ie trans people)
• Seeing the documents helps you understand who you are actually trying to convince when you talk to facebook staff

Do you think we can work on a collective ~campaign~ on this? or are there ongoing ones that would just need amplification?

• Dia said before that ”their own engineers and policy people repeat things civil society has said word for word.” It is concretely useful for civil society actors to make noise about this, to encourage this kind of internal dissent. Even if a lot of it falls on deaf ears internally, that kind of situation can build morale/conscience/confidence among the folks closest to the machinery.
• Dia co-runs a monthly content moderation call with Jillian from EFF where they talk about these issues.
• And there is a LOT Of interest in doing a campaign around how civil society has to do customer service for social media companies
• There is also a lot of interest in doing a campaign focused on the language issue.
• The only way we "win" is by global NGOs following the lead of folks from impacted communities and the "global south" and refusing to allow "western" issues to get the spotlight.
• The key is organizing, which is always the most time consuming but ultimately rewarding work.
• It's worth remembering that sometimes there is no good answer to these problems if we remain in the context of a globe-spanning, unaccountable multinational corporation that can control the majority of the population's ability to communicate.

How can we support you?

• dkg is calling for people with non-english language expertise, or expertise in specific conflict zones, or places that might be touched on in these leaks to get in touch -- They need that kind of expertise when thinking about how to safely redact documents so that we can release them.
• There are two kinds of translation skills needed:
  • for redaction, to make sure that we understand the non-english text and details that show up in the documents, and
  • for wide-spread dissemination -- to translate the majority-english documents for global consumption.
• We don't want to accidentally doxx a content-moderator who is raising issues internally.
• It's also a tough space to navigate! part of the problem is that the platforms want to automate their content moderation, and they want scale. But it's not at all clear that automated content moderation will work at scale. And, the only folks with the capacity to do non-automated content moderation at scale are (for example) the Chinese government -- not a great lead to follow.
• The range of spaces that FB covers is large enough that the docs could cover just about anything.
• They need support to work on the funding of the project.
• You can contact dkg and dia to help with your expertise.

Do you all operate under an explicit theory of change? I can imagine "Releasing accurate documents holds big companies accountable" being automatic but just curious if it goes further in intention/design

• dkg doesn't think that releasing these documents is sufficient for accountability, but it isn’t a necessary first step. And, you can see this as basically what the company "transparency reports" should have been from the beginning.
• As we were discussing above, the documents themselves can help in advocacy discussions that are already happening. they make it much harder for companies to spout PR.
• So, for example, Dia have now several times reminded facebook that I know that when these documents were released, 77% of "counter terrorism" content moderation in arabic language was incorrect.

How are the ongoing advocacy discussions going so far? Incredibly incremental changes that don't get to the heart of the issues. Ultimately in these discussions, we will reach an end to the harm reduction that can happen (that is basically what most direct company conversations are). ​​ How should we handle document drops like this in the future?

• civil society should try to immediately convene meetings to discuss how to respond.
• If it happened today dkg would probably reach out via the COMO list, in the IFF mattermost, and via a few signal groups, to try to convene a discussion.
• One thing dkg considers for handling future drops is whether it's possible to have a streamlined process for getting reasonable civil society review for future leaks. The redaction work dkg has been doing with the team for Gizmodo feels pretty ad-hoc. Hopefully that will help people to:

critique the process and raise concerns about how it could have been done better, and help people tackling the next drop have something of a roadmap, at least for the things we did right.

• Dia is planning to do a post-mortem when they finally see the documents released in redacted form, where they publish our process.

What other document drops do we need to see?

• Google and Youtube
• But any of the big hegemonic platforms deserve to have their secrets exposed.
• Think about all the purchase information, market control, and review power that amazon has. It's all governed by their internal tech that the general public can't see.
• dkg would love to see clearer information about policies about engagement with local law enforcement, policies about what gets banned/accepted, etc.