EVENT TRANSCRIPT: COVID-19 and Social Media – Meeting Challenges using Lessons Learned from Countering Terrorism
DATE: 3rd June, 2020, 3:00pm – 4:00pm
VENUE: Online
SPEAKERS: Dr. Erin Saltman, Jessica Zucker
EVENT MODERATOR: Nikita Malik
Nikita Malik 00:03
Good afternoon, everyone. Thank you so much for joining us today at our event at the Henry Jackson Society. We’re just going to give everybody another minute to begin streaming in and will very shortly begin our event. All right, good afternoon, everyone. Thank you so much for joining us today. We’re delighted to have on board two excellent speakers for what will be, I’m sure, a fascinating discussion. So before we start, I’m just going to briefly talk about what today’s event is going to focus on, and give some background on the speakers and also some brief housekeeping rules. So today’s event, the intersection between COVID and counterterrorism, or the policies by social media platforms on extremist and terrorist groups, is a very pertinent topic, and we’re delighted to have two panelists who have extensive knowledge in this area. So, today’s discussion will look at the extensive use of social media platforms by extremist groups and terrorist organizations for propaganda and recruitment purposes. We saw this especially by the Islamic State, and also the response by technology companies who took very important and continue to take very important proactive policy decisions on removing material and banning users from their sites. So today, we’ll be discussing whether some of these same techniques and lessons learned can be applied to new challenges following the COVID-19 pandemic, where possible counternarratives be employed to address conspiracy theories put forward by extremist organizations and actors trying to explain the causes of the pandemic for example, and which aspects of the current crisis of health misinformation, for example, require a unique approach. So today, the Henry Jackson Society is delighted to welcome representatives from Facebook to discuss some of these key research questions on current solutions being fostered in this space. The Henry Jackson Society, of which I’m sure all of you are familiar, but if you’re not, is a think tank, based in Westminster in London, that focuses on foreign policy issues, but also has a large center on radicalization and terrorism, for which I serve as the director. Today we have two excellent panelists who we’re honored to have on board. The first is Dr. Erin Saltman, who is Facebook’s policy head of counterterrorism and dangerous organizations for Europe, the Middle East and Africa. Her background and expertise include processes of radicalization within a range of regional and socio-political contexts. Her research and publications have focused on the evolving nature of online extremism and terrorism, gender dynamics within violent extremist organizations, and youth radicalization. She also manages Facebook’s very important work with the Global Internet Forum to Counterterrorism, also known as GFCT. We also have on board today, Jessica Zucker, who is a Product Policy Manager at Facebook, leading misinformation policy in Europe, the Middle East, and Africa. Zucker joined Facebook in 2019, after three years at Microsoft, where she worked in the United States and in Europe covering cyber security, election integrity, and misinformation. Prior to working in the tech sector, Mr. Zucker worked in nonprofit and government, including the US Department of State cyber policy office, the US Department of Defense Southern Command, and as a Fulbright scholar in South Korea, where she co-founded a nonprofit. A San Diego native, she holds a Bachelor’s degree in political science and economics from the University of Miami and a Master’s degree in Public Policy from Harvard University. So, before we kick off, the discussion today, for which we have an hour will be based around 40 minutes of our speakers talking about various thematic ideas. And we will leave around 15 to 20 minutes for question and answers ending promptly at four o’clock. I’ll just flag that the questions today should focus on the topic of our discussion. So COVID-19, dangerous organizations, and misinformation and the themes that these carry over to terrorism. So just to start, we will go through some thematic ideas. So I’ll begin by posing the first question, which is that technology companies have come a long way in addressing online harm over recent years. What do you, Erin and Jessica, feel are the persistent obstacles that continue to inhibit further progress? How are these obstacles unique to COVID-19? Or how are they the same as challenges that tech companies or Facebook in particular, have faced in the past?
Jessica Zucker 07:11
This is Jessica from Facebook. I want to thank everybody for joining us today and to the organizers, especially for having both Erin and I come and speak to you about such an important topic. I want to just first start by a little bit of background on what it is that Erin and I do at Facebook. And then we want to dig into some of these really important questions. So just to sort of set the scene, Erin and I are both on a team called ‘Content Policy.’ Our focus area is on developing policies and reading the rules for what’s allowed and not allowed on Facebook. And my area of focus, of course, is misinformation. How we really approach misinformation, which I think is important that sort of a starting point before we get into some of the challenges and next steps is really threefold. So we’ve always had a policy of both removing content that is harmful, working with fact checking organizations to limit the distribution of misinformation. That is not harmful, but it’s still misleading and connecting people with more access to information from authoritative sources. In the context of COVID-19. We’ve been applying this threefold approach to harmful misinformation and other kinds of health misinformation about COVID. Since January, when the WHO declared COVID-19 a pandemic, what we started by doing is removing harmful content. So this is stuff like misinformation that could lead to imminent physical harm. You know, drinking bleach can cure Coronavirus, for example. And for misinformation that doesn’t lead to imminent physical harm. We’re working with a network of 60 fact checking organizations around the world who are experts and journalists themselves and are writing content based on their rating. So, if they determine something to be false, we take a number of different enforcement actions on the platform such as reducing that distribution, so less people are likely to see it, and giving people more information by putting warning labels on that content. And the last part of our strategy and I think one of the most important parts of how we’re going to combat misinformation about COVID-19 is connecting people to reliable sources of information. We’ve done this in a number of different ways, and we’ve rolled out some new, different techniques for connecting people directly to health authorities. So, one thing that we’ve done since the beginning of the crisis is creating a dedicated center on Facebook to give people information about COVID-19 connect them with health authorities, provide them with tips on how to stay healthy and safe and this all exists in what we call the Coronavirus information hub. Another way that we’re doing this is through setting people dedicated pop up messages through Facebook as well as Instagram that direct people to their local health authorities. And some places were sending people to the who website in other places like you’re in the UK, we’re sending people directly to the NHS. And what we’ve seen is that this has actually been quite helpful. And around the world, I think we’ve already found that about 3500 people have used these different resources to connect directly with health authorities. Now taking this next step into, you know, digging into your question about what are some of the challenges and what can we do better? I think that this is a really unique opportunity to talk about definitions really as a starting place. And I know that this is something that Erin can talk a lot about in terms of how we’ve defined terrorism, and other kind of important terminology in that space. When it comes to defining misinformation, we’re still really at the starting point. One challenge is that nobody really agrees on what we mean by misinformation. People use terms like fake news, disinformation, misinformation, information operations, all interchangeably, but they actually mean a lot of different things to a lot of different people. Some people might use these terms to talk about things that they don’t agree with. They might also be referring to these terms when they’re talking about bad actors who are trying to influence or manipulate people. And so, when I talk about misinformation, and when Facebook uses that term, what we’re really talking about is false or misleading content, regardless of the intent behind it. But even if we agree on that definition itself, there’s still not always a fine line between what is misinformation or false information. What is opinion and what is satire. So, before we even think about what we can do to regulate to develop policies and cooperation around this issue, I think as a starting point, we really need to agree on what is the problem that we’re trying to solve as a first as a first step? And then sort of secondly, I think it’s an important area to think about how health misinformation and information around COVID-19 in particular,is different from other types of misinformation. You know, when it comes to help, we’re getting really clear guidance from sources of authority, like the who are local health authorities, who can really clearly say what is false, what is harmful. And based on that guidance, we can write our policies and force on them appropriately. Now, the same kind of authoritative figures just don’t exist in other spaces. Let’s take, you know, political misinformation or political opinion, you don’t have that same kind of who authority that can help us with those. We’re crafting those rules of what is true and what is false. So, with that, I think I’ll stop and turn it over to Erin, who can dig into this even more depth from her perspective.
Erin Saltman 12:34
You left me with a really fun one about political dividing and misinformation. Thanks, Jessica. Going back to kind of that original source question, I think it’s just a good thing to remind ourselves that normally in our day to day, Jessica’s world and my world don’t always overlap. so dangerous organizations have a myriad of other adversarial shifts that we’re constantly having to evolve around. We’re constantly having to question how we define a bad actor. If it’s in an organized framework, what taxonomy that might fall under, if they’re openly violent extremist. A lot of misinformation has nothing to do with real core dangerous organizations. But we do see in times of crisis, especially that dangerous organizations can key off certain narrative arcs, certain types of misinformation that might have xenophobic tropes attached, or certain types of misinformation that might be meant to coordinate an influence in a certain way to get a certain political outcome or to get a certain action or activity from either government or civil society. So that’s when our paths tend to cross. And admittedly, they have been crossing more and more in the case of having to go through this pandemic, and look at risk mitigation, and especially around increased protests and other things that we’re seeing manifest day by day. in that space. You know, the persistent obstacles, as I said, are the adversarial shifts, as well as definitions that can be applied globally, so that we are not defining bad actors based on any particular ideology or religion. We are looking at behaviors, bad actor behaviors that can be scaled up so that when we look across different socio-economic and political differentials, of which globally, we’re dealing with a lot of different types of harms, and a lot of different types of organized groups, that we are trying to create a policy that can be applied evenly as much as possible. So, in our community standards, I think it still remains that we’re pretty much the only tech company that has given an open definition of terrorism and an open definition of what we mean by a hate-based organization. That in and of itself is a challenge as you know. Bodies like the UN, don’t necessarily have an agreed upon definition from one country to another….there are slightly different frameworks for defining and listing terrorist entities. But when we say dangerous organizations, that’s not just terrorism, that’s also groups that might be non-violent, but have a hate-based mission that attacks people based on protected categories under human rights, or entities like criminal organizations, mass murders, violent non-state armed groups. So, we are not the lighthearted teams at Facebook. We are kind of staring real world violence in the face and real world, hate-based organizations in the face on a daily basis. So, when we look to policy, it’s one thing we have policy that says we ban these different groups that proclaim a hateful or violent mission. We don’t want them to have a presence on our apps, and we remove content that represents them as well as praises or supports them. And to date, we’ve identified a huge range of these groups around the world because of how they engage in coordinated violence against others. And we have to then supplement that with how do we apply that? What does that mean for Facebook to have this policy both proactively, when we find things ourselves or reactively? When civil society, individuals on our platform, or even governments flag things to us? How do we then consider that? And I would say that about three years ago, we started developing a real serious playbook and a series of automated techniques to detect content related mainly to the most obvious terrorist organizations. When you build up artificial intelligence and tooling and classifiers, you have to start with huge amounts of material. And so, groups like the so-called Islamic State, and Al Qaeda and their affiliates have much easier starting points, because there’s a huge amount of content. They have logos, they have named media wings, there is branding. As we know, there was a lot of decentralization that took place, a lot of rebranding. But we were able to develop lots of automated techniques, taking from other harm types, such as photo and video matching technology to be used, such as recidivism, to try to ensure that we could label if the same bad actor is trying to get back on the platform. That’s taken from technology developed through spam filters, things like automated recognition of certain audio. So, if we get audio releases around things, we can try to use that logo detection. So, these are all playbook techniques that we can use as well as network analysis. So, when we think of processes of radicalization, we really don’t see that this happens in a silo. And the same trend offline, can be seen online whereby the phenomena of a completely lone actor is very rare, if ever, and so we do see that people operate in these groups that usually have online-offline overlap. And we can do some network mapping around, when a bad actor has been removed, because of obvious violations. Let’s say we see somebody trying to recruit or propagandize. We can do in-network analysis, where tooling can help us perhaps, surface a small group of core individuals that had shared similarly violated content or been part of similarly violating groups. And then that can be triage to human review. And I think that represents the real crux of how to ensure we can get to the scale, but try to also get to the accuracy on all these harm types, whereby it’s almost never just human or just machines. A lot of our technology has to be trained to triage, whereby the machine learning tools and tooling in general is meant to get at scale, and meant to identify core behavioral markers or core signifiers, like the logos, the slur terms, etc. But if it’s not extremely clear, that that is in and of itself, violating that doesn’t need to be triage to a human review team. So it is the triage between the machine techniques and the human review techniques. And whereby I think we’ve come a long way in proactively finding things ourselves, if you look at terrorism content, we take down 98 to 99% of what we classify as terrorism, we find ourselves, and that is not just the Daesh and Al Qaeda-related content that includes what we label around white supremacy, terrorism, or even in cell terrorism, Buddhist extremist terrorism, extreme right wing certain types of proactive violence that would not just be considered hate based organizations. According to our definition, a lot of that is in fact terrorism. That’s where it’s good to see some government start labeling these groups like in the UK, having groups like National Action actually appear on terrorism lists instead of just hate based lists. But then we still are having the issue that adversarial shifts happen so quickly. Name changes rebrands decentralization where there might not be a logo, that it is crucial that we work with intelligence communities that we work with actual individuals and NGOs on the ground that have better insights into what this really looks like. And so as much as we’re proud of the proactive efforts, that 1% that we’re not finding ourselves on the global scale is still a huge amount of work and a huge lift that takes both academic partnerships, as well as strategic partnerships with NGOs, to kind of cultivate a constant evolution. So the job is never done really in these adversarial spaces. Maybe I’ll stop there for now.
Nikita Malik 20:45
Thank you so much. That’s such an interesting start, particularly around the challenges on definitions and the idea of defining misinformation, and as we know, the difficulties in defining who is a terrorist and who isn’t. And I just have a lot of questions. But if you have questions listening to this, please do continue to send them in the Q&A, as we make our way through the talk, and so on to the next theme, which, you know, we would think that one of the most challenging areas building from the analysis that we’ve just had, is around this idea of what is legally criminal. So existing legal frameworks on criminality, and other legal frameworks on forms of social harm, hate crime, etc, as you’ve touched on, but how do you think different stakeholders are navigating this and is Facebook’s understanding or even us as users of the Facebook platform is our understanding of when and where to intervene?
Jessica Zucker 21:55
Thanks for that Nikita. I can start by telling you a little bit about how we think about this from the misinformation side. So, when our team is drafting policies, we are trying to write policies that are globally relevant and applicable for 2 billion plus users worldwide. Our goal is always to develop policies that are rooted in strong principles. For misinformation, this largely boils down to ensuring that our platform is a space for freedom of expression and voice walls, ensuring that our users are safe and secure. We’re constantly making decisions every day on specific pieces of content that are edge cases, but also when we’re thinking about drafting our policy guidance, in order to make sure we’re reflecting a proper balance between freedom of expression and security. When it comes to misinformation that can lead to harm offline, we make a shift in our calculus. And anytime we see misinformation that can have an offline impact of imminent physical harm or imminent physical violence to people, we’re going to remove that from the platform. And we’ve had this policy in place for the past several years, how we think about defining this kind of harm in the context of misinformation is admittedly quite difficult. There’s no easy answer of what misinformation will always lead to harm or what that causality will always look like. And so, when we’re thinking about the best way to enforce this kind of policy, what we’re doing is we’re grounding our decision-making in our analysis and what experts are telling us. So for example, when it comes to issues of misinformation and potential physical violence, we’re going to work with on the ground experts. These people are oftentimes academics or human rights groups, NGOs, international organizations, people who have an expertise on a certain topic in a certain place, who can help give us more context about what that causality might be between both false information that’s posted online and what the offline impacts might be. And really, this is how we’re routing our approach to dealing with this kind of misinformation at a global level. When it comes specifically to harmful misinformation about health or about COVID-19, what we’ve been doing is partnering really closely with health experts, not just the WHO, but also local health authorities. And again, the same kind of community of people from freedom of expression advocates to human rights groups and NGOs are helping us make that on the ground calculus of what kinds of rumors, what kind of misinformation, might lead people to harm. In the case of COVID-19 what we’re really looking for is harmful misinformation that if somebody believed it, it could increase their likelihood that they might get sick, that they might spread the disease. And that’s why we’re removing harmful contents like false cures or false prevention information or things that deny the existence of COVID as even something that’s happening. We’re attributing symptoms that people might be feeling to 5g radiation. These are the kinds of things that, based on our conversations with these kinds of experts, we believe could lead to harm. And that’s why we’ve used this policy to enforce that kind of content and remove it from the platform.
Erin Saltman 25:12
Yeah, I think I think that Jessica covered a lot around that. I think that it’s always hard when we’re talking in the case of the UK, obviously, when we talk about what is legal or illegal, legal frameworks look very different in different parts of the world as well. And what is legal in one country can be illegal quite easily in another. It is obviously never what we want to do to kind of allow for illegal activities. And when we talk about the criminal nexus with all of this, obviously, some dangerous organizations have overlap with other forms of criminality. And it’s not just limited to Jessica’s and my teams, we’re also looking at the teams that deal with broader incitement that might have to coordinate with real world law enforcement, if we are seeing indicators, if we see something that we think is going to lead to real world harm. That’s not something we want to keep to ourselves. So, while we want to mostly preserve all the privacy aspects, if you unearth a real world harm scenario, that might be the only time that we’d ever want to proactively work with law enforcement and make sure that that does not actually take place in the real world, but also things like promoting criminal acts. And that can be really hard, especially with some of the things we’re seeing right now where maybe the promotion of use of Molotov cocktails or looting those or promotion and glorifying of criminal acts, even though they have a nexus with possibly dangerous orgs, possibly activists and other things. And we’re having to take very sensitive review over which policy bucket that falls under. And it’s never an easy job to decide, okay, here’s where it falls within our policy framework, I think may be of interest is just some of the developments over the last six months. Again, we’ve been working a lot with our threat intelligence team to leverage some of the strategies that they’ve built out around combating coordinated inauthentic behavior, because there’s a big difference between the whack a mole approach of removing one bad piece of content or debunking one piece of misinformation. It’s when these things start being coordinated when they have group dynamics that we really get the scale and potential harm in a different way than the one-offs. And so, working with our threat intelligence team, we’re trying to develop new tactics that target band groups presence across our apps. And the main way we’ve been trying to do this is identifying signals that indicate a band organization has a presence in the first place, and then proactively investigating associated accounts’ pages in groups, before just removing them in a whack a mole trying to see, what can we say does have core signals, what can we say is obviously violating in a wider terrain than just the one-offs. And once we remove that presence, we work to identify attempts by the group to come back on the platform so that we can kind of map what those tactics look like. And we’re also studying how different types of dangerous organizations might initially try to bypass our detection, things like recidivism, the returning of the same bad actors, as well as how they attempt to return to Facebook after we remove a series of accounts. And this is to try to strengthen our enforcement and create new barriers to keep them off the apps. But again, this is a space where we see a lot of adversarial shifts. And so, we know that it’s not just taking place on Facebook, this happens on a myriad of different apps. And so, we might see larger coordination across apps that aren’t owned by Facebook, on to Facebook, and vice versa. And it’s something that we’re constantly working on.
Nikita Malik 28:55
Thank you so much. That’s so interesting, especially in how all of these different spaces work together. And Facebook is often used as a space for all kinds of different things and to kind of pivot to different apps as well. And so, moving on to, you know, users and audiences something that we’ve been fascinated with for some time, is the identification of audiences, by the Facebook platform, who may be vulnerable to extremist and harmful content. And could you talk us through a little bit about how Facebook does this and what efforts if any, you take to intercept these harmful pathways of communication on your platform?
Jessica Zucker 29:48
I can go ahead and start. Erin will be able to speak in much more detail about how we’re able to intercept extremism content, but focusing in on how we’re trying to you know, give people access to more information and connect users with authoritative information as a way of intercepting them from seeing misinformation, we’re actually taking quite a few different steps. And to be honest, we’ve constantly been learning from what our users are telling us what research community is saying, and working on ways to improve this over time, we’ve made pretty substantial changes just in the past year to how we’re doing this, I’d be happy to talk more about what we’ve learned and what we’ve changed, but at a high level. A couple ways that we’re approaching this is, first, let’s sort of talk through what the experience is from a user, if you’re scrolling through your newsfeed. And let’s say, you come across something that is unmarked, it doesn’t look like it’s misinformation, and you go ahead and share it. And the next day, one of our 60 fact checking partners from around the world breaks that thing that you shared, what we’re going to do is we’re going to send you a notification that lets you know that something that you had shared previously, has been rated by a fact checking organization, and it gives you access directly to that debunking article. Now let’s take a different experience. When you’re scrolling through your newsfeed and you come across something that’s already been rated false by a fact checker, you’re going to see it with a warning label that’s already on top of that content. So, you can’t even see that content directly, you’re going to have to click through a few different screens, if you want to see it. What we do, as we put those click through options, if somebody has to click through to see to see that content, we give them the option of actually seeing the fact checking article first before going ahead to see that content. And what we’ve found is that research is showing based on the data from having done this is that this is actually quite an effective method of intercepting that engagement with misinformation. 95% of the time that people come across that warning label, they do not click through to see the original false rated content. And we think these kinds of results are really promising. We rolled out that new kind of warning label on misinformation across the platform in December, January of this year. And so, these initial kinds of findings are really helpful for us. And the reason why we made these changes in the first place is based on feedback that we got from our community from studies that we did from external experts who helped us understand what is the best way to communicate to people when they come across misinformation. And when it comes to content that we’re removing. So, I spoke about earlier that we remove misinformation when it can lead to physical harm offline, we still want to be able to give people access to information even as we’re removing their content for violating this policies. And one way that we’ve done this is for people who have liked, shared or commented on something that we removed for violating this policy, such as you know, drinking bleach, cures Coronavirus, or COVID doesn’t exist….these kinds of harmful health misinformation claims. What we’re doing is after we’ve removed that content, we’re sending people a dedicated personalized notification that lets them know that something that they interacted with, was removed for this policy and we’re linking people directly to the Mythbusters website, which if you haven’t seen that you should definitely take a look is a great running list of hoaxes that the who has debunked they include things like 5g causes Coronavirus, or the heat or sunlight can kill Coronavirus. And so again, even as we’ll be removing people’s content for violating this policy, we’re still trying to give people access to as much information as possible. And the last thing that we’re doing to kind of intercept and redirect people to authoritative information, instead of seeing fake or misleading or low-quality content. If people are trying to search for COVID or Coronavirus, or other kinds of similar iterations, we provide them with that same opportunity of redirecting them to those authoritative health expert websites. So if you search for COVID-19, and Instagram or Facebook, what you’ll see at the very top of the search results is a link that will prompt you to go check out information about COVID-19 from your local health authority. And again, we’re based here in the UK. So, this link will go directly to the to the NHS. But I’ll leave it to Erin to talk more about how we think about this in terms of intercepting extremist content.
Erin Saltman 34:23
Yeah, I just I think it’s really good to know that a lot of these tools are way that we talked about harms. We’re constantly working across our teams to see what worked for spam or child safety, and are those tools applicable to other harm types? Where can we use similar tactics for misinformation or dangerous organizations. Not every tool is applicable to every harm type. It really does have to be custom fit, but that’s also a constantly-evolving space. I think it’s a little bit different with some of the discussion of what we would consider vulnerable audience to actually stream isn’t because this sort of audience segmentation is completely dependent on socio-political, geographical, and ideological factors. So, are we talking about people we think might be vulnerable to Islamist extremism rhetoric in the UK? Or white supremacy-related dialogue in the UK? Or are we talking about other forms of xenophobic, radicalized networks, and there are many again, when you go around the world, so in each case, there are these different types of dangerous organizations or violent extremist threads in question. And different patterns, especially in different parts of the world of app usage, recruitment, strategy, coded language, or symbols, slogans and slurs that might be used to navigate away from being detected on our platforms. I really think that this goes back to, and I will just keep hammering, at home. The reason why it’s the cure for any of these topical questions is never going to be a tech platform doing it on its own. It’s never going to be that Facebook somehow solved terrorism and violent extremism whereby no other sector was able to do it. And I don’t think we should be naive. And we should humble ourselves that it’s only because of cross-sector collaborations that we can get to a better space here. I think particularly in the UK, having worked in different parts of the world, we are very lucky that we have a huge range of NGO experts that just focus on violent extremism and terrorism. You know, we have a Henry Jackson Society, but we also have groups like Hope not Hate and RUSI, and the Center of Analysis for the Radical Right, and ISD and Moonshot CVE. The fact that you can just spout off a range of groups like that, there’s very few other countries I can go to where I could list off the top of my head such a great number of expert researchers, as well as practitioners that are really deep diving on this is probably a testament to the fact that the UK and we can critique or love or loathe things like prevent, but the fact that it’s been a dialogue in this country more openly, in political spectrums, as well as within civil society for a decade plus, that did not exist in other countries, even as a dialogue for many, many years. So, I think that when we talk about improving the identification of audiences, the programs that I have seen that have worked the best, even particularly to the UK are times where each sector realizes what their best place to do. So, we know that Facebook will not be the credible voice of a counter-narrative or counter-speech or alternative narratives. Whereas Facebook might be better placed to flag to a harm type, and redirect you to health organizations, Facebook’s not going to be credible to say, “Hey, we think you might be extreme,” you might want to take a look at this thing that we made, it’s not going to land well with target audiences. So it really does come off of being a little bit more nuanced and developing programs like the online civil courage initiative, whereby we train a bunch of NGOs and work to upscale and optimize what their content does through ads targeting, and get really refined with target audiences or through the peer to peer network, whereby it allows students in universities to co-develop and create their own ideas for how to push back on hate speech and bigotry more innovatively. And if you want innovation online, go to anybody under the age of 25 and ask them how they’re using online apps and tools to engage. And that’s going to be where you see innovation. Same thing with more downstream with things like, you know, the redirect methodology was developed by jigsaw working with some NGOs, including Moonshot CVE here in the UK, but it’s been used by different platforms in different ways. And for Facebook, we found that, for example, when you search for bad things on Facebook, you usually go to other platforms like YouTube for finding specific pieces of content, but you go to Facebook to find specific people and groups or people in organizations. So, taking some methodologies like that we have been building out ways of connecting through search terms, certain violent extremist terms in a given country that has disengagement NGOs, like exit USA or exit Australia, and using those search terms to redirect you to disengagement NGOs. Now, those are very sensitive partnerships as well takes a lot of trust building. And it’s in its nascent phase still, but we’re seeing the learnings from the US and Australia markets where the redirection was around white supremacy terminology, and we are finding that it exponentially led to more people reaching out to an exit program and engaging with them to actually help them leave those moves. movements. And that’s, that’s results that you could never get on your own as a tech company. So, I think that’s where we need to be more proactive in partnership building. And we also have to consider just as a side note, before moving on, we have to consider transparency, as Jessica said, so if we remove you, if we redirect you, if we surface content to, because of what might be considered a quote, unquote, extremist indicator, we actually need to be transparent as to why you should never have content removed, and not know why you shouldn’t be able to question that. And so, the tradeoff between, you know, we want to be innovative, and we want to strategically communicate against extremism. But we also need to be transparent around those behaviors and indicators that might be triggering. You’ve been put in a feature blocker, you’re being surfaced content, and you’re questioning that. So that’s always a little bit of a tradeoff as well.
Nikita Malik 40:57
Thank you so much. So in the interest of time, and because we’ve had so many questions from our audience members, I think it’s fair that we move on to questions. And I’d encourage you if you haven’t had the chance yet to put your question in the question-and-answer box. I have many questions. So hopefully, if we have a little bit of time left, we can end on one of our questions on what a healthy space online would look like. But I’ll start with some of these questions now. So, the first one is from Alexander Carlisle. And if I’m correct, this is Lord Carlisle, he was the independent reviewer of terrorism legislation. So, I will leave him to ask his question now.
Lord Alexander Carlisle 41:55
My question was, how does Facebook deal with content that does not cause physical harm, but provides, for example, a bogus theological explanation of COVID-19, such as that it was sent to punish apostate Muslims? I presume that Facebook would wish to remove such content as plainly not being part of a permissible discourse.
Jessica Zucker 42:33
That’s a great question and an important one. So, taking a step back and thinking a little bit more about what I talked about at the beginning of when do we remove misinformation/when we don’t? Again, our principles for when we remove misinformation is based on whether that calculus between freedom of expression and safety and security needs to be rebalanced, we remove content because we think that it could lead to imminent physical harm offline. And that’s a really unique kind of case scenario, when we will remove misinformation, we’re not going to remove content from the platform just because it’s not true. We don’t have a rule in our community standards, that says that everything that you post on Facebook or on Instagram has to be 100% accurate, you can imagine how difficult that would be to enforce. But also, frankly, I don’t think that’s what people would want to think about, you know, our own personal experiences of when we may have posted content that was not true or had slight inaccuracies or exaggerations. And so that’s really why we don’t have a policy on having to post everything that is true. Now, that said, we still don’t want to have bad, spammy, misleading false information on the platform. And so, if it doesn’t meet our threshold of removing because it’s harmful. Our approach is to work with our network of independent factoring experts I mentioned before, we have 60 plus partners around the world covering 50 plus languages. And so, what we do with how we work with them is for content that is still kind of false, spammy, misleading, the kinds of examples that you’ve raised in your great question, we would work with them, and they would rate that kind of content. And based on their rating, we would take a number of different enforcement actions. So based on one false rating, what we would do is we would dramatically reduce the distribution of that content in the platform. So, in other words, it’s, it’s much less likely that people will come across it. And when they do come across it, we put a strong warning label on top of it. If you try to share it, we put a lot of friction into that. So, if you try to share content that’s been rated false, we give you a pop-up window that says, something that you’re about to share has been rated false by a fact checker. Are you sure you want to share it? Or do you want to see the fact checking article instead? And we think that this is really important because again, it really goes back to our core principles of ensuring that people Have a space for free expression. But also we’re trying to make sure that we can connect people with authoritative sources of information, whether that’s from fact-checking, debunking, or as I mentioned before, connecting directly with health authorities to get accurate information from health experts. And just to give you some stats on this based on 7500 factchecking articles about COVID-19 misinformation in just the month of April alone, we applied ratings, so that warning screen and the demotions of content to 50 million pieces of content across the platform. And this is just related to COVID-19 misinformation. And as I mentioned earlier, the data that we found is indicating that this strategy is quite effective 95% of time, people are not clicking through to actually see the content that’s been rated false once they see that warning screen. Now, just you know, to close and to ask Erin to jump in here. Obviously, we’re trying to get better, you know, we’re constantly working on ways to detect misinformation faster. We’re always trying to expand our third party, fact checking partnerships, our partnerships with local organizations that can help us understand that causality between misinformation offline harm. So there’s certainly a lot more work to do. But I do think that these findings are helpful and indicating that we’re moving in the right direction.
Erin Saltman 46:26
Well, I think I think Jessica, you covered kind of the pooling aspects of that and some of the numbers behind it that have some positive indications in a big way to start with, but just maybe, maybe Lord Carlisle, knowing some of the trends or posts that you might be referring to with relation to Islamist extremist and or terrorism trends. I mean, definitely content does not have to have a link to physical real-world harm to be removed more broadly. So, a lot of hate speech is not physical harm, and it would be removed, but also any content that we do link back to being explicitly related to labeled dangerous organizations. So, when we do look across religious communities, we know that we’re not going to be the arbiter of truth when it comes to religious societal hierarchy questioning or questioning what is or is not ordained by God more widely. I don’t think our communities would appreciate Facebook being the arbiter of that big existential question. But if we are seeing certain pieces of content, when you do look across Muslim community groups, most of them are putting out very good factual and important messaging about quarantine about staying safe and healthy. We have seen certain groups linked to Daesh, linked to Al Qaeda networks, linked to other Islamist extremist groups that are putting out messaging around trying to weaponize and or self-promote because of the pandemic. Anything linked to those groups, propaganda coming from those groups would naturally come down under a dangerous organizations policy. So that’s where it goes out of Jessica’s wheelhouse, and into our territory, whereby if we see that the source of a piece of content is from one of these dangerous orgs, in and of itself would come down. So once we have identified you as a violent extremist and or terrorist group, even if you’re putting out a cooking show, that’s going to come down because you in and of itself, those groups and their content would be removed. So, it goes a little bit across both of our fields to speak to that.
Nikita Malik 48:29
Great, um, so the next question is from Colin Jack. And so, Colin’s question is, once you have made a decision in relation to the removal of an insertion, is there any method of appeal, if there is still disagreement?
Erin Saltman 48:51
Yeah, I can jump in there. I think that this is probably one of the biggest mechanisms that was necessary to put in place a few years ago, which was the right to appeal. So, we always had the functionality that if your profile or group, a larger complex entity was removed, you could always appeal. But appeals were rolled out a few years ago to cover everything down to the content level. Now, there are a few things you can’t appeal. Like if we have identified and it is matched against the fact that you are sharing child exploitation, or bomb making instructions. There are some things that you can appeal if it is a hard match, and that’s what you’ve been sharing. Those things are not allowed. However, we do know, especially in the extremist space, there’s a lot of satire, there’s a lot of difficult memes going both ways that sometimes it’s hard to read the humor read in between the lines. And so, we do know that sometimes content might come down erroneously, it’s usually not because of a machine error. It’s usually really hard for people in different parts of the world around the clock to assess the nuance of the way that something might be shared or posts, so down to content level that can be appealed. And what happens through the appeal process is that it then goes through a double review, and gets escalated if there’s not agreement, even third reviewer at a higher level. And I think what’s also important is if that one piece of content that was removed, we realized we were wrong, and it gets reinstated. It has an amplification effect. So maybe 100, people that and all 100 posts got removed, but only one person appealed it. If that got appealed, all 100 posts would be put back on. So, you can go to the community standards enforcement report, our most recent transparency report just came out a couple of weeks ago. And it has numbers and rates around appeals and content restoration. And you will see in some cases, that content restored is higher than the number of appeals because of that amplification effect of it only takes one appeal, or to perhaps have quite a larger amount of content restored.
Jessica Zucker 51:04
They can just add in that what Erin’s covered is the appeals process for content that we’ve removed from the platform. We also have an appeals process for fact-checking as well. So again, it’s not Facebook or a team of people like Erin and myself making those decisions about what is false on the platform. We’re working with this network of 60 fact-checking organizations. But if you have content that you’ve posted, that has been rated false, and you disagree with either the eligibility of that content from being rated in the first place, or if you disagree with how the fact checking organization has applied their own code of principles, for example, you can appeal directly to the fact-checking organizations. And above that, there’s also a mechanism in place to sort of check the checkers. So, all of our fact-checking organizations are part of the International Fact-Checking Network, which is a global, nonprofit organization. And they have a very rigorous process where every year they are certifying the fact checking organizations and recertifying those organizations based on code of principles around you know, upholding certain standards, transparency sources and methods, non-political biases, etc. And so, there’s sort of a multi-layered approach to ensuring that even the people that are fact checking content are also getting checked themselves.
Nikita Malik 52:27
Great, thank you. So I’m just worried that we’re running out of time, I’m going to take two more questions based on the theme. So, the first question is actually to on the theme of anti-vax, we’ve had two questions on this. The first is from Daniel Terhim who says what about the anti-vaccination movement who aren’t a dangerous organization are affiliated with a CBE, which is countering violent extremism, but will still spread misinformation about effective treatment on platforms such as Instagram and Facebook. Would they be classified under bad actors, or their activity trace despite the absence of a hate-based mission? And then the second was from Bex Felton, who said, given recent analysis on how anti Vax activists exploit community-based algorithms and closed group dynamics. How do you believe Facebook should be mitigating the risk of immediate harm to health?
Erin Saltman 53:48
Sure, I mean, from a high level, and then you know, Jessica can take it from the different level, just from my perspective, we have seen certain narratives go across different lines, we have seen anti-vaccination movements, even employed by the Taliban or by certain dangerous organizations. But we’ve also seen it much more widely. As somebody that originates from California, even some of the more liberal hippie movements have gotten really into it in recent years. So, when we think that something’s going to lead to harm, I mean, there’s, there’s that nexus of some things are not just black and white. And I think we could get better in the gray space for things like wider extremism, which I know Nikita knows about, and that we continue to debate and research. But for things like anti-vaccinations and other bits of that might be harmed. Some of it’s based in opinion, some of it’s based in misinformation. Again, this is where we need to decide what is being debate in a public way where it’s your choice, whether or not you get vaccinated, what is actually going to then lead to subsequent harm. A lot of this comes into the down ranking space to make sure that it’s not Being more highly promoted that it’s not actually going to be surfaced as your top results. There’s some more work to be done around that. But I know it’s extremely sensitive, especially in places like America, but also in parts of Africa and the Middle East, if we’re honest, and even parts of Europe, so that again goes back to the fact-checkers, but we are looking for just what we consider bad behavior attached to the debate around vaccination. So there might be other harm principles that come into play, because when we say “anti-vax”, it’s definitely not one thing. There’s lots of different group dynamics within the anti-vaccination space. So, it’s a kind of multi-pronged question at the end of the day, I think. And as far as the exploitation of algorithms, as I said, it’s a constant labeling, it starts with labeling. So how, what is the behavior that we are deciding is going to cause harm? It doesn’t have to be real world violence to be harmful. And then how can we take that real indicator or behavior again, that is agnostic of wider principles of any ideology, or religion, etc. But how can we take what is identified as a hard bad behavior and put friction there, put down ranking, make sure it’s not being surfaced as easy? Sometimes it’s a case where people might be sharing a link that leads to a harmful third-party site, how do we make sure that that link is not allowed to be shared on Facebook? And so those are there’s kind of a multi-pronged effort, but anti-vax is a many-headed beast, I think. But lots of efforts around the down ranking and sort of friction causing has already begun. I don’t know if you want to add to that at all.
Jessica Zucker 56:39
Yeah, I just want to underscore what you said, Erin, what do you think is really important that anti-vax sentiment is not the same as vaccine misinformation. Those two things are quite different. And sometimes they do overlap. When it comes to the vaccine misinformation itself, exactly what Erin said that our approach to COVID has been for the past couple of years, to try to give people more accurate information from health authorities themselves. We’ve done this through the same kind of techniques, such as redirecting people to help authorities when they’re trying to search for vaccines, or vaccine misinformation and Instagram or Facebook, or when they’re trying to join groups that are focused on vaccines as well. But again, even if these vaccine misinformation focused communities are spreading misinformation that would violate our policies, the same kind of rules applies as how we’ve talked about for the past hour. So, you know, if it’s harmful, and it could lead to somebody getting COVID, or spreading COVID, we’re still going to remove it, regardless of who’s posting it. We definitely know that there’s going to be more work to be done, particularly as we get closer to having a vaccine for COVID. And just as we’ve approached our policy around this space, from the beginning, we’re going to be in constant contact with health authorities who have local health experts, we’re going to be really close to things like the ongoing vaccine trials and can help us understand what is what is false, frankly, as we see more information coming on the platform about a potential vaccine in the future.
Nikita Malik 58:16
Thank you so much. And the last question, which is from John Moulton. Is there a way to see who has been forwarding information that has been debunked by fact checkers, to transparently help users determine the veracity of their information sources?
Jessica Zucker 58:34
I can take that one start. So, I mentioned earlier, there’s a couple of different ways that we’re trying to do like user interception of messaging. And I think that this really applies to the question that you’ve asked, which is really a great one. There’s quite a few things that we’ve built into the Facebook product that will indicate when content has been fact checked when you’re trying to share it if it’s been fact checked. The idea being that like at each step of the process, people are getting information about something that they’re trying to read or share has been reinforced by a fact checker, similarly, like all of our fact-checking organizations have their own dedicated websites. And we publish a list of our fact checking partners on our Facebook Help Center. And so, it’s quite easy to just go to those websites directly if you want to see, you know, running list of all the kinds of content that they’ve been fact checking. I think that AFP has done a fantastic job of this. They have offices around the world. And so, they’re basically collating all the hoaxes that they’ve debunked from APAC to IMIA from the US and Latin America and putting on a running list and database. Similarly, the International Fact-Checking Network has done the same thing. And so, these are just a couple of resources that I think will be helpful and trying to get more information about, you know, what content is being spread and I’d be happy to share with Nikita some of these resources. So, we can make sure that we’re following up with you all offline.
Erin Saltman 1:00:04
I know we have zero time left. But in the nexus to violent extremism and hate-based organizations, we’ve also been doing some really interesting partnerships with certain NGOs in different parts of the world to track what they think the top misinformation that has a real world harm nexus to it is and working with the third party NGO, to identify down rank cause friction redirect to better information. And in a case study, one of the last times that I was in Pakistan, for example, even a sometimes difficult space to navigate in the online information space, they were able to track about 30 different narrative arcs that they thought were leading to potential violence or real-world harm. And over the course of three months down rank to the point that traction on those narratives had dropped about 85%. So, there’s some really interesting ways to collaborate across sectors to identify what different harm looks like, and decrease the traction on it, including limiting forwarding capacity. So even in WhatsApp, because of learned lessons of how forwarded misinformation might operate, is now restricted, the amount of times any piece of content can be forwarded. So actually, limiting forward capacities knowing that the forwarding mechanism could be something that that helps something go viral in an unhealthy way. So just a couple final parting things of food for thought. I’m sure you guys have extra questions. I’m happy to, we can follow up with Nikita afterwards. If there’s some of those questions, we can send some links, any links that we mentioned during this talk just so that they go around? I’m sure we didn’t comprehensively cover even the tip of the iceberg of questions that people might have.
Nikita Malik 1:01:39
Well, we’re so grateful for your time and the conversation was absolutely fascinating. There have been tons of questions. Unfortunately, we haven’t been able to go through all of them. So I just echo what Erin said, please do contact us at the Henry Jackson Society [email protected] If you have further questions and want more resources, and we will try and get back to you. And all of our research by the HJS is also available for free on our website. So, I hope that this is the beginning of a wider discussion on very pertinent on topical issues. And thank you so much, Aaron and Jessica for your time. We really appreciate it. And everyone else enjoy your afternoon.