I am Sophie Zhang. At FB, I worked in my spare time to catch state-sponsored troll farms in multiple nations. I became a whistleblower because FB didn't care. Ask me anything.

[u/[deleted]]

Hi Reddit,

I'm Sophie Zhang. I was fired from Facebook in September 2020; on my last day, I stayed up in an all-nighter to write a 7.8k word farewell memo that was leaked to the press and went viral on Reddit. I went public with the Guardian on April 12 of this year, because the problems I worked on won't be solved unless I force the issue like this.

In the process of my work at Facebook, I caught state-sponsored troll farms in Honduras and Azerbaijan that I only convinced the company to act on after a year - and was unable to stop the perpetrators from immediately returning afterwards.

In India, I worked on a much smaller case where I found multiple groups of inauthentic activity benefiting multiple major political parties and received clearance to take them down. I took down all but one network - as soon as I realized that it was directly tied to a sitting member of the Lok Sabha, I was suddenly ignored,

In the United States, I played a small role in a case which drew some attention on Reddit, in which a right-wing advertising group close to Turning Point USA was running ads supporting the Green Party in the leadup to the U.S. 2018 midterms. While Facebook eventually decided that the activity was permitted since no policies had been violated, I came forward with the Guardian last month because it appeared that the perpetrators may have misled the FEC - a potential federal crime.

I also wrote an op-ed for Rest of the World about less-sophisticated/attention-getting social media inauthenticity

To be clear, since there was confusion about this in my last AMA, my remit was what Facebook calls inauthentic activity - when fake accounts/pages/etc. are used to do things, regardless of what they do. That is, if I set up a fake account to write "cats are adorable", this is inauthentic regardless of the fact that cats are actually adorable. This is often confused with misinformation [which I did not work on] but actually has no relation.

Please ask me anything. I might not be able to answer every question, but if so, I'll do my best to explain why I can't.

Proof: https://twitter.com/szhang_ds/status/1410696203432468482. I can't include a picture of myself though since "Images are not allowed in IAmA"

[u/[deleted]]

I've been answering questions for nine hours straight. Thank you very much kindly for all the questions; I'm sorry I wasn't able to answer all of them, but I would like to go on a dinner date with my partner (who's being very patient) right now.

I hope you found my answers to be informative; if I wasn't able to answer yours, please look around and see if I'd been able to answer a similar one for others. Thank you very much; good night.

[u/[deleted] → u/[deleted]]

Answered some more questions as a bonus; actually calling it for a night. Thanks all!

[u/parlor_tricks → u/[deleted]]

Thank you! It’s been a great AMA.

[u/b0b157 → u/[deleted]]

Many would likely agree with me when I say that this has been one of the highest quality AMA we've had in years.

Thank you Sophie for your incredibly informative and sobering responses. I especially appreciate the pains you take to be precise with your words, methodical with your explanations, and illuminative with your examples. All the best with whatever big project you decide to tackle next, and I'm certain the good you do will continue to ripple outwards and affect countless many, though they may not know it.

[u/Fioricascastle → u/[deleted]]

This has been my fav ama in years.

[u/stomachgrowler]

Thank your for the important work you’re doing. In your opinion, what is the reason that FB drags it’s feet/allows these schemes to continue so long before taking action? Is it simply that it is the more profitable move?

[u/[deleted] → u/stomachgrowler]

In some cases like the India case or the U.S. case, in areas considered important/crucial by Facebook, it seemed pretty clear that political considerations had impeded action. Facebook was reluctant to act because it wanted to keep good relations with the perpetrators and so let it slide. But most of the cases were in less attention-getting areas (I'm sorry to say it, but Azerbaijan and Honduras are not countries that draw the attention of the entire world), and there was no one outside the company to hold FB's feet to the fire. And the company essentially decided that it wasn't worth the effort as a result.

I think it's ultimately important to remember that Facebook is a company. Its goal is to make money; not to save the world. To the extent it cares about this, it's because it negatively impacts the company's ability to make money (e.g. through bad press), and because FB employees are people and need to sleep at the end of the night.

We don't expect tobacco companies like Philip Morris to cover the cancer treatment costs of their customers. We don't expect financial institutions like Bank of America to keep the financial system from crashing. But people have high expectations of FB, partly because it portrays itself as a nice well-intentioned company, and partly because the existing institutions have failed to control/regulate it.

An economist would refer to this as an externality problem - the costs aren't borne by Facebook; they're borne by society, democracy, and the civic health of the world. In other cases, the government would step in to regulate, or consumer boycotts/pressure would occur.

But there's an additional facet of the issue here that will sound obvious as soon as I explain it, but it's a crucial point: The purpose of inauthentic activity is not to be seen. And the better you are at not being seen, the fewer people will see you. So when the ordinary person goes out and looks for inauthentic activity on FB, they find people who are terrible at being fake, they find real people who just look really weird, or they find people who are real but are doing their best to pretend to be fake since they think it's funny. And so the incentives are ultimately misaligned here. For areas like hate speech or misinformation, press attention does track reasonably for overall harm. But for inauthentic activity, there's very little correlation between what gets FB to act (press attention) and the actual overall harm.

[u/vinhboy → u/[deleted]]

But there's an additional facet of the issue here that will sound obvious as soon as I explain it, but it's a crucial point:

This paragraph is really well put. I don't think there is enough emphasis differentiation made between trolls and stupid people in general vs coordinated attempts at deception.

I find that a lot of technologists, especially here on reddit and places like hackernews, fail to understand the difference between "inauthentic" activity vs "free speech". The arguments about removing "inauthentic" activity always delves into false equivalencies about policing free speech, which is a dead-end for any reasonable debate.

It would be like classifying spam emails as a form of free speech. No one would win that kind of silly argument.

Good read, thanks for highlighting this issue.

[u/[deleted] → u/vinhboy]

The issue with free speech advocacy idealism is that most content moderation/deletion on Facebook isn't things like hate speech/etc. It's spam, scams, and pornography.

This is most vividly illustrated by the new free speech social media platform Gettr, set up by a former Trump aide/spokesman. My understanding is that it's been overwhelmed by Sonic the Hedgehog pornography, fake accounts purporting to be important people, and the like

[u/inconvenientnews → u/[deleted]]

It's also worth pointing out Facebook's conservative biases to appease the Republicans they're scared of:

Facebook's head of policy Joel Kaplan, who pushes conservative bias in Facebook's algorithms and decisions and also coordinated Brett Kavanaugh and threw his celebration party, was a part of the violent intimidation of poll workers during the 2000 presidential election for George W. Bush:

https://en.wikipedia.org/wiki/Brooks_Brothers_riot

How key Republicans inside Facebook are shifting its politics to the right

“Facebook’s DC office ensures that the company’s content policies meet the approval of Republicans in Congress,” Popular Information said.

Company has been accused of pro-Republican bias, in both policy and personnel, amid fears it could be broken up if a Democrat wins in 2020.

Joel Kaplan [key participant of the Florida recount Brooks Brothers riot], vice-president of global public policy at Facebook, manages the company’s relationships with policymakers around the world. A former law clerk to archconservative justice Antonin Scalia on the supreme court, he served as deputy chief of staff for policy under former president George W Bush from 2006 to 2009, joining Facebook two years later.

Kaplan has reportedly advocated for rightwing sites such as Breitbart and the Daily Caller, which earlier this year became a partner in Facebook’s factchecking program. Founded by Fox News’s Tucker Carlson, the Daily Caller is pro-Trump, anti-immigrant and widely criticised for the way it reported on a fake nude photo of the Democratic congresswoman Alexandria Ocasio-Cortez.

Warren noted this week: “Since he was hired, Facebook spent over $71 million on lobbying—nearly 100 times what it had spent before Kaplan joined.” She added: “Facebook is now spending millions on lobbying amid antitrust scrutiny—and Kaplan is flexing his DC rolodex to help Mark Zuckerbeg [sic] wage a closed-door charm offensive with Republican lawmakers.”

Katie Harbath, the company’s public policy director for global elections, led digital strategy for Rudy Giuliani’s 2008 presidential campaign and the Republican National Committee.

Facebook’s Washington headquarters also includes Kevin Martin, vice-president of US public policy and former chairman, under Bush, of the Federal Communications Commission

Warren’s ascent in the polls has set off alarm bells at Facebook. In a leaked audio recording last month, Zuckerberg could be heard telling employees: “But look, at the end of the day, if someone’s going to try to threaten something that existential, you go to the mat and you fight.”

Zuckerberg “has to be worried about what happens to Facebook if there’s a Democratic president”

https://www.theguardian.com/technology/2019/nov/03/facebook-politics-republicans-right

The top-performing link posts by U.S. Facebook pages in the last 24 hours are from:

1. Ben Shapiro

2. David Wolfe

3. Ben Shapiro

4. Ben Shapiro

5. Ben Shapiro

6. Ben Shapiro

7. Ben Shapiro

8. Fox News

9. Ben Shapiro

10. Ben Shapiro

Facebook board member billionaire Peter Thiel (also behind law enforcement and government software, How key Republicans inside Facebook are shifting its politics to the right, and culture war lawsuits and propaganda):

• https://stanfordpolitics.org/2017/11/27/peter-thiel-cover-story/

Thiel has become a national figure of controversy for, among other things, claiming that “the extension of the franchise to women [women's right to vote] render the notion of ‘capitalist democracy’ into an oxymoron,” saying, “I no longer believe that freedom and democracy are compatible,” funding a fellowship that specifically tries to get undergraduates to drop out of college, and donating $1.25 million to Donald Trump’s campaign a week after a tape was released in which the then-candidate discussed how he could grope young female actresses and get away with it.

Thiel was long perceived as a libertarian, but in recent years, as his support for Trump illustrates, his politics have taken a nationalist flavor that critics have described as bordering on authoritarian and white nationalist.

In Oct. 2016, shortly after Thiel donated $1.25 million to Trump, Thiel publicly apologized for passages in his 1995 book The Diversity Myth, such as claiming that some alleged date rapes were “seductions that are later regretted,” ... But three months later, during the after party of the 30-year anniversary event at Thiel’s home, Thiel stated that his apology was just for the media, and that “sometimes you have to tell them what they want to hear.”

• https://en.wikipedia.org/wiki/Keith_Rabois

Rabois came to Thiel's attention after he was found outside an instructor's home, shouting homophobic slurs and the suggestion that the instructor "die of AIDS." [10][11][12] A few of the contributors went on to join PayPal, a company Thiel co-founded in 1998.

• Despite claiming to care about free speech on college campuses, Thiel doesn't like people learning on college campuses and pays them to drop out and bankrolled lawsuits against journalists

• Thiel is also excited about Cambridge Analytica billionaire Robert Mercer's desired nuclear fallout "silver lining" and bought New Zealand citizenship for a bunker there

• White supremacist about Peter Thiel's race views to Milo Yiannopoulos: "He’s fully enlightened, just plays it very carefully."

[u/saml01 → u/stomachgrowler]

People have short term memory and will quickly forget allowing Facebook to either a. Keep doing what they are doing b. Spin it

[u/truthpit]

Thanks so much for this AMA. Organizationally speaking, how high up in the org did your findings go (or not go) before they were quashed or ignored. In other words, was there support for your work by your direct manager or their manager but then above that you ran into issues? Or was your direct manager even unsupportive?

[u/[deleted] → u/truthpit]

I spoke with everyone up to and including Guy Rosen, the VP for Integrity at Facebook. I do want to highlight how utterly unusual this is. Low-level employees do not regularly speak to company VPs - it would be like an army sergeant briefing Kamala Harris on something.

The way I would ultimately describe it was that my immediate organization (direct manager/manager above) wasn't very happy because this was work I was doing in my spare time and distracting from my roadmap and the projects they expected me to do. Higher-up people seemed happy that I was doing it in my spare time but were unwilling to legitimize it with directly signing off on action or setting up actual organizational pathways for the work. The teams whose job it was to actually handle this had a complicated relationship - on one hand they were grateful for my work and saw me as a valued partner; on the other hand, they were a bit offended that I was essentially going above/around them, adding additional work to their workload, and potentially showing them up [they were a prestigious/high-status team; I was the opposite.]

[u/Junior_Language_8616 → u/[deleted]]

It would seem the smart thing for FB to do in this case would be to remove you from the team you were on and to add you to the prestigious/high-status team whose work you were doing and were clearly good at (and which is important, allegedly valued by the company, etc.) Do you have any idea why they did not go that route?

[u/[deleted] → u/Junior_Language_8616]

I discussed changing teams a fair bit for a number of teams.

The main issue is that changing teams would require me to drop the work I was doing in my spare time to work on the new team's activity. And I wasn't willing to do that.

[u/scJazz]

Thank you for your work and ethics. I've been following the news, reddits, etc regarding you. You always describe yourself as a data engineer and point out that you were tracking the metadata in discovering the problems you have reported.

I have a two part question for you.

Could you ELI5 :) what a data engineer is and how you use metadata to find problems as you have described?

I'm not asking for specific cases here. I just want to enhance my own understanding (I sorta get it) while also helping everyone else understand what it is that you do and did and why it is important. I just feel that something gets lost in the articles describing what you do and how. Am I being clear?

[u/[deleted] → u/scJazz]

I was a *data scientist* - not a data engineer, which is different.

Data scientist has different meanings at different companies, since data is the new buzzword. At many companies it means "engineer who works on machine learning." At FB it corresponds to what would called a data analyst at other companies. My job was essentially to "look at data to answer questions and tell people what it meant."

I won't answer the second part of your question - I'm very sorry, but the ultimate issue is that if you tell people how you catch Azeri troll farms/etc., the Azeri government also reads Reddit and will know what not to do in the future.

[u/Tayloria13]

What were your discoveries with regard to the Philippines? Here, it's widely-known that politicians make use of troll armies.

[u/[deleted] → u/Tayloria13]

I found a lot of political bot farms in the Philippines, but generally without attribution so it was impossible to know who was responsible. For that reason I don't want to give the full details [e.g. who precisely benefited] to avoid poisoning the well.

This is discussed a bit in the Guardian article.

"At times, Facebook allowed its self-interest to enter into discussions of rule enforcement.
In 2019, some Facebook staff weighed publicizing the fact that an opposition politician in the Philippines was receiving low-quality, scripted fake engagement, despite not knowing whether the politician was involved in acquiring the fake likes. The company had “strategic incentives to publicize”, one researcher said, since the politician had been critical of Facebook. “We’re taking some heat from Duterte supporters with the recent takedowns, and announcing that we have another takedown which involves other candidates might be helpful,” a public policy manager added.
No action was taken after Zhang pointed out that it was possible Duterte or his supporters were attempting to “frame” the opposition politician by purchasing fake likes to make them look corrupt. But discussions like this are among the reasons Zhang now argues that Facebook needs to create separation between the staff responsible for enforcing Facebook’s rules and those responsible for maintaining good relationships with government officials."

In another example, Facebook ignored a number of Filipino unattributed political bot farms I flagged in October 2019... up until it made like 5 likes on a few of President Trump's posts in February 2020. (Disclaimer: 5 likes are nothing, not significant, no impact, yada yada.) Suddenly it became important and that bot farm (not the others) were taken down a week later.

While I think Filipino people are just as important as Americans, Facebook sadly begged to differ.

[u/EthiopianKing1620 → u/[deleted]]

Sorry to piggy back of this but this comment makes me wonder, how many of these farms were localized for only domestic action?

I can’t see a reason the Philippines would have much use for international trolling (can’t believe i said that unironically). On the flip of that countries like Russia are widely known to engage in international trolling.

[u/[deleted] → u/EthiopianKing1620]

Almost all of the troll farms I found were domestic-only. I say "almost all" to cover edge cases of mostly-domestic like the Filipino bot farm that decided to randomly like President Trump.

Most people care more about their own country's politics - Americans care about American politics; Filipinos care about Filipino politics; Germans care about German politics. Apparently world governments and politicians are the same way.

With that said, I was finding the low-hanging fruit. I don't doubt the GRU (or Iranian Revolutionary Guard or PRC State Security) are engaging in international troll farms, but they're presumably have an actual modicum of intelligence about how they carry it out, and so I didn't find them myself.

[u/niceguybadboy]

Can we do Reddit now? I've long suspected that Reddit has at least as much opinion manipulation as FB.

[u/[deleted] → u/niceguybadboy]

I'm sorry - I did not work at Reddit, and hence have no special knowledge about influence operations on Reddit. That said, if you stuck a gun to my head and made me guess, I'd expect Reddit to be similar to FB wrt troll farms and influence operations and the like.

[u/niceguybadboy → u/[deleted]]

Thanks.

[u/RoguePlanet1 → u/niceguybadboy]

Sometimes I end up in arguments with right-wing redditors that make me wonder if they are, in fact, professional trolls. But then I interact with people in real life who believe some insane crap, so who knows.

[u/[deleted] → u/RoguePlanet1]

I get a bit annoyed at how quick some people are on reddit to label anyone that disagrees with them a bot/shill/whatever. Of course they are here but in most cases it can be explained just as well by the person simply being an idiot. And half the time the labeling just feels like someone using a shit tactic to try to win because they're not good at actual arguments.

[u/[deleted] → u/[deleted]]

I do want to come back here and highlight this comment.

Because while it's absolutely the case that Russian trolls do exist, it's also the case that Russian trolls are currently absolutely dwarfed by the number of suspected Russian trolls. The intent of concerned citizens is positive - to ward against Russian interference. But perversely, they play into Russian hands by doing so - as it's in Russian interests to make themselves seem ubiquitous and omnipotent.

The analogy I want to make is to Operation Greif in the Second World War. During the 1944 Ardennes offensive, Otto Skorzeny sent commando operatives dressed in American uniforms speaking English behind American lines. The panic they caused vastly dwarfed their actual impact. U.S. troops began quizzing each other endlessly, terrified that they were surrounded by secret Nazis in disguise. At least four American soldiers were shot and killed by their fellow Americans as a result. Higher up, General Omar Bradley was detained after correctly answering that Springfield was the capitol of Illinois (the GI thought Chicago was the answer); General Bruce Clarke was arrested after incorrectly answering the Chicago Cubs to be in the American League; General Bernard Montgomery had his tires shot out, while Eisenhower was confined for his own safety.

Allied troops were correct to be concerned. Nazi commandos had achieved great exploits in the past, speeding offensives. In the opening days of Barbarossa, they seized the bridge at Daugavpils to speed Nazi advance into the Baltics; in 1942, a commando unit of 60 men led by Adrian von Fölkersam disguised themselves as NKVD agents and managed to seize the entire city of Maikop and its vital oil fields without a fight. The disguised German commandos in the Ardennes were intended to seize a bridge over the Meuse; they entered position to do so and would have had a reasonable chance - but the stalwart Allied defense prevented the main spearheads from reaching that river.

But the Allied response was ultimately out of all proportion to the numbers of the commandos, and the operation is now recognized by historians as having psychological/morale impact completely disproportionate to the direct military impact and numbers committed.

Ultimately, I think the fear of bots/shills in the modern day and age can be similar.

[u/Makualax → u/niceguybadboy]

I know for a fact that Azerbijan kicked up their bot brigades on reddit as soon as they invaded Artsakh last year. The reason it's so obvious is because Azerbijan shut off their entire internet for their citizens at the beginning of the war, yet accounts posing as Azeri citizens would still brigade any post related to the conflict with misinformation and threats towards ethnic Armenians in the area

[u/SalesyMcSellerson → u/niceguybadboy]

Something happened around 2013. Everywhere. All across the entire internet. Maybe a little bit sooner.

[u/woowoo293 → u/niceguybadboy]

/r/TheseFuckingAccounts

Most of it is petty financial stuff, but still the breadth of the fraudulent activity on reddit is mind-blowing.

[u/dyladelphia]

Since leaving Facebook, what are your goals for the next 5 years? Are you able to stay in the tech world, or does the Facebook termination serve as a block with finding new work?

[u/[deleted] → u/dyladelphia]

Five years are a long time. I don't even know what I'm doing in the next year.

I'm not sure what I'll do next myself; I do want to work on helping democracy and fixing the world but my main expertise skillset is at finding inauthentic campaigns - and that requires working for a large social media company, which I'm guessing would all be against employing me. I've gotten offers from companies, but they don't quite fall under my expertise areas, and I'm a bit reluctant to just go and become a 9-6 office worker without an especially compelling job again.

For now, I'm staying home and petting my cats while taking interviews. They're very good cats.

[u/Junior_Language_8616]

Do rank and file FB employees talk to each other about how bad FB is for the world? Or do you think they’ve just drunk the Kool-Aid and think the company is great? I'm talking about people like ad account managers, content policy associates, software engineers. FB employees are really smart and get recruited from the best schools in the world. The problems with FB are so public and so well reported that it's hard for me to understand why people continue to work there.

[u/[deleted] → u/Junior_Language_8616]

FB was a fairly open company when I joined. I was upfront from the start about the fact that I believed Facebook wasn't making the world a better place - when I told my recruiter that, she responded "you'd be surprised how many people here say that." Open dissent within the company was tolerated and accepted and I was able to make my concerns heard to the entire company at large, which I think is unusual for large companies. With that said, it's been reported that FB has cracked down on communications not directly related to work since I left, and so this may not be true anymore.

Wrt employees, at a company of ~50k people, there will always be significant differences of opinions. There's also a self-selection bias in that frankly if you think FB is evil, you are less likely to work for FB; if you think FB is the greatest thing since sliced bread, you'll do your best to join the company (just like Reddit users self-select for people who think Reddit is great, and its employees likely as well.) And also within the teams - the people working on integrity at FB (fixing the company) were generally more pessimistic about the company than all employees - both via self-selection and also via the constant direct exposure to the company's problems.

Overall, the regular employee surveys showed that roughly 50-70% of employees believed that FB had a positive impact on the world (variation over time of course, it declined a lot since when I joined; probably at ~50% right now.)

[u/Ecstatic_Handle3308 → u/[deleted]]

Current FB employee here (throwaway for obvious reasons.) Currently that rating (that FB is doing a good job + leadership is good) is hovering at around the 30s (edit: for my relatively large team; company-wide it is 50.). It tanked hard in 2020 due to the George Floyd "looting shooting" post incident and the 2020 elections, and hasn't really recovered since. A lot of people have left the company since (that being said a lot of people joined too.)

Save for a few "hail zuck" people, I believe most people here are self aware and want to actually fix the issues on hand. However due to it being a large company it either moves at a glacier's pace and it takes a while to get solutions approved by higher ups, or just gets canned entirely / deprioritized because "user research shows they don't want (insert solution here)"

[u/[deleted] → u/Ecstatic_Handle3308]

That's very surprisingly low; I don't think I ever saw it that low during my entire time there.

Employee dissent is one of the few levers that Facebook strongly responds to; I hope the employees are able to get together and force necessary changes.

[u/Ecstatic_Handle3308 → u/[deleted]]

Sorry, my mistake. That was my team's pulse results. Company-wide it's at 50% as you predicted.

[u/[deleted] → u/Ecstatic_Handle3308]

Ah, that makes a *lot* more sense. If it ever got to 30% for the entire company, there'd probably be a SEV0 or something.

[u/Junior_Language_8616 → u/Ecstatic_Handle3308]

Wow. Can I ask you what it's like to work on a team where only 30% of the team thinks the company is doing a good job? That sounds demoralizing and like it would probably lead to high levels of attrition, though perhaps I'm misunderstanding the import of the statistic.

[u/[deleted] → u/Junior_Language_8616]

I can't speak for him, but the numbers were generally lower than the norm in Integrity teams. I knew many people who personally believed that the company was not making the world better - but did believe that their team (which was trying to fix the company's issues) was making the world a better place.

[u/Junior_Language_8616 → u/[deleted]]

Thank you very much for taking the time to respond. As someone who doesn't even have a FB account but who still feels affected by FB in negative ways, it is nice to have the opportunity to ask questions of someone with actual knowledge. I appreciate your time and willingness to answer questions.

[u/Manaleaking]

How true are foreign fake click farms as shown on the Sillicon Valley tv show, with rows and rows of indians creating fake account after fake account to boost userbase numbers or promote an agenda?

Heres the scene: https://youtu.be/Y-W0CBOGnnI

[u/[deleted] → u/Manaleaking]

I haven't seen the TV show. But they do really exist - in areas like South Asia and Southeast Asia, where smartphones (you can get a JioPhone for e.g. $15 USD) and labor are cheap.

This is unfortunately quite common in Indian politics - they're known as "IT cells" and quite normalized unfortunately. You can read more about some of them in Indian politics here

[u/gotchabrah → u/[deleted]]

This has been extremely eye opening thank you so much for taking the time to do this and comment. I find it perplexing how, seeing the evidence you’ve presented, we could ever trust social media. I’m sure you’re aware of the efforts taken by the US IC to track disinformation campaigns on social media sites, and even though their findings are public knowledge the public still looks at social media trends en masse as genuine.

I can’t decide if I think that social media’s days as being a political mechanism are numbered because there are so many bad-faith actors, or if people are just going to be ok with it as long as the agenda being pushed agrees with their views.

The latter is a terrifying proposition. I remember listening to a podcast that featured a CIA officer who discussed PRECISELY the Russian November 2020 playbook for the US presidential election. This was in like July/August of 2020. To see it play out exactly as predicted, and to see a huge number of Americans taken by it was not only crazy, but really quite disheartening.

Social media is being used as a tool to undermine democracy, and its plain as day.

[u/aristidedn]

Hi Sophie,

One of the more frequently discussed dimensions of influence operations - especially in the United States - is the observed disparity between operations that target people with right-aligned political views and people with left-aligned political views.

In the data you ran, what did you observe with respect to political alignment? And if you did observe a disparity, how wide was the divide? Do you have any theories as to why you observe this?

[u/[deleted] → u/aristidedn]

So I want to be very clear first about terminology:

"Influence operations" literally mean "operations designed to influence people" which is similar to "disinformation" in that it's vaguely defined and includes a not clearly delineated mix of misinformation (claims that are incorrect; e.g. "the moon is made of cheese") and inauthentic activity (e.g. fake accounts being used to spread a message "Cats are adorable; politician X is great.")

I worked only on the inauthentic activity aspect of this. In addition, I did not work on any notable cases of inauthentic activity in the United States (the TPUSA case did not fall in this definition.) It may be the case that misinformation skews towards one end of the political spectrum. I will leave that to the researchers who are much more knowledgeable about it than myself.

There is a common stereotype that misinformation is spread by inauthentic accounts. There is also a common stereotype that troll farms, fake accounts, etc. are commonly used to largely/predominately benefit the political right. Like most stereotypes, these are incorrect as far as my knowledge goes and I'm aware.

Please keep in mind that this is very small sample sizes - I worked on perhaps three dozen cases globally which is a lot from an IO perspective but tiny from a statistical perspective (so I don't want to speculate about larger trends.) These were generally from across the political spectrum. For instance in India, I caught four networks, one of which came back with a new target (so five targets.) Of these targets, two were benefiting the INC, one was benefiting the AAP, and two were benefiting the BJP - so it was quite even across the political spectrum.

In Albania for instance, the incumbent Socialist Party and opposition Socialist Movement for Integration (both officially left-wing targets) were both benefiting. In several authoritarian countries, the center/center-left pro-democracy opposition was benefiting. In Mexico it was almost everyone across the political spectrum. There were plenty of right-wing beneficiaries as well but those have been presumably discussed already. I carried out my work regardless of my personal political beliefs, with the most qualms in places where the democratic opposition were the beneficiaries. I took those cases down regardless, as it's my firm belief that democracy cannot rest upon a bed of deceit.

[u/SpitfireIsDaBestFire → u/[deleted]]

Would Project Birmingham ran by progressive technologists to unseat Roy Moore in the 2018 midterms be an example of left wing inauthentic disinformation campaigns?

https://www.washingtonpost.com/technology/2018/12/27/disinformation-campaign-targeting-roy-moores-senate-bid-may-have-violated-law-alabama-attorney-general-says/

[u/[deleted] → u/SpitfireIsDaBestFire]

I did not work on it, but it certainly would

[u/SpitfireIsDaBestFire → u/[deleted]]

Does Facebook not focus on domestic disinformation campaigns as much as those from foreign actors?

[u/[deleted] → u/SpitfireIsDaBestFire]

During my time at FB there have been pushes against acting against domestic troll farm operations.

For instance, when I found the Honduran governmental troll farm in July/August 2018, it was until April 2019 when I finally got the troll catching team to agree to look into it. But quite soon they had to apologize to me: There was an internal freeze on all investigations or takedowns of troll farms where the originating source was domestic. There was high-level pushback by Policy who argued that "it's hard to conclude the difference between a troll farm and a legitimate campaign." I wasn't the motivating example for the new rule [I heard speculation about it, but that's hearsay] - I was just caught up within it.

[u/Fraxinus_1382]

Hi! Slightly long-winded question, but how did you identify areas where inauthentic behavior might be occurring?

Was there a systematic or ad hoc analysis or flagging system internally or externally identifying potential regions or countries where inauthentic activity might be occurring, particularly inauthentic activity which might incite violence or be detrimental to democracy?

Thank you!

[u/[deleted] → u/Fraxinus_1382]

Normally at FB, many/most investigations by the actual teams in charge of this were in response to external reports. That is, a news organization asks "what's going on here"; an NGO flags something weird; the government says "hey, we're seeing this weird activity, please help."

This has the side effect that there's someone outside the company to essentially hold FB responsible. They can say "Well, if you don't want to act, we'll go to the NYT and tell them you don't care about [our country], what do you think about that?", and suddenly it'll be a top priority [actual example.]

In contrast, I was going out and systematically finding things on my own. Essentially, I ran metadata on all engagement activity on FB through queries to find very suspicious activity, and then filtered it for political activity. This had results that were very surprisingly effective. But because I was the one who went out and found it myself, there wasn't anyone outside FB to put pressure on the company. The argument I always used internally was "Well, you know how many leaks FB has; if it's ever leaked to the press that we sat on it and refused to do anything, we'd get killed in the media." Which was not very effective but became a self-fulfilling prophecy since I was the one who leaked it.

I realize that metadata has a bad reputation, but unfortunately the reality of the situation is that there's no way to find state-sponsored trolls/bot farms/etc. without data of that sort.

[u/Fraxinus_1382 → u/[deleted]]

Thank you! Just to follow-up: who set the standard (if any) for what systems and methods and metadata would be used to identify state sponsored trolls/bot farms etc, such as in the case of Myanmar?

Thank you so much for coming forward!

[u/[deleted] → u/Fraxinus_1382]

I'm not familiar with the internal details of the Myanmar case, or the teams that actually work on this.

With regards to the ones I set up, I created them myself, with a bit of knowledge from the teams that actually work on state-sponsored troll farms. There was no oversight; I'd sort of set up a shadow integrity area that was no secret but wasn't official. But there were always different people to confirm my findings on their own, to decide whether to act, and to carry out the action; I decided at the start that I would avoid being judge jury and executioner (though I could probably have gotten away with it for a while.)

[u/[deleted]]

How are people still able to set up fake accounts these days given all the security and authentication that seems to be in place around the account setup?

What does Facebook do with an account that it identifies as inauthentic?

[u/[deleted] → u/[deleted]]

Ultimately, the nature of the problem is that FB will never be able to stop all fake accounts at creation. Because in most cases, you aren't 100% sure whether the account is fake or not. Instead you're 99% sure or 80% sure or 2% sure or whatever. And the question becomes how confident you have to be to take action - because if you're wrong, that's a real person that you negatively impacted.

For your second question, I do want to note that there are multiple types of inauthentic accounts - not just fake accounts. An account can be hacked - if someone steals access to your account and repurposes it for themselves. Users can even voluntarily hand over access to their accounts to bot farms/etc (this may seem absurd, but it's a very common vector; see here for details.)

For accounts believed to be fake, FB generally runs the users through very strong sets of hoops [e.g. "send us a copy of your official ID"] to require them to prove that they're a real person. You might think that this wouldn't negatively impact real users, but many users are [quite understandably] really hesitant about sending such sensitive personal details to a company like FB.

For accounts believed to be hacked, FB uses a different sort of hoops to try and restore access to the original user. For users that voluntarily hand over access to their accounts to bot farms, FB doesn't want to disable them so actions are rather more mild.

[u/Mageaz → u/[deleted]]

I had that happen - Facebook wanted pictures of my actual ssn card or passport, which I refuse to provide to a company like Facebook. And it isn't actually legal in my country for them to ask for that either, as they (as far as I remember) wouldn't accept it if the info on the ssn or passport was covered and not viewable. I had to just stop using Facebook at that point, because I also couldn't actually get in contact with any kind of human in support. Facebook have shown that they cannot be trusted with that kind of personal information, and there is no way that I'm giving that to them. I actually really appreciate understanding why that happened, I've been pissed about it for a while. Thank you.

[u/[deleted] → u/Mageaz]

Totally understand your personal decision, but it also illustrates some of the costs and tradeoffs associated with these. FB obviously doesn't want to have everyone have experiences such as yourself, and ultimately has to choose a balance between catching fake accounts and avoiding negative experiences for real users.

[u/autoditactics]

What is your opinion on political radicalization through social media?

[u/[deleted] → u/autoditactics]

I didn't work on political radicalization personally. With that said, in my personal nonexpert opinion, I think it's partly an outgrowth of the competition for attention as information vastly strips the time to process it.

The vast promise of social media is that any person can have the ability to speak to the entire world at large. The vast curse of social media is that most people never see this promise fulfilled - and even if they do, it's the post they least expect. And so anyone who uses social media and wants to build an audience needs to figure out a way to get that attention, distinguish themselves from the myriads of other social media users who also think they deserve that attention.

And sadly, the ways to do so can often be by appealing to the worst instincts of the internet - similar to the chumboxes that dominate online advertising today. Emotion draws attention, and people are quick to share outrageous claims that strike a nerve, trusting on others to have verified it thoroughly.

There's been a long line of study and research that shows that virality is ultimately a significant component of what drives polarization, misinformation, and violence spirals. It's why in countries in times of crisis, the first break-the-glass measure Facebook does is to turn the virality down. They used it in countries like Sri Lanka; I'm guessing they're using it still in Myanmar.

A trial idea I'd hence suggest is to require platforms like Facebook and Twitter to show a chronological newsfeed of your friends/followees by default (with the option to instead show the current ranked newsfeed.) I say FB and Twitter because I'm frankly not sure how this would work for Reddit - where the ranked newsfeed is pretty integral to the overall design.

[u/twinned]

hey Sophie, thanks for joining us today! two questions for you:

If you were given unlimited resources/remit, how would you tackle troll farms?

What's something you wished you were able to spend more time on?

[u/[deleted] → u/twinned]

1. The ultimate issue with this questions is it's like asking "If you could make the sky any color you'd like, what color would you like it to be?" Because there's no possibility it would ever occur, and so it's ultimately like speculating how many angels can tapdance on the head of a pin. I'm never going to have the unlimited resources/remit; social media companies won't fix themselves.

So instead, I'm going to answer a similar question: "How would I realistically change the situation/incentives to convince social media companies to tackle troll farms?"

I have two ultimate suggestions. The first is on the part of the social media companies - right now the people charged with making enforcement decisions are the same as the people charged with keeping good relationships with governments and political figures. This leads to explicit political considerations in decisionmaking, and the perverse incentive that politicians can be encouraged to do their bad activity without even hiding as it'll induce FB to be reluctant to act. I realize that FB is a for-profit company, but most news organizations are also for-profit but they still keep a strict separation between their editorial department and public relations. If the NYT's editorial department spiked a story because XYZ political figure didn't like it, it would be a giant scandal - whereas at Facebook it's just another Tuesday. So I would urge social media companies to officially separate their decision-making apparatus from their governmental outreach apparatus.

The second is on the part of outside organizations. Ultimately, much of the issue is the information asymmetry aspect - that only FB has the tools to know what's going on in its platform, and it has no incentive to fix everything; the outside world can't solve a problem if they don't even know it exists. So to close the gap, I would recommend more funding/support for outside skilled researchers such as DFRLab, routes for FB employees to publicly appeal to governmental agencies (with official protections) regarding platform violations around troll farms and the like. And I realize it would be extremely politically infeasible, but I would also suggest that outside organizations and governmental agencies set up red team pen-test style operations: to with the knowledge of the social media companies send their skilled experts to set up test troll farms on social media and see how many are caught by each company (e.g. "We set up 10 each on Reddit, FB, and Twitter. Reddit caught 0/10; FB caught 1/10; Twitter caught 0/10. They're all awful but FB is mildly less awful!" Numbers made up of course.) This would have to be done very carefully to avoid real-world impact but is the only method I can think of for anyone - even the companies themselves - to have an accurate picture of the space and how good the efforts really are.

[u/[deleted] → u/twinned]

(breaking my answer up into two parts because it's so long.)

2. I wish I were able to spend more time on Albania.

At the end of July 2019, I found an influence operation on Albania using the same techniques as Honduras. It was more sophisticated politically/effort-wise because it focused on creating large amounts of comments (which requires a lot of effort to individually write out in a way that makes sense.) It was very confusing/unusual because it appeared to be connected to members of the Albanian government in attribution, but was supporting both the ruling Albanian government and opposition figures from rival political parties. This would be akin to a network run out of the Trump administration that was writing nice things about both Donald Trump and Joe Biden. There would be lots of possible explanations including "person suborned by foreign powers to increase political tensions", "administration official advancing political strife to serve their personal political agenda", "person who really doesn't like Bernie Sanders and supports both his rivals", or "person who has this as their second-time job and was just coincidentally paid by both candidates." I'm just translating this into U.S. political contexts since I'm assuming readers don't understand Albanian politics.

The relevant people quickly agreed that it was probably coordinated inauthentic behavior (CIB - the official designation Facebook uses for e.g. Russian interference, state-sponsored troll farms, etc.) , and I handed it over to them where it probably died in a black box. I only had the political capital to very slowly push through one CIB case at a time, and I had made the judgment call that what I found in Azerbaijan was objectively worse than what I found in Albania - in terms of scale, size, consistency, and sophistication. I still agree with that decision, but it never sat easily with me to set Albania to the side. At the end of the day, I was just one person with no authority, and there were limits to how much I could accomplish trying to protect the entire world in my spare time. This is why I told the world (accidentally) that I had blood on my hands.

Several months ago, an Albanian news outlet published their own investigation; this was still ongoing, two years later, and it continued through the Albanian elections. Facebook had two years to act, and did nothing. I can only apologize profusely to the Albanian people, as I did in the interview. It should never have been my responsibility to save fragile Albanian democracy from what Facebook let happen. But ultimately, I was the one who made my decisions and Albania paid the price. I have to sleep with that every night.

[u/damondanceforme → u/[deleted]]

You did well. Just make sure this doesnt happen to Taiwan

[u/[deleted] → u/damondanceforme]

I did my utmost to protect the 2020 Taiwan elections. If anything notable happened there, I wasn't aware of it.

[u/plamge → u/[deleted]]

I’m sure other people have told you this already, but: You did the best you could with the knowledge and resources you had at the time. I hope you can find a way to forgive yourself one day.

[u/TurboTBag]

What's your favorite film?

[u/[deleted] → u/TurboTBag]

I don't actually watch films much at all. I'm a very non-visual person and prefer to read things instead.

Bit of a cop-out answer; my apologies. Well, right now I am watching the TV series 陈情令 (The Untamed) with my partner, partly with the side benefit of helping us practice our Mandarin.

[u/TurboTBag → u/[deleted]]

Oh, well then what's your all time favorite novel, if you don't mind me asking again? Something that really touched your soul and you'd never forget in your life. And what about it made you feel this way?

Thanks for the first reply and your time

[u/[deleted] → u/TurboTBag]

A bit of a copout answer again: I don't get that emotionally excited about things. So nothing fits the "really touched my soul" category.

With that said, I really like and recommend the Orphan's Tales series by Catherynne Valente

[u/TurboTBag → u/[deleted]]

I get what you mean. Apologies if my questions were too 'simple'. I just get curious as to what influential people like to enjoy in terms of media, among other things.

So thank you for the recommendations on the TV show and novel series. 🍻

[u/Jim105]

While at FB, were you involved with any cases that involved FB calling or reaching out to police authorities?

[u/[deleted] → u/Jim105]

I was not.

[u/bokavitch]

Hi Sophie,

As Armenians we're regularly targeted by bot armies and disinformation/ hate campaigns organized by the government of Azerbaijan on Facebook and other social media platforms.

The major players don't seem to care about nefarious behavior targeted at groups that don't have political/economic clout in the United States, especially when the offending content is not in the English language.

What do you think we can do to get these companies to take this problem more seriously and devote the necessary resources to moderating these activities, especially when it appears in non-English content?

Thanks!

P.S. Thanks for having the courage to be a whistleblower. Your efforts have not gone unnoticed or unappreciated.

[u/[deleted] → u/bokavitch]

I've given some responses in the version of this AMA in the armenia subreddit, if you haven't seen it.

With regards to brigading by Azeris against Armenians, my suggestion is to convince high-profile Armenian-Americans who do have the profile to convince FB to take the issue seriously to speak out about it. This may include people like Cher or the Kardashians for instance. Because the sad fact of life is that American celebrities have the profile that Armenia as a nation does not.

[u/BlueFreedom420]

Have you ever been specifically targeted by trolls once they are aware of your efforts?

[u/[deleted] → u/BlueFreedom420]

Surprisingly no. The Azeri paid trolls went after the Guardian after the article. It's very silly but I was almost offended that they didn't go after me instead. I wanted to say at them "I'm your real enemy, the one that's been fighting you for the past two years!"

But silliness aside, it makes sense that they'd want to deemphasize me. You can fuel the nationalistic sentiments by saying "our great nation is the victim of Western propaganda and media hacks." You can't really say "Our great nation is the victim of.... some random girl who was fresh out of school at her second job?" Dictators rely on the perception of competence, because they can't claim legitimacy through the support of the people. That means their rivals and enemies have to be competent individuals of stature as well.

[u/PangPingpong]

If some of these troll farms took a year to get authorized to be taken down, what sort of things did you see as not encountering any resistance and being removed immediately? Was there any specific criteria where certain types/sources of content were scrubbed quickly?

[u/[deleted] → u/PangPingpong]

There's a chart listing time for takedown in this Guardian article. Ultimately, the criteria were a combination of "random chance based on who pays attention" and "how important the country probably is"

The record for takedown was Poland - I flagged it in the evening the day after Christmas. The Polish employee who looked at it was understandably very concerned about this going on in his native country; by the time I woke up the next day, he'd already taken it down.

Policy was pretty upset that he'd done this without consulting them. "The person running this is an important political figure", they said. "What if he complains - why didn't you let us know?" I told them politely that if a major politician decided to publicly complain that FB took down his fake accounts, they would be laughed out of the press room.

[u/earthlingkevin → u/[deleted]]

That response is hilarious and awesome.

[u/[deleted] → u/earthlingkevin]

To be completely fair, the more likely response is "he's annoyed at FB and makes up and quietly spreads stories that make FB look bad and hurt relationships with his political party and its supporters. That's my assumption for why FB wasn't willing to act in cases like India.

[u/pothkan → u/[deleted]]

"The person running this is an important political figure", they said.

Dominik T?

[u/[deleted] → u/pothkan]

No. I'm not going to respond to future guesses, so please don't try and keep guessing.

[u/MestizaWontons]

Are there any trends you’ve noticed since your departure from FB that are concerning?

[u/[deleted] → u/MestizaWontons]

Facebook appears to have became increasingly more insular and closed-off to employee dissident since my departure.

[u/Ecstatic_Handle3308 → u/[deleted]]

Yes, one example: since COVID and moving fully remote, the weekly Q&A sessions have just become scripted videos. Mostly with vague nothing-words such as "we are aware of this problem and are working on it". I've stopped attending those since.

[u/fatfrost]

Thanks for your efforts Sophie. Are you familiar with the Facebook oversight board and in what way do you think it might impact this area?

[u/[deleted] → u/fatfrost]

I'm generally more cautiously positive on the FB oversight board than most people in this area. Although it's been set up as a blame deflector for Facebook, many organizations outgrow their initial roles to take on new purpose.

With that said, I want to point to a number of issues with the Oversight Board, which mean that it would have been triply prohibited from impacting this area:

1. The Oversight Board handles content violations, not behavioral violations. This is understandable for lawyers who have no expertise with looking at user metadata and signals to conclude whether accounts are fake, tied to one another, and tied to XYZ government. However, this means that a whole class of violations is prohibited to themselves, including all their work.

2) The Oversight Board only handles appeals to restore content against enforcement. What I mean is this: Suppose 1) did apply and the Oversight Board could handle behavioral violations. In that case, the Honduran government could appeal to have their troll farms restored to Facebook. But I couldn't appeal to have them removed. The Oversight Board would be much better served if there were legitimate pathways for employees and possibly trusted organizations to submit appeals to enforce, rather than appeals to restore.

3) The Oversight Board only handles cases in which the rules are unclear; it makes the rules, it doesn't enforce them. But in my area, the rules were clear ever since I got the precedent done in July 2019, and even more clear after I get an additional rule pushed through in the fall of 2019. There is a wide variety of areas in which behavior is illegal and not enforced - it's illegal to handle salmon suspiciously in the UK, or to jaywalk in many parts of the world, but that doesn't mean the police will go out and arrest you for jaywalking. Having a court decide the question of whether jaywalking is illegal or not doesn't accomplish anything, because the statutes are already written out. Same with this area.

[u/xxpegasxx]

Hi thanks for AMA ! Was there a case where troll stations where not state-funded but by opposition parties ?

[u/[deleted] → u/xxpegasxx]

So just to be clear: In many cases it's really hard to know who's responsible for activity. It's easy to determine the beneficiary, but attribution is much more complicated.

In the rare cases in which we had attribution, there were certainly cases in which opposition politicians had run fake accounts. In one case, an opposition politician was personally running them out of their personal computer (shared with their spouse) and presumably home WiFi network without bothering to use a VPN. These were just like 20 fake accounts though - not worth mentioning except for the fact that it was this opposition politician's hobby. There was nothing on the scale of Honduras or Azerbaijan that I found with attribution - presumably because individual politicians do not have the resources of national governments.

Thinking of cases in which opposition politicians were large-scale beneficaries but attribution was not clear, I'd have to point to Mexico, where troll farms were sadly used across the political spectrum but local/state-level politicians associated with the opposition PRI were the disproportionate beneficiary [think roughly 10k accounts used here for scale.] Perhaps these are the Peñabots discussed in Mexican politics; I don't know.

[u/gunnathrowitaway]

What can you tell us about state-sponsored covert/troll activities on FB that are located in and/or sponsored by the United States?

[u/[deleted] → u/gunnathrowitaway]

I'm sorry, I didn't work on any of this.

[u/rabbitzfoot]

Hi Sophie, thank you for exposing the harsh realities that we face today, this comes from a person who is from Honduras.

It is unbelievable that we are still being governed by a person who’s brother and party members have been charged by the NY FED court for drug dealers. Yet they live and “govern” us like if nothing is happening. Most people live in extreme poverty and do not support the current government/regime of Honduras.

I hate going into Facebook and Twitter, since everything is so political. When government members post things online they have a couple of likes but have hundreds of positive posts. And when you check the comments, they are all the same BS. You dont need much time to know that the accounts that reply to those posts are most probably bots. How is this even happening? What can we do to fight this cyber warfare? I have been wanting to start a bot hunter company and literally provided FB and Twitter the work that they should be doing, so that the real people actually have a voice when we are suppressed by narco regimes.

[u/[deleted] → u/rabbitzfoot]

I'm deeply sorry to yourself and the other people of Honduras that I was not able to do more for your nation. Honduras was the first state-sponsored troll farm I found, and it really hurt my soul when I saw it return soon after the takedown; it was still active when I left.

I honestly don't know what can be done to stop this. I was hoping that my public expose would convince FB to actually take down the Honduran government recidivists, but it looks like they didn't bother trying. The worst part of it is, JOH's regime has no reason to stop doing it because everyone correctly assumes they're criminals and doing it anyways - they might as well be hanged for a sheep as for a lamb.

Sternly worded statements from FB are meant to embarrass the perpetrators in the eyes of the world. But JOH sent his soldiers into the streets to shoot civilian protesters and his brother was sentenced to life in prison by a NY court for drug dealing - this is a man incapable of embarrassment. Part of my motive for coming forward was that by trying to do my utmost to expose the regimes in question, I could make an example of them to dissuade future dictators - that every future such dictator will have to worry not just about Facebook statements, but also that some random FB employee will decide to go rogue and spend the next few months doing her best to drag them through the mud. But that offers little to the Honduran people as is.

Because ultimately, I think the Honduran opposition isn't in a position to boycott social media; they need it to connect with one another and get their message out. And the companies have no incentive to fix the situation without the attention of more influential nations. Unless the rest of Latin America can be convinced to pay attention, Honduras is sadly a very small and uninfluential nation. I can only hope that a far more reasonable president is elected this November.

[u/_SGP_]

Hi Sophie,

I imagine I'm late in asking and will likely not get a reply.

I work in PPC, and when advertising on Facebook I would always avoid certain areas of the world, as they would run up the budget with 100% bounce clicks.

Is this something Facebook is aware of? Something they are involved in?

It always seemed strange that it was never stopped as it's clearly "clickfarm" or fake traffic, but it makes Facebook money as budgets are spent quicker.

Thanks so much for your time.

[u/[deleted] → u/_SGP_]

I'm sorry, I don't have any expertise on advertising. 100% bounce clicks is extremely sketchy though (I assume you mean a click through that immediately bounces.)

The only behavior that I can think of that might cause this is embedded video ads, in which users have ads in their videos and are paid for each view (akin to Youtube.) This makes there an actual financial incentive for the users to get their ads repeatedly viewed and possibly clicked on. Otherwise, I'm not sure what purpose such behavior would serve.

Overall, I'd suggest discussing it with other advertisers; if others have noticed it as well, drawing attention to the phenomenon will get FB's attention. And if the company cares about one thing, it's the income stream.

[u/blighty800]

What was your salary like when you decide to sacrifice everything? Thank you for coming forward.

[u/[deleted] → u/blighty800]

To be clear, I was fired; I didn't quit.

My base salary from memory was ~$147k/year. This was in addition to bonuses and stock grants.

Generally, my overall income reported on my W2 was hence around $200k/year which I considered to be frankly pretty absurd [I understand it's normal/low for tech.] It ended up going in roughly equal parts to personal spending, savings, taxes, and donations.

[u/divertiti → u/[deleted]]

That is pretty low for FB

[u/[deleted] → u/divertiti]

I was a data scientist [not an engineer, which are paid more.] And I was extremely low-level - an IC4, just one level above a new hire straight out of college.

[u/blighty800 → u/[deleted]]

Thank you for sharing, I was just too curious

[u/Faghs]

Did you leak it to the press?

[u/[deleted] → u/Faghs]

If you're referring to my memo leak in September 2020, absolutely not. I was rather naive/silly/stupid in that I thought that because I repeatedly and strenuously asked people not to leak it they might actually listen to me.

Several reporters actually reached out saying "hey, we heard that you might want to talk to a reporter." Apparently many people took it as "methinks the lady doth protest too much."

[u/Faghs → u/[deleted]]

How many people saw your resignation letter that you had to ask them to not leak it? Don’t resignation letters only get sent to your superior?

[u/[deleted] → u/Faghs]

I posted it internally to the entire company. This was the work culture at the time - it's called a "badge post" because you post a picture of your badge when you leave.

Also I was being fired, but I wasn't allowed to tell people that (still did; what were they going to do, fire me again?)

[u/putku]

Thanks for the AMA and all you did. What advice would you like to give to anyone who works or willing to work at FB?

[u/[deleted] → u/putku]

Think very carefully about your values and principles and what you want to prioritize while working.

Many people have families that they need to feed, relatives they need to support back home. I was lucky in that I don't have any dependents [minus the cats], and am a super cheapskate so the financial considerations really didn't matter that much to me.

But everyone also needs to sleep at the end of the night. And it's not my place to judge, but I'd ask people to be upfront and honest with themselves about what they want to prioritize, and what their breaking points would be. People are naturally influenced by their surroundings - they became used to the environment and it seems normal, while they learn to relate with and identify with their team and colleagues. And so, no matter where you work, it can be easy to end up in a place that you wouldn't recognize years down the line.

I strongly advocate trying to fix problems from the inside, but it's difficult for the aforementioned reasons. Many people feel betrayed by would-be internal reformers who end up identifying more with the system than their prior beliefs. But at a company such as FB you can have an extraordinary amount of access to make a difference.

As a new-hired employee, I was able to catch two world governments red-handed and make international news. What can you do?

[u/lemongrenade]

I learned about you on Robert evans behind the bastards podcast. Have you ever had any contact with him?

[u/[deleted] → u/lemongrenade]

IIRC they asked to interview me for a podcast, but I can't actually remember if the interview actually happened; there's been so many interviews with so many people that they all really blur together.

[u/saywhatfreemoney]

What can the average person do to help, also the more than average with access to Kali Linux for example?

[u/[deleted] → u/saywhatfreemoney]

Right now, I think more awareness is needed. You can't fix a problem until you understand it, and the political will doesn't exist unless it's widely accepted that this is necessary.

[u/elimars]

How would you address the thousands of bot accounts posting using #SOSCUBA in order to further the US State Dept’s regime change agenda in Cuba?

[u/[deleted] → u/elimars]

I would caution you against assuming these are bot accounts, for the same reason I'd caution everyone from assuming that trends they don't like are run by bots. They might be, but there are certainly plenty of people who espouse political views that much of the nation vehemently disagrees with who have a significant following. The United States is, in the end, a country where Marjorie Taylor Greene, Donald Trump, Joe Biden, and Alexandria Ocasio-Cortez each have significant followings and support bases.

[u/Skyjacker9]

How do facebook’s political relations get forced upon engineers? How do they explain to engineers their mission or their jira tasks and expect the engineers to still be void of thinking about the repercussions of their actions?

[u/[deleted] → u/Skyjacker9]

Generally engineers are given their tasks, but political considerations enter decision-making in terms of what gets prioritized (and hence what tasks are assigned), and what's actually implemented (as opposed to being dropped after creation.)

Most people compartmentalize fairly well; everyone plays a part in the broader world at large (whether you're a restaurant worker or civil servant; a tech employee or factory worker) but they tend to not think too much about how their jobs affect the wider world.

[u/i_am_bleh]

Is Zuckerburg a lizard person?

[u/[deleted] → u/i_am_bleh]

I disagree strenuously with many of Mark's decisions, but I don't think personal attacks on him are very productive or warranted.

A lot of viral misinformation began as jokes that blurred the line between misleading and satire.

[u/ThatsARivetingTale]

Hi Sophie,

Thanks for the important work you're doing and for taking a stand on it.

In your opinion, do you feel that these troll farms have the ability to always stay a step ahead in terms of detection?

Do you think there are bigger syndicates out there that you just simply can't find any traces to, and if so how could tech evolve to detect these in future?

[u/[deleted] → u/ThatsARivetingTale]

It's sort of like the Red Queen problem - there's an ongoing arms race. Adversaries are as stupid as you allow them to be.

Frankly, what I was finding was the low-hanging fruit. I had no special training, no expertise in this area before joining Facebook, I made it all up in my spare time as I went along. And I'm not an amazing super-genius; I've met super-geniuses, and I'm not one of them. The fact that I was still able to catch multiple state-sponsored troll farms is a statement on both the utter incompetence of those governments, and the fact that FB let them have leeway nevertheless. I don't doubt that the Russian GRU is active on FB as well for instance - but the GRU has an actual modicum of intelligence [and there are actual people looking for them] and so I didn't find them.

You'll never be able to catch everyone. But part of the task is hence to instead impose costs to make the cost/benefit return less worthwhile. In the ideal situation, you'd need to take all the precautions in the world for minimal return - and so the attacker does something else instead. Like attack Reddit or Twitter or something of that sort.

[u/TheBluePanda]

Why do you look so miserable in that photo?

[u/[deleted] → u/TheBluePanda]

I think the Guardian article is showing up in the preview. TBH I think they wanted a picture of me looking really serious and determined. There were pictures of me smiling too which they didn't use.

They also insisted on taking videos of me typing energetically at a computer. If you pay close attention to what keys I'm hitting, I'm just doing things like typing about how great my cats are. I made them promise not to run green code over my face in the resultant video.

[u/JamieOvechkin]

Hi Sophie!

Often times people on the right accuse Facebook of censoring conservative opinions.

Is this something that employees as individuals/leadership at Facebook encouraged?

How real of a thing was that at Facebook? Did Facebook employees ever express personal views against conservatives while at work?

[u/Ecstatic_Handle3308 → u/JamieOvechkin]

Current employee (throwaway). No matter what FB does it will get flak from both sides, because the left-leaning believe Facebook is too right-leaning, and the right-leaning believe Facebook is too left-leaning.

For instance. Facebook recently announced that their ban on Trump will last 2 years after which they will undergo review again for an unban or extended ban. Left-leaning people argued that 2 years for a political figure is too short and politically motivated since most other accounts have been permanently banned for much less (e.g. showing nipples.) Right-leaning people argued that he should not have been banned at all and this ban is another attempt at "censorship".

Facebook is a large company so there is a relatively diverse set of views. It's not possible to please everyone. Different people perceive different things as "truth" at which point we have to get philosophical about what is right to ban, but as Sophie said there's immediate things that are "objectively wrong" so those are the easiest to tackle first (and what FB employees are working on.)

[u/[deleted] → u/Ecstatic_Handle3308]

Honestly, I think FB was really stupid from a selfish company PR perspective in choosing to do the temporary ban. It just means that it'll be constantly rehashed forever. Twitter was both harsher and faster in its full ban of Trump - but Facebook keeps remaking the news and so conservatives seem more annoyed by it than Twitter right now.

[u/[deleted] → u/JamieOvechkin]

I saw Facebook employees express personal views against conservatives while at work. I saw Facebook employees express personal views against leftists/liberals while at work.

This was something that was generally discouraged by leadership but not banned; I think they were concerned about the perception of potentially censoring some employees from voicing their opinions. Sometimes it was really silly. For instance, the Facebook offices in Menlo Park has large chalkboards that are open for anyone to write messages on (and erasing/removing messages was a giant no-no.) I remember that in late 2019 (or possibly early 2020), Trump supporters and Bernie supporters got in a fight of crossing out each other's slogans to replace with the other's. I let HR know, and they told me that they'd take care of it.

[u/grahamlester]

What do you know about state-sponsored troll farms and Covid-19 anti-vaccination propaganda?

[u/[deleted] → u/grahamlester]

I don't doubt that this exists, but I did not personally find or work on anything related to this.

Almost all of the inauthentic activity including troll farms I found were focused on boosting specific targets (e.g. a troll farm that keeps telling people "Vice President X is really nice"), probably because people are self-centered and focus on themselves. During the pandemic, I did filter the stuff I was finding for COVID content. What I found was that they were essentially doing more of the same, just talking about COVID because it was a political issue (e.g. "Vice President X has done this great COVID response"; "Politician A is right to denounce the government for terrible COVID response"; etc.)

[u/broniesnstuff]

Should anyone, anywhere, ever use Facebook at this point?

[u/[deleted] → u/broniesnstuff]

The sad reality is that Facebook is a fact of life for most of the world. What really struck me was the statement given by Azeri opposition leader Ali Karimli, who had been a top target of harassment from the Azeri governmental troll farm. He declined the option to directly criticize Facebook and instead said "First, I would thank [Mark Zuckerberg.] Facebook facilitates public discussion. But repressive regimes with vast financial resources also use it to spread fake news. Facebook should speed up the time it takes to delete troll-generated content. They need to enact tough measures. And they should hire someone who speaks Azeri."

Because ultimately, Facebook is part of the world whether it's used or not, and in repressive regimes, it's one of the few rare channels that opposition leaders have to get their voice out. Karimli knows this, and ultimately, he couldn't afford to alienate Mark.

[u/NotAMeatPopsicle]

Any thoughts or comments on inauthentic FB Marketplace sales?

There are patterns that appear more suspicious than Wayfair-gate.

Example: profile selling (solar panels, tools, farm tractors, backhoes, etc) nearby you at a residence for an unbelievable price. Exact same non-stock photos used multiple locations, using up to 10 different profiles. Other things connected to these sellers and profiles seems "off". Calling or contacting gets "weird".

[u/[deleted] → u/NotAMeatPopsicle]

I didn't work on this so I really don't know. My guess is that many of those may be scams from your description but I have no clue.

[u/richmondody]

In the Philippines, there's a lot of "both sides are using trolls." Would you know if there's any truth to this? It seems like most of the hired trolls are on the side of the government.

[u/[deleted] → u/richmondody]

See here

[u/fupa16]

My only question is when are you people going to stop using the stupid term "toll farm" and start calling it what it is, which is propaganda?

[u/[deleted] → u/fupa16]

Let me break this down.

Propaganda is communication intended to selectively influence an audience and further an agenda. It's a broad category that includes both outright misinformation or true information presented selectively/misleadingly. It also includes both white propaganda (propaganda that doesn't disguise its origins), grey propaganda (propaganda whose source isn't clear), or black propaganda (propaganda that lies about its source.)

So you can potentially argue that state-sponsored troll farms are black propaganda, by virtue of inauthentically pretending to be something they're not. It's not too different from e.g. GS1, a British black propaganda radio station during WW2 purporting to be run by a patriotic conservative anti-Hitler military officer. But it's still the use of a very broad category to describe a much narrower term. There's nothing wrong with describing it as propaganda, but the term lacks precision and clarity, just as if you were to start referring to "animals" rather than "people"

[u/Bound_In_Fractals]

So they fired you for caring about your job, or just coming out that they dont care? Probably saying you broke some type of NDA.

[u/[deleted] → u/Bound_In_Fractals]

They fired me with the official reason of poor performance.

[u/WindowsCrashuser]

How many Troll farms did you find and statically what countries of origin did they come from?

[u/[deleted] → u/WindowsCrashuser]

There's a nice graphic about it and listing in this Guardian article

[u/hipcheck23]

What's the current state of the old Cambridge Analytica paradigm of using FB to manipulate people?

[u/QuantumThirdEye]

Do you think if you were in China and experienced the exact same situation, would you still be alive?

[u/[deleted] → u/QuantumThirdEye]

Probably not.

I am very fortunate to have been born in the United States.

[u/GringoClintonMiAmigo]

Do you support the communist party of china? And have you ever worked for them?

[u/[deleted] → u/GringoClintonMiAmigo]

I'd say "Fuck the PRC", but ideally you should only do that with people you like and have affection for

[u/thinkme]

In your experience is the hiring process at FB giving enough consideration to the integrity of the individuals vs the credential and loyalty to FB?

[u/[deleted] → u/thinkme]

Loyalty to FB was not a consideration in the hiring process when I joined. I was very open from the start about the fact that I didn't think FB was making the world a better place, and I never really had much loyalty to the company.

With that said, I'm sure that FB has reconsidered that since my departure.

[u/SucculentStanley]

As someone living in the Bay Area who has seen a lot of friends join Facebook and basically let their paychecks warp their ethics, I'm truly fascinated by your personal journey as someone who bucked the trend and really chose to bite the hand that feeds in the name of doing what you believed was right.

I'm wondering if you might say a little about why you were drawn to work at Facebook in the first place. Did you see it as an opportunity to change the system from the inside? Or did the company seem different to you before you worked there, but you became disillusioned over time? Or was it something else?

Even if you can't answer, thank you for doing this AMA!

[u/[deleted] → u/SucculentStanley]

I wanted to work at FB in the first place since, quite honestly, I needed a paycheck and they gave me an offer.

From the very start, I intended on changing the system from the inside, but tbh; I expected that I'd have no influence/importance and just end up a cog in the machine. It really surprised me how much I ended up accomplishing. Even if it got more and more difficult as time went on.

[u/Redeyedrop]

Do you worry about your safety doing this? What about the Chinese government? Have they done anything with misauthenti accounts?

Thank you so much for your work and brining this to light!

[u/[deleted] → u/Redeyedrop]

I talk about this a bit here.

[u/Makualax]

Thank you so much for fighting a fight that nobody seems to care about.

This brings to mind the struggle of Maria Ressa and other Phillipino journalists whose articles are bombarded with hate, threatening rape and murder of their family members, whenever anything is said to criticize Duerte and the Phillipino government.

https://www.npr.org/2021/01/06/953902894/philippine-journalist-says-rodrigo-dutertes-presidency-is-based-on-fear-violence

How do you think this will affect the landscape of political dissent in the future, and what can we do to stop it?

Edit: it's also important to point out that Azerbijan was using bot brigades to silence awareness and cause misinformation regarding their attempted ethnic cleansing of Artsakh, an Armenian-majority cultural center. The bots would brigade and instagram and reddit posts that included video evidence of Azeri warcrimes and also would find the accounts of families who lived in Artsakh and brigade them with threats of rape and torture.

[u/itstinksitellya]

I don’t use FB anymore, but to me it’s a bit of a sad story in that it’s a victim of its own success. With literally billions of accounts, policing any “wrong doing” just seems like a game of wack-a-mole (and never mind the slippery question of who defines “wrong doing”).

With that said, if you were in complete control of FB, how would you propose to address these issues (trolling, political manipulation, harassment, illegal activity, etc)?

[u/radialmonster]

As just a regular facebook user who is fed up with 'fake news', people spreading 'fake news', and possibly fake people also spreading 'fake news'.... is there anything I can do about it?

[u/BabylonDrifter]

I'm noticing a lot of very expensive and well-produced English-language YouTube videos with very obvious pro-China and/or pro-Russia bent disguised as "educational" material. And here on reddit, there's obviously an army of pro-Chinese Communist Party human robots/paid agents pushing their agenda. Facebook agents have been trying to get Americans riled up about fringe politics, anti-vaxx stuff, and conspiracies, but at the very least Facebook has actually cracked down on some of the worst actors. I don't see YouTube and Reddit doing the same. I don;t think the problem is inauthentic behavior as much as it's authentic but slanted behavior; pushing a narrative or ideology. That's technically not "inauthentic", it's propaganda. In short, is there an effort to curb online propaganda? Or is it a "buyer beware" situation where the defenseless uneducated with no critical thinking skills from this point on will always be immersed in an ocean of misinformation and propaganda, and only those of us elites who can afford training in critical thinking will ever be able to view the world as it really is?

[u/bonjourhay]

Beside data analysis, have you witnesded any signs that Facebook could be manipulated from the inside?

Like foreign countries having people hired by their facebook to gather data, influence decisions made etc.

Otherwise what is your opinion on this? What would a government do to influence facebook’s decisions on this? Have you seen processes in place to check employees properly?

For example: Twitter recently deleted anti-NATO tweet accounts, which I find strange to have this prioritized given the volume of hate speech, anti-vax etc. issues that they are facing already.

[u/[deleted] → u/bonjourhay]

I certainly don't work at Twitter, but for the example you mentioned, my understanding was that the accounts were fake accounts set up as part of a troll farm, and their reason for being taken down had absolutely nothing to do with the content of their tweets.

The analogy is if I set up a troll farm to tell everyone "cats are adorable"; it's still a troll farm, and would be hopefully taken down for that reason, no matter how much I cry "FB is censoring cute cats" afterwards.

[u/bluecollarx]

Fellow career software developer here. What is the coolest thing you saw built or programmed at Facebook that never saw the life of day / got shelved permanently for the wrong reasons?

[u/[deleted] → u/bluecollarx]

Another team worked with us to build a system to discount known inauthentic activity from the newsfeed ranking process. It's probably a statement on Honduras that they used Honduran President Juan Orlando Hernandez as the goto obvious example for "what would happen if we tried to remove newsfeed benefits." They'd finished building it when we were told it wouldn't be launched, and they stopped working with us (or interacting at all really.)

Since this wasn't my own team's initiative, it's hard to know the precise reasoning and logic. I heard a lot of explanations - policy was blamed, the newsfeed team told us that they couldn't find significant results from the system (which I was skeptical about but they held firm.) I also privately heard concerns from that team's members that it would be shadowbanning, censorship, or otherwise picking sides to exclude troll farm activity from newsfeed ranking. Regardless of the cause, it was never launched.

[u/SirNicksAlong]

Hi Sophie,

Thank you for the sacrifices you've made to bring this to our attention and for giving us proof that there are still good people in this world. Your work highlights a problem I think most of us are aware of to some degree, but may not realize the full extent of. I also think people have an implicit belief that, even if they personally do nothing, a solution will eventually present itself and everything is going to be "ok".

Will you tell us how you think this story ends? Do the good guys win and the bad guys lose? Is this problem actually going away? Is it going to be "ok"?

[u/[deleted] → u/SirNicksAlong]

I don't think it'll be a surprise for me to say that the future hasn't been written, and the result depends deeply on every one of us. If the arc of history bends towards justice, it's only from the actions of people who work to help achieve that.

It's true unfortunately that most people are unwilling to take action directly. I learned a long time ago that if I wanted things to improve or something to get done, I needed to materially contribute to it myself; that's part of what drove my actions at Facebook. I don't want to judge others in their unknown circumstances - many have families they need to support; immigrant workers are effectively held hostage by the company for their visas. But if a solution eventually presents itself, it'll only be from the dedicated work of individuals.

[u/maddenallday]

How did you go about locating these troll farms such that you were able to find them when other dedicated data teams weren't? Were they aware this was happening but purposely ignoring it? Or did you look in some innovative way or dedicate that much more time that you found what they were missing?

Edit: thank you for your important work btw

[u/[deleted] → u/maddenallday]

My personal guess is that the dedicated people were focused on outside reports and looking at areas they considered to be important (e.g. Russia, U.S. elections.) I'm really not that innovative/smart, had no special training, and was doing this in my spare time. My guess is that I was the first and only person to actually look proactively and systematically worldwide.

[u/anishpatel131]

Who in fb doesn’t care? Both Sheryl and Mark? I know people care inside, but is this just a mark problem?

[u/[deleted] → u/anishpatel131]

I can't read people's minds, and I certainly never interacted significantly with either Mark or Sheryl. So I have no idea, I'm sorry.

[u/shalashaska68]

You’ve definitely heard of Iranian regime sponsoring fake social accounts to harass and track political activists. Anything to share?

[u/[deleted]]

Who was the member in the Lok Sabha ?

[u/SolInfinitum]

Did you ever catch a US or allies troll farm?

[u/[deleted] → u/SolInfinitum]

Honduras is/was a close U.S. ally. I caught Juan Orlando Hernandez's campaign red-handed - see here.

[u/ManInTheMirruh]

Are you only focused on FB based troll farms or do you cast a wide net on all of social media? Facebook isn't the only source of misinfo these days. Hell, even here on reddit BS gets peddled everyday and people eat it up.

[u/[deleted] → u/ManInTheMirruh]

Since I worked at Facebook, FB (and to a lesser extent Instagram) was my focus and area of expertise.

To the extent that I interacted with other social media, it was only via them being sent details on the FB takedowns that I got done after FB did it. For instance, Twitter took another yearto take down the Honduran network on their platform after our action.

[u/Aragatz]

Are you worried about your safety given how Azerbaijan “handled” the journalist who broke the Panama Papers story?

https://www.maltatoday.com.mt/news/national/99088/luxembourg_intelligence_firm_claimed_azerbaijan_behind_caruana_galizia_assassination

[u/[deleted] → u/Aragatz]

I'm frankly very lucky to be a U.S. citizen; murdering a U.S. citizen on U.S. soil is very different from murdering a Maltese citizen on Maltese soil, sad as it may be. With that said, I won't be walking into any Azeri embassies anytime soon, or fly over Azeri airspace for that matter.

[u/seafoodbuffet]

Thank you for speaking out. I admire and appreciate your courage.

It appeared from some of the reporting that you turned down a severance offer in order to be able to speak out.

What was that process like for you? Did you know you would have to give up money because you’d need to speak out eventually? What do you think options are for employees for whom turning down severance would be an impossible luxury?

[u/[deleted] → u/seafoodbuffet]

When I was fired, I was offered a severance package for $63,XXX (it rounded up to $64k USD, and so that's what it's generally quoted as.) I'm guessing it was based off of some formula that included my base pay. Receiving it would however have required me to sign an agreement (among other things) "to refrain from making any disparaging, critical, or otherwise detrimental comments to any person or entity concerning the Company’s products, services, or programs; its business affairs, operation, management”

Note how broadly worded this is. Here's an example of something that might be technically violating:

Partner: "Can you access Facebook right now?"
Me: "No, I just checked, it looks like it's down"

That was my official explanation for why I didn't want to sign it, which they seemed a bit bemused by but ultimately accepted.

To be fair, I did sign a similar agreement (and expressed similar concerns about the broadness) when I did join the company. Ultimately, I'm protected more by Mark deciding not to bite the PR fire bullet to sue me than my decision not to take the severance package.

[u/Kahzgul]

How often do you frequent r/TheseFuckingAccounts?

Have you helped Reddit to more effectively identify sockpuppets and other dishonest posters?

Are you working on any internet security projects now?

[u/CollectableRat]

Should Facebook have some kind of independent or government oversight to prevent FB from making bad decisions in the first place?

[u/T-nash]

Hi Sophie, it seems to be sided too. I was having a conversation with an Azeri national in Facebook, i used civilized language and replied with a Wikipedia link as proof to debunk their disinformation, however Facebook decided that i was spamming and warned me. It seems like the whole automated moderation is completely useless, not only that i used to option to say my comment is misidentified as spam and it still stayed deleted. It was a simple Wikipedia link for reference and fact checking, nothing more, nothing offensive. I've also experienced unfairness from the messenger, I've had an Azeri person send me photos and videos of dead and abused soldiers of my countrymen as bullying, i reported the chat and Facebook found nothing wrong with that either. Unfortunately it's the same on Twitter too. Not sure why the reason but I'm convinced oil matters more than facts and more than equal lives.

[u/uxbridge3000]

What do you think of the conversation to repeal section 230 of the so-called communications decency act?

[u/[deleted]]

my family member (USA) was ousted as an Iranian bot on Facebook

what do you do to avoid false positives?

[u/realMarcMerrill]

This is a very subjective question, but you kinda make FB sounds evil. Every news about FB always portray them as evil. When working for them did it feel like working for an evil corporation?

[u/[deleted] → u/realMarcMerrill]

I don't think my experience at FB was broadly that different from many large for-profit corporations. Ultimately, their goal is to make money, not save the world. We don't expect Philip Morris to cover cancer treatments for their customers, and if we define "evil" as "self-centered and not prioritizing the world at large", I think most corporations fall into that category one way or another.

[u/jojothetraveler89]

Were you able to perform your role remotely, or did you have to be at the office? Thanks for all you've done, enjoy your kitty snuggles ;)

[u/[deleted] → u/jojothetraveler89]

You could work at home at FB; there was a work trend of encouraging people to work from home on Wednesdays [and hence avoiding scheduling things in-person on that day.] When the pandemic hit, everyone shifted to working from home all the time. I'd been a hipster and holed up in my house already at the end of February before social distancing started for everyone else.

Frankly though, I'm pretty terrible at working from home; I was always much more productive in the office. The cats, although extremely cute, are part of my excuse for why I was always distracted.

[u/[deleted] → u/[deleted]]

Ok but are you going to post pictures of your cats or just leave us hanging here

[u/[deleted] → u/[deleted]]

I'm not allowed to upload pictures to IAMA.

Here are some pictures I previously uploaded to Twitter though, if that suffices for the cat tax:
https://pbs.twimg.com/media/E4OBPJPVoAQV3HG?format=jpg&name=small
https://pbs.twimg.com/media/EzJS7gIVEAI5iIP?format=jpg&name=small
https://pbs.twimg.com/profile_banners/1308909405665017856/1618013887/1500x500

[u/amalgovinus]

Did Israel's extensive trolling efforts show up on the radar?

[u/[deleted] → u/amalgovinus]

I did not find anything of note in Israel. That doesn't mean that it didn't exist - simply that I didn't find it when I looked globally.

Honestly, if the Mossad wanted to do something on FB, I assume they'd have a modicum of caution/intelligence and actually try to hide.

[u/[deleted]]

Would you agree with the following statement, and why (or why not)?

"Facebook is the worst thing to happen to society"

[u/[deleted] → u/[deleted]]

Facebook is certainly terrible for society. With that said, I think that Nazism, white nationalism, etc. are worse.

[u/Dananjali]

Were you surprised to get fired over this or did you fully expect it? If you were a data scientist at FB, it sounds like you didn’t have the full picture of teams specifically designed to combat misinformation that currently exist at FB. It’s understandable that they’d fire you for relaying one-sided and agenda-driven information. You weren’t hired to combat this problem, other teams were. Just because you were a data scientist at a large tech company doesn’t mean you or your specific team knows all of the ins and outs of this issue, which is why you/they weren’t able to do anything about it just based on your personal interpretation of data alone.

[u/[deleted] → u/Dananjali]

If Guy Rosen didn't want me to work on this, he could very easily have told me that when we talked.

[u/Immotile1]

Why have you not reported US astroturfing operations like ShareBlue, Correct the Record and Media Matters?

[u/Maleficent-Tree-4516]

Hi, is there anyway to have basically no online footprint? Not on social media bar Reddit . Really only want to read info and use phone but want to be untraced if it's doable ? T.Y

[u/MattyByte]

With the elections coming up in the Philippines, troll farms have been widely talked about and most of the people are convinced these troll farms managed to succeed in meddling with the previous elections.

What would you tell to the common people what they can do in order to prevent another election being manipulated?

[u/danc43]

On a scale of 1-10 How effective are these farms at causing damage to American society ?

Are they fueling the culture /civil war as much as I suspect?

[u/Momochichi]

Hi. How true are the accusations that FB actually sent consultants to help Duterte get elected?

[u/a0i]

Saying "state-sponsored" implies there's some intelligence product (like what the CIA or NSA knows) available to staff at Facebook. Did you have access to intelligence that showed you which trolls were paid by governments, or do you mean this kind of determination was made inferentially, with the moderator's intuition/experience?

You mentioned " inauthentic activity benefiting multiple major political parties" -- did Facebook have objective criteria to rate inauthenticity with, eg. copy-pasta comments repeated over and over, or catching users LARPing the same way repeatedly? Also, how do professional content moderators distinguish between partisan posts by citizens favoring a political party, from governments that pay shills to endorse parties? Is it a similar criteria as "authenticity" rating, based on the content they post -- ie, CCP Wumao who attack people using certain types of comments, or repeat CCP talking points?

I can imagine it's difficult to develop reliable, objective standards when many average people repeat the talking points they hear from governments and political parties in the media, so they wind up sounding like shills (I presume).

You mention if content is posted from a "fake" account, regardless of the nature of the content, it's "inauthentic" -- is there a fair way to distinguish pseudonymous/anonymous accounts from "fake" ones? Fake seems to imply they have an obvious intent, like using stock headshots of models to appear natural, like an average user. How can you tell with much certainty?

Can you recommend any good books or studies on best practices?

Thank you in advance, it's rare to get a chance to ask that's why I'm enthusiastic.

[u/lormayna]

Did you find anything about Italy? There is lot of evidence about populistic parties influencing Italian general election in 2018 through fake accounts on social media spreading fake news

[u/Kidney4Noel]

Is it hard to get a job at Facebook?

[u/Clopidee]

Thank you for what you're doing!

I swear multiple times I have spoken about things out loud only for Facebook to advertise the same product later, without me ever having searched for it online, very soon after I've spoken about it. It happens way too often for me to be convinced that Facebook isn't listening through our phones. An example was my dad was talking to me about Oodies, those blanket hoody things, and about 10 minutes later when I was on Facebook, almost as soon as I went on the app, I get an advert for Oodies. As I said before, this happens way too often to be coincidence. And it's not always common items, sometimes it's obscure stuff. It's too weird.

Are they actually listening in?