Spammers, hackers, political propagandists, and other nefarious users have always tried to game the systems that social media sites put in place to protect their platforms. It’s a never-ending battle; as companies like Twitter and Facebook become more sophisticated, so do the trolls. And so last week, after Facebook shared new details about a tool it built to analyze text found in images like memes, some people began brainstorming how to thwart it.
Social media companies are under tremendous pressure from lawmakers, journalists, and users to be more transparent about how they decide what content should be removed and how their algorithms work, especially after they’ve made a number of high-profile mistakes. While many companies are now more forthcoming, they’ve also been reluctant to reveal too much about their systems because, they say, ill-intentioned actors will use the information to game them.
Last Tuesday, Facebook did reveal the details of how it uses a tool called Rosetta to help automatically detect things like memes that violate its hate speech policy or images that spread already debunked hoaxes; the company says it uses it to process one billion public images and videos uploaded to Facebook each day.