In March 2017, Facebook launched an ambitious project to prevent suicide with artificial intelligence.
Following a string of suicides that were live-streamed on the platform, the effort to use an algorithm to detect signs of potential self-harm sought to proactively address a serious problem.
But over a year later, following a wave of privacy scandals that brought Facebook’s data-use into question, the idea of Facebook creating and storing actionable mental health data without user-consent has numerous privacy experts worried about whether Facebook can be trusted to make and store inferences about the most intimate details of our minds.
Facebook is creating new health information about users, but it isn’t held to the same privacy standard as healthcare providers
The algorithm touches nearly every post on Facebook, rating each piece of content on a scale from zero to one, with one expressing the highest likelihood of “imminent harm,” according to a Facebook representative.
That data creation process alone raises concern for Natasha Duarte, a policy analyst at the Center for Democracy and Technology.
“I think this should be considered sensitive health information,” she said. “Anyone who is collecting this type of information or who is making these types of inferences about people should be considering it as sensitive health information and treating it really sensitively as such.”
Data protection laws that govern health information in the US currently don’t apply to the data that is created by Facebook’s suicide prevention algorithm, according to Duarte. In the US, information about a person’s health is protected by the Health Insurance Portability and Accountability Act (HIPAA) which mandates specific privacy protections, including encryption and sharing restrictions, when handling health records. But these rules only apply to organizations providing healthcare services such as hospitals and insurance companies.