US intelligence agencies have lost access to Dataminr, a company that turns social media data into an advanced notification system, according to the Wall Street Journal. While that may sound like a win for privacy, it’s a bit more complicated in practice.
The move leaves government officials without a valuable tool. Somewhat less clear is what sort of stand, if any, Twitter is taking.
Big Bad Data
There are a few threads to untangle here, and plenty of unanswered questions. Dataminr has been in business since 2009, and its main gig is scouring social media for patterns that might indicate breaking news, using algorithms to give those patterns a context and identity, and delivering the result in the form of a real-time breaking news alert. It’s like when you get NYT news alerts on your phone, but on big-data steroids, and only available to clients with a big enough checkbook.
Until recently, according to the Journal, various US defense agencies were among those clients.
“From the government perspective, it’s a good tool, because it gives real-time alerts to things that are happening before anyone really knows what’s going on,” says Aki Peritz, a former CIA counterterrorism expert and current adjunct professor at American University. “We want to allow law enforcement and the intelligence services to know bad things are happening in real time.”
In addition to those real-time benefits—the Journal reports that Dataminr, in fact, alerted the US intelligence committee to last fall’s Paris attacks—it’s important to note that those agencies are also indirect investors in Dataminr, through a venture-capital program called In-Q-Tel. That investment reportedly allowed for a “pilot program,” which has concluded.
When Twitter tells Dataminr to jump, its algorithms calculate exactly how high.
So what’s the issue, and more specifically, where does Twitter come in? Like the intelligence community, Twitter is an investor in Dataminr, though its five-percent stake doesn’t carry nearly as much weight as the firehose it provides. Dataminr is the only outside company with full access to Twitter’s real-time data and permission to sell that data. Without that access, its business model would be directly threatened. So when Twitter tells it to jump, its algorithms calculate exactly how high.
“Dataminr uses public Tweets to sell breaking news alerts to media organizations such as Dow Jones and government agencies such as the World Health Organization, for non-surveillance purposes,” says a Twitter spokesperson. “We have never authorized Dataminr or any third party to sell data to a government or intelligence agency for surveillance purposes. This is a longstanding Twitter policy, not a new development.”
Further complicating matters is that the Department of Homeland Security reportedly has an existing contract with Dataminr as well, unaffected by the current imbroglio. Neither Dataminr nor DHS immediately responded to inquiries.
Arm’s Length
If Twitter wants to distance itself from the government’s surveillance apparatus, it’s picked an opportune time to do so. Apple recently won a high-stakes face-off with the FBI over whether the feds could compel it to write software that undermines an iPhone’s safety, and WhatsApp turned on end-to-end encryption by default for a billion users worldwide. Pushing back against the intelligence community is trending.
Yet there are substantive differences between Twitter’s actions and those of its contemporaries. Apple was fighting a potentially dangerous legal precedent, and the encryption wave protects digital conversations that have a reasonable expectation of privacy. Dataminr, though, packages tweets, which are public.
“It’s merely a way to sort tweets into a way that makes sense for the clients, whether it’s a hedge fund, or the media, or an intelligence service,” says Peritz. The information is searchable to anyone; Dataminr has just concocted a better way to search, and to draw conclusions from the results.
Privacy advocates argue that it’s not so simple, and that severing Dataminr’s ties with the intelligence community is clearly in line with Twitter’s long-time stance on user protection.
“The company has responsibilities to respect human rights,” says Peter Micek, a lawyer with digital rights group Access Now. “Twitter does not want to aid surveillance beyond what it’s legally required to.”
Micek notes that this position isn’t just fluff; it’s part of Twitter’s developer agreement. It states, along with a few other restrictions designed to safeguard users, that its partners will not “allow or assist any government entities, law enforcement, or other organizations to conduct surveillance on Content.”
This, though, somehow manages to raise even more questions. If Twitter has this clause, why did Dataminr partner with defense agencies in the first place? Why does it still contract with DHS? Does an early-notification system, still widely available to private companies, constitute the sorts of abuses Twitter’s developer agreement specifies? And what does cutting off the intelligence community accomplish beyond making them slower to respond to international incidents?
“Let’s say Dataminr works with CNN, and then CNN puts something up on the screen, and people in the government see it and learn about it that way,” says Peritz. “Why cut out the US government in the first place? It’s internally illogical.”
That the data is both public and beneficial is not enough to sway Micek, who says that while the data is technically public, the sort of intelligence garnered by Dataminr should require a warrant for the government to attain.
“It’s easy to say that this information is already out there, but access to that fire hose of data allows upstream collection that the government does, and provides an unprecedented scope and scale of surveillance.”
It’s a thorny issue, made even more so by the lack of clarity from the involved parties. If Dataminr simply sorts public knowledge, then denying the intelligence community that information makes little sense, especially given that DHS still has access. If, on the other hand, Dataminr provides a level of insight that should only be accessible with a warrant, giving that same information to an unregulated hedge fund seems problematic as well.
For now, at least, the government will have to go back to combing through Twitter on its own. It’s not the worst fate. It’s just an odd one.