Pentagon accidentally exposes web-monitoring operation

The Department of Defense accidentally exposed an intelligence-gathering operation, thanks to an online storage misconfiguration.

DOD was reportedly collecting billions of public internet posts from social media, news sites, and web forums and storing them on Amazon S3 repositories. But it neglected to make those storage servers private. So anyone with a free Amazon AWS account could browse and download the data, according to Chris Vickery, a security researcher at UpGuard.

Vickery noticed the problem in September. "The data exposed in one of the three buckets is estimated to contain at least 1.8 billion posts of scraped internet content over the past 8 years," UpGuard said in a Friday report.

Much of the data was scraped from news sites, web forums, and social media services such as Facebook and Twitter. The information includes content relating to Iraqi and Pakistani politics and ISIS, but also social media posts made by Americans.

In a Twitter direct message, Vickery told PCMag he "made sure the [storage] buckets we discovered were secured before anything was brought to media attention." However, he has no idea if anyone else, like malicious parties, ever accessed the data.

DOD didn't immediately respond to a request for comment. But the Pentagon confirmed the accidental leak to CNN.

Why the Defense Department was collecting this information isn't clear. But it certainly raises eyebrows at a time when concerns persist about US surveillance programs. It also comes as US agencies are struggling on the cybersecurity front. The National Security Agency, for instance, failed to stop breaches of its own classified hacking tools.

"Even the most sensitive intelligence organizations are not immune to sizable cyber risk," UpGuard said in its Friday report.

The Defense Department isn't the only one to commit the security slip-up with AWS cloud storage. Earlier this year, UpGuard found that Verizon and Dow Jones made the same mistake, effectively exposing their private customer data to the public.

This article originally appeared on PCMag.com.