【2016 Archives】

A little over a week after kicking off a firestorm by announcing the FBI was reopening the investigation into the Hillary Clinton email scandal016 Archives FBI director James Comey pulled a "whoops, my bad" on Sunday and said there was nothing to see here, move along, please.

Except it's not that easy. Not in 2016.

There are a lot of questions left unresolved, but the most prominent among them right now is, "how in the hell did the FBI actually scan all these new emails in eight days?"

Thousands and thousands of emails

Investigators found thousands of emails "pertinent" to the Clinton email investigation on devices belonging to Anthony Weiner and his wife, Huma Abedin, a top Clinton aide. The emails in question were found during an investigation into Weiner's alleged sexting relationship with an underage teen.

Upwards of 650,000 emails were said to be involved, according to a published report in the Wall Street Journal, and it seemed the issue wouldn't be resolved until after the election and, yet, here we are.

So how did the FBI clear this up so fast?

That is a lot of emails so, sure, there's going to be some skepticism about the FBI's ability to scan through all 650,000 of them in a little more than a week. As soon as Comey's announcement hit the news on Sunday, Donald Trump was first to question the speed of the investigation. As per usual, his followers were not far behind -- jumping on Twitter to add fuel to the speculation.

This Tweet is currently unavailable. It might be loading or has been removed.

How the FBI scanned the emails so quickly

So what's going on? Well, contrary to what Trump may say, the entire process isn't nearly as egregious as he suggests.

To start with, the number of emails the FBI needed to look at was dramatically reduced right away as many of the emails were duplicates of emails the FBI already had its hands on or were personal emails from personal email accounts.

Newsweekalso notes that Weiner told investigators that he had synced Abedin's phone to his laptop for backup purposes, likely creating even more email duplicates.

This Tweet is currently unavailable. It might be loading or has been removed.

Trump and his surrogates seemed to imply that the FBI is manually reading each and every email and then deciding how each individual message figures in to the investigation. Except that's not how it works; there are other methods to weed out duplicate and unrelated emails.

This Tweet is currently unavailable. It might be loading or has been removed.

WIREDhas an excellent run-down of the process involved in the investigation which basically boils down to this: use search to filter out irrelevant emails and duplicates and use a process called "hashing" to filter out those superfluous messages.

That hashing process converts portions of text into shorter character strings that uniquely represent the text: running a hash function on that same text will always produce the same short string of characters, but any tiny change in the text produces a different hash string. And that allows a program to quickly compare and match text samples.

Such a process, a good example of which is shown over at Errata Security, should take no more than a few hours, something confirmed by NSA whistle-blower Edward Snowden.

This Tweet is currently unavailable. It might be loading or has been removed.

It's unknown how many of the reported 650,000 emails were duplicates and how many were left for the FBI to actually investigate. But these searches and filters likely narrowed the field of new emails to be investigated by the FBI down to a more manageable number.

A resolution, sort of

So, for now, this seems to all be resolved. Kind of.

The problem is, for voters on both sides of the political divide, there are now plentiful reasons to completely distrust the FBI and any of its discoveries.

To Republicans, certain there is something nefarious going on here, Comey's actions are signs of incompetence in terms of bungling an investigation in which surely there's something here that should lead to a Clinton indictment.

This Tweet is currently unavailable. It might be loading or has been removed.

And to Democrats, Comey's actions are a sign of incompetence. They believe he bungled an investigation in such a way that the FBI, which is supposed to be a neutral party, is playing the worst sort of partisan politics on the eve (literally, now) of a hotly contested election.

As for how much of an impact this new email ordeal will have on the election, it's hard to say. But data wiz Nate Silver did drop this nugget last night.

This Tweet is currently unavailable. It might be loading or has been removed.

Whatever the case, we should know soon how big of an impact this email saga has at the polls.