EXCLUSIVE: State Department Trashed 30,000 Log-in Key Fobs After Hack

The State Department over the past few months replaced some 30,000 network log-in fobs and digital tokens that employees had been using to access its systems remotely, after the agency's unclassified network was hacked, according to a department official. 

During the switchover, some State personnel said they were not able to access work outside the office for months. 

"All of us had to turn them in and go through a very extended procedure of changing every aspect of our internal passwording," said one foreign service officer. "Every one of us had to create new passwords and new PIN numbers to go along with our fobs. They changed the type of format that you use to create a PIN to make it more secure and they changed the requirements for your basic State Department password to make it more secure."

The decision to replace tens of thousands of passcodes and two-step authentication credentials "was based on sound cybersecurity best practices,” the State official said.

More than half of the credentials were "soft tokens" or digital credentials used as the second step after entering a basic password. Those tokens were updated immediately and the rest were shipped with a few days, according to State. 

Revoking an entire inventory of log-in credentials all at once speaks to the serious nature of the attack.

"That is a substantial move," said John Dickson, an information security specialist who earlier served in the Air Force Information Warfare Center. While the data at risk was unclassified, it could have been very sensitive government information related to foreign affairs, he added. 

“Doing that costs a lot of money and logistically is a bloody nightmare," said Dickson, now chairman of the San Antonio Chamber of Commerce Cybersecurity Committee. In an unclassified system, "if you're a nation state, you might find [operations security]-related things that tell a lot about intentions and a lot of context about how the State Department is operating.”

The expense of canceling and re-issuing an entire department's IDs in one fell swoop can range from hundreds of thousands of dollars to millions of dollars, he estimated.

Which Computer Device Can You Trust? 

Both the White House and State Department, for a time, delayed fully eradicating malicious activity after hackers simultaneously attacked each of their networks beginning last fall. This was intentionally done to ascertain the extent of the breaches. 

On Nov. 14, State disconnected its main unclassified system, which handled email, to improve security. 

However, Thursday night, The Wall Street Journal reported the government, for at least three months now, has still been unable to boot the hackers entirely. 

Whenever a threat actor uses a "rootkit," a worm that allows hackers to mask virtually all their activity, investigations become incredibly difficult, at times impossible, said Dickson, also a principal at cyber consulting firm Denim Group.

It "contributes to a hall of mirrors situation where defenders don’t know what to trust," he said. When organizations don't know if their devices are safe, they "consider building devices from ground up, which is the doomsday scenario.”

He likened the situation to what Sony was forced to do after hackers, who federal officials believe to be North Korean, destroyed corporate hard drives and leaked troves of data.  

On Friday, State said there is robust security in place to protect the department's computer systems and information, including the unclassified system.

"We deal successfully with thousands of attacks every day and we deal with them in conjunction with other relevant government agencies," the department official said.

The credentials at issue were manufactured by security firm RSA, which itself was the victim of a 2011 cyber assault. Suspected Chinese hackers stole company trade secrets involving the fobs' mechanics. That same year, intruders, thought to be the same group, made use of RSA's secret sauce to break into an RSA-protected Lockheed Martin network. 

State officials say they got rid of those insecure tokens in 2011.

As evidenced by the Lockheed hack, insecure tokens can be exploited by hackers to gain access to an organization’s networks. There’s no indication that is the case with the State breach.

"If the fob is vulnerable, depending on the level of access of the person, the bad guy could get through," said Robert Bagnall, founder of The Maverick Group computer security firm and a former Air Force intelligence officer. But “even if the fobs are secure, there are other ways for the hackers to get in," like through an employee accidentally clicking on a malicious email that downloads a rootkit.