The Internet of Things is fast turning into the Internet-of-Things-We-Can’t-Afford. Almost daily now we are hearing about virtual shakedowns wherein attackers demand payment in Bitcoin virtual currency from a bank, e-retailer or online service. Those who don’t pay the ransom see their sites knocked offline in coordinated cyberattacks. This story examines one contributor to the problem, and asks whether we should demand better security from ISPs, software and hardware makers.
These attacks are fueled in part by an explosion in the number of Internet-connected things that are either misconfigured or shipped in a default insecure state. In June I wrote about robot networks or “botnets” of hacked Internet routers that were all made and shipped by networking firm Ubiquiti. Attackers were able to compromise the routers because Ubiquiti shipped them with remote administration switched on by default and protected by a factory default password pair (ubnt/ubnt or no password at all).
That story followed on reports from security firm Imperva (see Lax Security Opens the Door for Mass-Scale Hijacking of SOHO Routers) which found a botnet of tens of thousands of hijacked Ubiquiti routers being used to launch massive ransom-based denial-of-service attacks. Imperva discovered that those tens of thousands of hacked devices were so easy to remotely control that each router was being exploited by several different extortion groups or individual criminal actors. The company also found those actors used the hacked routers to continuously scan the Internet for more vulnerable routers.
Last week, researchers in Vienna, Austria-based security firm SEC Consult released data suggesting that there are more than 600,000 vulnerable Ubiquiti routers in use by Internet service providers (ISPs) and their customers. All are sitting on the Internet wide open and permitting anyone to abuse them for these digital shakedowns.
These vulnerable devices tend to coalesce in distinct geographical pools with deeper pools in countries with more ISPs that shipped them direct to customers without modification. SEC Consult said it found heavy concentrations of the exposed Ubiquiti devices in Brazil (480,000), Thailand (170,000) and the United States (77,000).
SEC Consult cautions that the actual number of vulnerable Ubiquiti systems may be closer to 1.1 million. Turns out, the devices ship with a cryptographic certificate embedded in the router’s built-in software (or “firmware”) that further weakens security on the devices and makes them trivial to discover on the open Internet. Indeed, the Censys Project, a scan-driven Internet search engine that allows anyone to quickly find hosts that use that certificate, shows exactly where each exposed router resides online.
The Imperva research from May 2015 touched a nerve among some Ubiquiti customers who thought the company should be doing more to help customers secure these routers. In a May 2015 discussion thread on the company’s support site, Ubiquiti’s vice president of technology applications Matt Harding said the router maker briefly disabled remote access on new devices, only to reverse that move after pushback from ISPs and other customers who wanted the feature turned back on.
In a statement sent to KrebsOnSecurity via email, Hardy said the company doesn’t market its products to home users, and that it sells its products to industry professionals and ISPs.
“Because of this we originally shipped with the products’ configurations as flexible as possible and relied on the ISPs to secure their equipment appropriately,” he said. “Some ISPs use self-built provisioning scripts and intentionally locking down devices out of the box would interfere with the provisioning workflows of many customers.”
Hardy said it’s common in the networking equipment industry to ship with a default password for initial use. While this may be true, it seems far less common that networking companies ship hardware that allows remote administration over the Internet by default. He added that beginning with firmware version 5.5.2 — originally released in August 2012 — Ubiquiti devices have included very persistent messaging in the user interface to remind customers to follow best practices and change their passwords.
“Any devices shipping since then would have this reminder and users would have to intentionally ignore it to install equipment with default credentials,” he wrote. Hardy noted that the company also provides a management platform that ISPs can use to change all default device passwords in bulk.
When companies ship products, software or services with built-in, by-design vulnerabilities, good citizens of the Internet suffer for it. Protonmail — an email service dedicated to privacy enthusiasts — has been offline for much of the past week thanks to one of these shakedowns.
[NB: While no one is claiming that compromised routers were involved in the Protonmail attacks, the situation with Ubiquiti is an example of the type of vulnerability that allows attackers to get in and abuse these devices for nefarious purposes without the legitimate users ever even knowing they are unwittingly facilitating criminal activity (and also making themselves a target of data theft)].
Protonmail received a ransom demand: Pay Bitcoins or be knocked offline. The sad part? The company paid the ransom and soon got hit by what appears to be a second extortion group that likely smelled blood in the water.
The criminal or group that extorted Protonmail, which self-identifies as the “Armada Collective,” also tried to extort VFEmail, another email service provider. VFE’s Rick Romero blogged about the extortion demand, which turned into a full-blown outage for his ISP when he ignored it. The attack caused major disruption for other customers on his ISP’s network, and now Romero says he’s having to look for another provider. But he said he never paid the ransom.
“It took out my [hosting] provider and THEIR upstream providers,” he said in an email. “After the 3rd attack took down their datacenter, I got kicked out.”
For his part, Romero places a large portion of the blame for the attacks on the ISP community.
“Who can see this bandwidth? Who can stop this,” Romero asked in his online column. “I once had an argument with a nice German fellow – they have very strict privacy laws – about what the ISP can block. You can’t block anything in the EU. In the US we’re fighting for open access, and for good reason – but we still have to be responsible netizens. I think the ISP should have the flexibility to block potentially harmful traffic – whether it be email spam, fraud, or denial of service attacks.”
So, hardware makers definitely could be doing more, but ISPs probably have a much bigger role to play in fighting large scale attacks. Indeed, many security experts and recent victims of these Bitcoin shakedowns say the ISP community could be doing a lot more to make it difficult for attackers to exploit these exposed devices.
This is how the former cyber advisor to Presidents Clinton and Bush sees it. Richard Clarke, now chairman and CEO of Good Harbor Consulting, said at a conference last year that the ISPs could stop an awful lot of what’s going with malware and denial-of-service attacks, but they don’t.
“They don’t, they ship it on, and in some cases they actually make money by shipping it on,” Clarke said at a May 2014 conference by the Information Systems Security Association (ISSA). “Denial-of-service attacks actually make money for the ISPs, huge volumes of data coming down the line. Why don’t we require ISPs to do everything that the technology allows to stop [denial-of-service] attacks and to identify and kill malware before it gets to its destination. They could do it.”
One basic step that many ISPs can but are not taking to blunt these attacks involves a network security standard that was developed and released more than a dozen years ago. Known as BCP38, its use prevents abusable resources on an ISPs network (hacked Ubiquiti routers, e.g.) from being leveraged in especially destructive and powerful denial-of-service attacks.
Back in the day, attackers focused on having huge armies of bot-infected computers they controlled from afar. These days an attacker needs far fewer resources to launch even more destructive attacks that let the assailant both mask his true origin online and amplify the bandwidth of his attacks.
Using a technique called traffic amplification, the attacker reflects his traffic from one or more third-party machines toward the intended target. In this type of assault, the attacker sends a message to a third party, while spoofing the Internet address of the victim. When the third party replies to the message, the reply is sent to the victim — and the reply is much larger than the original message, thereby amplifying the size of the attack.
BCP-38 is designed to filter such spoofed traffic, so that it never even traverses the network of an ISP that’s adopted the anti-spoofing measures. This blog post from the Internet Society does a good job of explaining why many ISPs ultimately decide not to implement BCP38.
As the Internet of Things grows, we can scarcely afford a massive glut of things that are insecure-by-design. One reason is that this stuff has far too long a half-life, and it will remain in our Internet’s land and streams for many years to come.
Okay, so maybe that’s putting it a bit too dramatically, but I don’t think by much. Mass-deployed, insecure-by-default devices are difficult and expensive to clean up and/or harden for security, and the costs of that vulnerability are felt across the Internet and around the globe.
Dan Geer, chief information security officer for In-Q-Tel — the Central Intelligence Agency’s (CIA) venture capital arm — perhaps said it most eloquently in a May 2014 address at Cambridge. Geer talks about the Internet of Things in terms of a broader class of software+hardware devices dubbed “embedded systems” and their tendency to remain use long after they become potential security liabilities:
Perhaps what is needed is for embedded systems to be more like
humans, and I most assuredly do not mean artificially intelligent.
By “more like humans” I mean this: Embedded systems, if having no
remote management interface and thus out of reach, are a life form
and as the purpose of life is to end, an embedded system without a
remote management interface must be so designed as to be certain
to die no later than some fixed time. Conversely, an embedded
system with a remote management interface must be sufficiently
self-protecting that it is capable of refusing a command. Inevitable
death and purposive resistance are two aspects of the human condition
we need to replicate, not somehow imagine that to overcome them is
to improve the future.
Jeremiah Grossman, chief technology officer at WhiteHat Security, said the world will eventually grasp how important it is to build security into all of the new types of devices being plugged into the Internet each day.
“Only after Internet of Things devices get hacked en masse, and only after billions of internet-connected devices are deployed in the wild,” Grossman said. “We know this future is coming, and there isn’t a lot we can do to stop it. The question I’m asking myself today, is when that day comes, and it will, how can we address the IoT problem 5-10 years from now with billion of those insecure devices in circulation? I don’t have a good answer yet, but we’ve got time.”