Remote management tools have made many administrators’ job easier, allowing them to perform administrative tasks (e.g. reinstalling an operating system) without the need to travel to the physical location of the server.
Most servers these days have a Baseboard Management Controller (BMC) – based on the Intelligent Platform Management Interface (IPMI) standard – built into them, allowing administrators to perform administrative tasks remotely.
BMC has the ability to reboot the server, reinstall software and perform many other administrative tasks, thus if compromised, give attackers complete control of the server.
Anthony J. Bonkoski and his team – Russ Bielawski and J. Alex Halderman – from the University of Michigan presented a taunting finding they discovered with BMC at USENIX earlier this month titled “Illuminating the Security Issues with Lights-Out Server Management.”
To start it off, Bonkoski and his team bought a server manufactured by Supermicro – the SYS-5017C-LF rack-mounted system. To perform administrative tasks remotely on this server, administrators simply has to login to the user-friendly, web-based administrative interface provided by the BMC.
Surprising, or not, they discovered a buffer-overflow vulnerability on the “username” and “password” fields used to log into the BMC’s web-based management interface. Making the matter worst, BMC is connected to the Internet by default, allowing anyone on the Internet to exploit the vulnerability, and gain complete control of the server.
Bonkoski and his team scanned the Internet for servers running Supermicro’s implementation of the IPMI management software, to understand how serious this security flaw was. They found 41,545 servers running Supermicro’s insecure BMC software.
They also found 40,413 servers running Dell’s implementation of IPMI management software, and 23,376 servers running HP’s IPMI implementation. Although Bonkoski and his team have yet to look into the IPMI implementation on Dell’s and HP’s servers, it’s bad security configuration to allow access to these interfaces via the Internet. IPMI management interface should only be available via a private management network, not the Internet.
Good Security Hygiene
Supermicro was made aware of this report and has since published support information on their IPMI page – Best Practices for managing servers with IPMI features enabled in Datacentres – advising customers to not connect server’s IPMI interfaces to the Internet. It also suggested changing default passwords on the BMC and taking other precautious to make the type of attacks described by Bonkoski more difficult.
Prior to that, Supermicro’s use guide only “provides detailed instructions on which firewall ports to open to allow remote connections,” said Bonkoski.
Why is this happening?
Upon learning about the buffer-overflow vulnerability on BMC’s web-based management interface, an audience got enraged, saying “these are elementary mistakes that we teach our undergraduates not to make.” But why does such security vulnerability still exist in Supermicro’s IPMI implementation?
Bonkoski believes that the issue is largely cultural. BMC’s software was written by programmers from a different community from the mainstream programmers. Embedded computing programmers produce software for small, low-powered chips that runs in other cars, appliances and industrial equipment. These devices are not traditionally connected to the Internet, and so embedded software community hasn’t learned the painful lesson of security the mainstream programmers have been trained for in the last decade. Embedded computing programmers have to change their engineering culture so that security is a top priority, not an afterthought.
Reference: Researchers figure out how to hack tens of thousands of servers.