Purdue doubles the work life of computers with overtime
WEST LAFAYETTE, Ind. — Computers are lazy — they only work when they have to, and almost all of them spend most of their time loafing.
|
"A corporation's CFO sees that computers are 1,000 times more powerful than they were 15 years ago and wonders, 'Why aren't we producing more with them?' If you think about it for more than 90 seconds it doesn't make sense," says Gerry McCartney, Purdue's interim vice president for information technology and chief information officer. "At Purdue, we're harvesting computer downtime and putting it to good use."
At Purdue more than 4,300 computers of all sizes — from desktop machines used by students to do homework and check e-mail, up to large, powerful research computers — are linked together in what is known as a pool. If a computer anywhere in the pool becomes available, even for a few minutes, a waiting job is sent to it for processing.
To enable this type of distributed computing, Purdue uses a version of an open source application called Condor, which was developed for use by scientists and engineers at the University of Wisconsin-Madison. Open source software, such as the operating system Linux and the Firefox browser, are collaborative, non-commercial methods to develop software. Many businesses, including investment banks J.P. Morgan Chase and UBS, and semiconductor manufacturer Micron Technology, are users of Condor, too.
Michael Ryan, chief technologist of the compute backbone for JPMorgan, says computing cycles, which are sometimes referred to as MIPS (an acronym for "millions of instructions per second") are a valuable investment.
"A MIP is a terrible thing to waste," Ryan jokes. "On Wall Street, we are unique in that how many calculations can you get done in an eight-hour window can mean gigantic savings for the bank in real dollars. The more accurately you can model your risk, the more money you can save your company."
McCartney says organizations can benefit from using distributed computing.
"Almost everyone who has a desktop computer has thought, 'These things are more powerful than a mainframe computer from the 1970s and all I use it for is e-mail and to do word processing. I wonder if it could be used to do more?' And there is a way to do that," he says. "Any university or business should learn to do this in order to maximize their resources.
"People are looking past this technology because it's not sexy. Technology people are interested in new machines and 'big iron' computers. But distributed computing is the future, whether it's sexy or not."
Even at a buzzing major research university, computers are idle about half the time. There are nights and weekend periods when the machines are available, and there are periods during the day when no jobs are running.
The computers in the Condor pool at Purdue are used roughly 45 percent of the time for their intended purpose and 45 percent to run Condor jobs. They get to take a computing coffee break for the other 10 percent of the time.
Miron Livny, a professor of computer science at the University of Wisconsin-Madison, developed Condor and says that because it is open source software, it is difficult to determine how many businesses are using it.
"I can only tell you those who went public with it," Livny says. "Many corporations don't like to talk about their use of Condor for obvious reasons."
Livny says businesses are attracted not only by the utility of the software, but also by the advantages of open source delivery. "The companies appreciate the support they get from the Condor community, and the fact that they can hold a copy of the software."
Ryan says even corporations are willing to participate in the open source movement.
"Wall Street, overall, is more comfortable using open source tools, and banks on Wall Street are increasingly looking at using Condor," he says.
Distributed computing using systems such as Condor are more effective on some types of problems than others. Condor supports high-throughput computing, in which millions of simple jobs can be assigned to a large number of machines.
"Some corporations, such as those in the insurance or financial industries, run hundreds of thousands or millions of jobs on a routine basis," Livny says. "Distributed computing allows them to do this with the resources they already own. It allows them to do things at the scale that they need to do to compete in an ever evolving market."
One significant benefit of using Condor is that it reduces the need to expand large data centers or machine rooms that house centralized computers at many corporations and universities.
"These days, it's often not the computers that dictate how much computing we can do, but the data centers and their lust for electrical power and air conditioning," McCartney says. "The price of computer cycles is falling, but the price of powering and cooling all of those cycles is going up fast enough to make your head hurt."
Purdue's Rosen Center for Advanced Computing publishes a daily graph showing the university's Condor usage.
Writer: Steve Tally, Information Technology at Purdue, (765) 494-9809, tally@purdue.edu
Sources: Gerry McCartney, (765) 496-2270, mccart@purdue.edu
Michael Ryan, (212) 622-4191, mike.ryan@jpmorgan.com
Miron Livny, (608) 262-0856, miron@cs.wisc.edu
Purdue News Service: (765) 494-2096; purduenews@purdue.edu
GRAPHIC CAPTION:Purdue University has been able to double the output of nearly 5,000 of its computers by using a system that captures computer cycles when the computers aren't being used. The Purdue system includes the open source software Condor and, in many weeks, the amount of computing done using the Condor system (shown in blue on the chart) equals or exceeds the work being done on the computers by their regular users. (Purdue University graphic/Ty Filby, Information Technology at Purdue)
A publication-quality graphic is available at https://www.purdue.edu/uns/images/+2006/condorgraph.jpg
To the News Service home page