Case Study: Clusters and Image Processing, Part I - page 4
People typically do not move to another operating system and architecture once their application is already in place unless there are some large benefits to doing so. ImageLinks made the change because of anticipated monetary and time gains. Of course, in the corporate world, time is money! So, these two different types of savings are interrelated.
The easiest savings to quantify is money. The major thing to consider here is the move from SGI hardware to commodity hardware (PCs). The trickle-down effect brought about savings in almost every aspect of the operation.
As I mentioned earlier, ImageLinks initially used SGI Indigo 2s for its image processing. This choice required that all hardware and peripherals be purchased through SGI to avoid warranty problems. Unfortunately, when you cannot comparison shop for components, more often than not these components end up being more expensive than they would for similar power and capabilities under other computer architectures. Specifically, ImageLinks went from spending $15,000 a month on leases for its SGI and Unix machines, to a one-time purchase of $25,000 for all the hardware components for its Linux setup.
When the hardware and operating system are expensive, typically the software is too. This problem occurs for a number of reasons, including the reality of a vertical market with few customers supporting the research and development efforts of the software manufacturer. Such high-end software tends to be modular, where you purchase a base package and then buy modules for different functionality. Software, modules, and licenses were running ImageLinks over $75,000 per year. Though the company created its own tools, it still required the development environment, compilers, and other components in order to build and maintain the code.
Moving to Linux allowed ImageLinks to take advantage of freely available Open Source tools for its development environment. Its software budget for 2000 was around $10,000, with no sacrifice in functionality or quality.
Just as significant as the upfront costs saved by the change is the savings in time. I already detailed that the application's compilation was reduced from 12 hours to 2 hours. There is also the issue of image-processing jobs. These were reduced sometimes from several days of computing time to just a few hours, in some cases.
All of ImageLinks's software development has also moved to the Linux platform. This change has increased programming efficiency immensely. If the development team needs the latest compiler, linker, debugger, editor, or configuration tool, it can just turn to the Internet and download the needed tool. The compile times, as I mentioned before, are also faster and produce binaries approximately half the size of the SGI compilers. Smaller programs run faster because they require less RAM, which in turn speeds up image-processing times.
To give you a feel for the kinds of time savings the folks at ImageLinks have experienced, they kindly provided me with the results of one of their test runs. In this run, they perform the following tasks for a collection of satellite images:
- Utilize the fusing component of their software to merge all the images correctly into a single image, correcting for rotation, time of day, and properly assembling overlaps.
- Take the resulting image and project it onto a three-dimensional model of the earth, finding its proper resting place.
- Utilize the results of the first two stages to create a map of the area represented by the sum of the satellite images.
The total resulting output from this operation is 1GB of data. Table 1.1 details the speeds documented under three different hardware options.
NOTE: See the section "Implementing the Beowulf Cluster" in Part 2 next week for details on the cluster mentioned in Table 1.
Table 1 Image-processing job speed comparisons.
SGI 200MHz R10000 Indigo 2
Linux dual PIII 650MHz
4 hours, 25 minutes
Solid state disks (SSDs) made a splash in consumer technology, and now the technology has its eyes on the enterprise storage market. Download this eBook to see what SSDs can do for your infrastructure and review the pros and cons of this potentially game-changing storage technology.
- 1Linux Top 3: CoreOS, Oracle Enterprise Linux 7 and Ubuntu 14.10
- 2Linux Top 3: Debian Dumps SPARC, Ubuntu Takes Over Linux 3.13 and the Core Infrastructure Initiative
- 3Linux Top 3: Fedora, Ubuntu and Gluster Lose Community Leaders
- 4Red Hat Enterprise Linux 7 Finally Hits the Big Time
- 5Linux Top 3: Tails 1.0, OpenMandriva Lx 2014.0 and Debian 7.5