Case Study: Clusters and Image Processing, Part I

By: Dee-Ann LeBlanc
Sunday, March 17, 2002 06:47:40 PM EST
URL: http://www.linuxplanet.com/linuxplanet/reports/4101/1/

The Case

In 1998, Mark Lucas and the other folks at ImageLinks, Inc., already had a product. Forty programmers and scientists had spent 14 years building the original version for the Harris Corporation. This software was classified during the cold war and finally declassified in the early 1990s. ImageLinks spun off of Harris to apply this same technology in the commercial remote sensing market, which involves methods used to get information about the earth from somewhere other than the earth's surface.

This program is a collection of tools that work together to provide image analysis. Pictures taken by remote sensing equipment--typically from a satellite or a digital camera on a high-flying airplane--are fed into the program, mapped onto a three-dimensional model of the earth, and manipulated, merged with other images from other equipment, or who knows what else.

In the early days of ImageLinks, the software ran on a network of a dozen SGI Indigo 2 systems with R10000 processors. This network was managed over Asynchronous Transfer Mode (ATM), a networking technology that can achieve data-transfer rates of 10Gbps, using an SGI R8000 four-processor Challenge server. This collection of processors and networking was extremely high-end equipment for the time. ImageLinks had to have a computing powerhouse to accomplish such processor- and bandwidth-intensive tasks.

NOTE: An R10000 CPU is one in a series of 64-bit, MIPS-class processors. This level of CPU is traditionally used in high-end servers and mainframes. MIPS processors are also referred to as superscalar because they can handle multiple instructions simultaneously from start to finish.

Mark and some of the other technical folk at ImageLinks were experimenting with Linux at home. They began wondering how the application would run on a set of Linux boxes instead of on the SGIs. Their theory was that they would not make much of a gain. However, if the result was comparable, a move to Linux could save them thousands of dollars a year on software and licenses.

There was only one way to find out. They had to give it a try.

The Initial Experiment

After lunch one day, Mark and some of his colleagues decided to take the plunge. Rather than convincing management to take the risk, they went and bought components for a Linux box using one of their credit cards. This box was set up within their own network. Then Dave Burken and Ken Melero began the six-month side project of porting (converting) all 5,000 source code files so they would compile properly under Linux.

The Implementation

The sheer immensity of porting 5,000 files helps to explain why the Linux version of their program took so long to produce. There was another issue to deal with, however. The C++ compiler (gcc) that came with Red Hat Linux 5.2 did not fully support a feature that the ImageLinks team had used heavily--templates. For those not familiar with C++, a template provides a way of creating reusable source code. You create a generic template and then associate specific objects to that template. When a compiler that supports templates comes across object code that points to a template, it takes the data given in the code and plugs it into the template definition, quickly creating the new object.

Because the compiler did not have full template support, Dave and Ken had to try to work around this issue--not a simple task when there are 5,000 source files involved! When Red Hat 6 came out, their work became a lot easier. The version of gcc that shipped with this release supported C++ templates. Suddenly, the work went much faster.

After a total of six months' work, the port was complete. The hypothesis for this experiment was that the SGI and Intel Linux performance should be similar. The team thought the PC performance might even be slightly slower. However, if it was comparable enough, the savings in licenses and hardware would be significant enough to take the performance hit.

Imagine their surprise when they compiled the 5,000 source files and discovered that what took 12 hours to compile on an SGI Indigo 2 took only two hours on an Intel Linux box!

This immense software package is used to analyze satellite and digital aerial photos and create detailed maps that are remarkably free of traditional distortion problems. It has to handle all the numerical data in floating-point format for the best accuracy in the results.

The next step of the initial experiment involved handing the software an image and seeing how long it took to process that image. The hypothesis was that the Linux boxes would perform close to how the SGI boxes did, or perhaps a little slower.

Once again, Linux performed far better than expected. The details are examined in the next section.

The Analysis

The results were so dramatic that it was obvious to ImageLinks that using Linux gave them some sort of advantage over their previous setup. As much as Linux proponents might like to just take this evidence at face value, it is educational to take a look at why Linux turned out to be, by far, the superior solution in this situation.

The folks at ImageLinks feel that two different issues contributed to their results. First is the compiler (gcc), which provides tighter final code than the compiler on the SGI. Direct evidence of this fact is that the binary version of the application produced by gcc is almost half the size of the one produced by the SGI. A smaller binary also contributes to the programming running faster, and it makes more RAM available for other applications and loading data.

Another reason for the faster performance was the move from high-end workstations to commodity computing, which refers to using computers that most people can afford. Changing to PCs from a vendor-locked solution--where ImageLinks could not purchase upgrades or peripherals without voiding its warranty unless it did so directly from SGI--brought the price of each component down, so faster parts were economical. Specifically, ImageLinks could suddenly afford faster CPUs, hard drives, CD-ROM drives, SCSI interfaces, and more RAM.

The Full Conversion

The folks at ImageLinks were so pleased with their experiment's results that they then began converting all their image-processing and production operations to Linux. What I cover here is just a sample of the functionality that was moved from the SGI boxes to Linux:

  • Each application that loads a specific image format into the processor

  • Each sensor model specifically designed to emulate a particular satellite camera or digital camera used from a high-flying plane, including where the sensor was and how it was angled when the image was taken

  • The component that builds the three-dimensional model of the earth and then projects the two-dimensional image onto it

  • The application that does tonal and color balancing

  • The application that translates pictures to map projections

  • The image-sharpening application

  • The application that takes images brought in from different kinds of sensors--satellite or aerial photography, for example--and merges these images into one

  • The application that does brightness and contrast adjustments

  • The application that adds text overlaid onto the images

  • Each application that converts files and images to other formats

  • The application that takes multiple images that contain the same area but from different orientations and merges them properly into a larger image

  • The libraries (collections of source code that you can call from inside your own programs) used to implement parallel computing, such as clustering

Every single one of these applications now runs on Linux. In fact, the whole suite runs on a Beowulf cluster of 13 machines, built with the assistance of Dr. Salim AlAoui from Florida Tech University. This collection of programs is an ideal application for cluster technology. Performing floating-point calculations for every pixel in an image of these sizes and working within three-dimensional space is mightily CPU intensive. The software had to be adjusted so that it supported parallel computing using the aforementioned libraries.

The Beowulf cluster at ImageLinks consists of 12 single-processor 650MHz PIII slave machines, each with 384MB of RAM and 36GB SCSI hard drives, and a master machine identical to these except that it has two 650MHz PIII processors. A cluster must be interconnected so that all the machines can work together. This particular setup is networked with 100BT Ethernet switching.

Once again, experimentation paid off for ImageLinks. With each processor added to the cluster, they get another level of speed improvement. Think of the improvement in terms of how long it would take to do the job on one computer. If it would take 10 hours to process an image on one of the computers in the cluster, that same image can be processed in just one hour on a 10-computer cluster.

The Savings: Money and Time

People typically do not move to another operating system and architecture once their application is already in place unless there are some large benefits to doing so. ImageLinks made the change because of anticipated monetary and time gains. Of course, in the corporate world, time is money! So, these two different types of savings are interrelated.

The Money

The easiest savings to quantify is money. The major thing to consider here is the move from SGI hardware to commodity hardware (PCs). The trickle-down effect brought about savings in almost every aspect of the operation.

As I mentioned earlier, ImageLinks initially used SGI Indigo 2s for its image processing. This choice required that all hardware and peripherals be purchased through SGI to avoid warranty problems. Unfortunately, when you cannot comparison shop for components, more often than not these components end up being more expensive than they would for similar power and capabilities under other computer architectures. Specifically, ImageLinks went from spending $15,000 a month on leases for its SGI and Unix machines, to a one-time purchase of $25,000 for all the hardware components for its Linux setup.

When the hardware and operating system are expensive, typically the software is too. This problem occurs for a number of reasons, including the reality of a vertical market with few customers supporting the research and development efforts of the software manufacturer. Such high-end software tends to be modular, where you purchase a base package and then buy modules for different functionality. Software, modules, and licenses were running ImageLinks over $75,000 per year. Though the company created its own tools, it still required the development environment, compilers, and other components in order to build and maintain the code.

Moving to Linux allowed ImageLinks to take advantage of freely available Open Source tools for its development environment. Its software budget for 2000 was around $10,000, with no sacrifice in functionality or quality.

The Time

Just as significant as the upfront costs saved by the change is the savings in time. I already detailed that the application's compilation was reduced from 12 hours to 2 hours. There is also the issue of image-processing jobs. These were reduced sometimes from several days of computing time to just a few hours, in some cases.

All of ImageLinks's software development has also moved to the Linux platform. This change has increased programming efficiency immensely. If the development team needs the latest compiler, linker, debugger, editor, or configuration tool, it can just turn to the Internet and download the needed tool. The compile times, as I mentioned before, are also faster and produce binaries approximately half the size of the SGI compilers. Smaller programs run faster because they require less RAM, which in turn speeds up image-processing times.

To give you a feel for the kinds of time savings the folks at ImageLinks have experienced, they kindly provided me with the results of one of their test runs. In this run, they perform the following tasks for a collection of satellite images:

  • Utilize the fusing component of their software to merge all the images correctly into a single image, correcting for rotation, time of day, and properly assembling overlaps.

  • Take the resulting image and project it onto a three-dimensional model of the earth, finding its proper resting place.

  • Utilize the results of the first two stages to create a map of the area represented by the sum of the satellite images.

The total resulting output from this operation is 1GB of data. Table 1.1 details the speeds documented under three different hardware options.

NOTE: See the section "Implementing the Beowulf Cluster" in Part 2 next week for details on the cluster mentioned in Table 1.

Table 1 Image-processing job speed comparisons.

Hardware

Resulting Speed

SGI 200MHz R10000 Indigo 2

14 hours

Linux dual PIII 650MHz

4 hours, 25 minutes

Beowulf cluster

21 minutes

The Savings: Advantages

One of the biggest reasons I hear companies, organizations, and individuals mention for at least considering an Open Source solution is the lack of software vendor lock-in. Once again, although this was not the primary consideration for the ImageLinks changeover, it has become a distinct advantage. In the past, ImageLinks ran into any number of bugs in the commercial, proprietary software it purchased.

One excellent example of the value of Open Source and dealing with bugs involved wxWindows. This is a cross-platform development environment for Unix flavors, Windows versions, and MacOS. While working on software with their remote sensing Open Source group--see the section "Giving Back to the Community" for more on this--the folks at ImageLinks encountered a memory leak problem that led to segmentation faults. Because this tool was Open Source, ImageLinks was able to track down the problem and then send a fix to the wxWindows team.

NOTE: ImageLinks now uses the Open Source version of wxWindows for all its current GUI development. Doing this ensures that everything interfaces cleanly and also makes it easier in the long run to add other GUIs along the way because ImageLinks has access to all the source code.

Bugs are not the only problem with vendor lock-in. ImageLinks cannot totally dispense of its SGI machines because not all the vendors whose proprietary software it uses have Linux versions of their tools or are willing to produce them. Fortunately, some vendors were willing to take this step. Modis Solutions' (once known as Open Ware)

GUI toolkit libraries, for example, are interwoven with much of the proprietary software ImageLinks uses because this package was developed before modern design practices dictated separating the GUI from the functionality. ImageLinks might have been hard-pressed to decide whether it really wanted to allocate the resources to the immense effort of replacing those libraries. Fortunately, OpenWare responded to ImageLinks's request to provide a compiled version of its libraries that would work under Linux. This, of course, means that OpenWare can now offer this port to its other clients as well, and perhaps increase its client base. Another company, however, was not so keen on building a new version of its offerings.

ERDAS produces the Imagine remote sensing tool. Versions of this program are available for the Compac Tru64 (Digital) AlphaStation, Hewlett-Packard 9000 series 700 and 800 workstations, IBM RS/6000, Microsoft Windows 95, Microsoft Windows 98, Microsoft Windows NT 4, Microsoft Windows 2000 Professional, Microsoft Windows 2000 Server, SGI, Sun Solaris, and SunOS platforms. Unfortunately, ERDAS is not interested in producing a version of this tool for Linux. Mark Lucas points out that ImageLinks would expect to pay the same for the licensing fees for the Linux version that it does now for the SGI version, and would even be willing to assist with porting the application. Dialog about this issue continues between ImageLinks and ERDAS.

Preparing a software version for Linux does not require entering the Open Source realm. Some vendors do seem to equate the two, however. Since ERDAS does not have a version of their tool for the Linux platform, ImageLinks and others find themselves looking for either Open Source or proprietary solutions that allow them to stay with their platform of choice. Open Source will probably get precedence, however, unless the new tool is available on a number of platforms and the vendor seems amicable to porting it if necessary later. Open Source might get precedence because if an Open Source program that provides the same functionality as ERDAS exists, the folks at ImageLinks can port it to whatever platform they need on their own.

The movement to Open Source and Linux also began another set of considerations. Once the Open Source programming tools were selected, the programming team began wondering what else it could make use of for ImageLinks' needs. The two programs the company finds the most useful these days are mySQL and PHP3. Using these two programs together has allowed for the building of a custom job-tracking package that enables ImageLinks to keep track of the time and resources (and more) spent on each particular project.

Giving Back to the Community

The remote sensing field is too large for one company or organization to be able to develop a full range of top-notch tools and have them all be fully featured. After its initial experience with moving to Linux and Open Source tools, ImageLinks was pleased with the robust software that Open Source provides. It decided to merge the intellectual resources of the remote sensing community and the peer review methods of Open Source. In early February, 1999, ImageLinks paid a domain name fee of US $70 for www.remotesensing.org.

On February 5th, the Web site went live. ImageLinks sent email to two different remote sensing and image-processing mailing lists to announce the site and a new organization--Open Source Remote Sensing (OSRS). This new organization had its first external member within 15 minutes. The second announcement was placed on www.freshmeat.net, a clearinghouse of technical news and gossip. After a week, OSRS had 100 members.

The first task that OSRS took on was to gather all the existing public domain source code available for the industry. Every day people tracked down programs and libraries that met the criteria. Each of these items was added to the site, and more and more kept coming in. Commercial developers in the industry stepped up to the plate as well, perhaps seeing a way to both contribute to the community and ensure that they were a part of it--a sound business move, and one that ImageLinks itself surely must have considered. The source for some existing commercial code was handed to OSRS, and other items were written by the industry expressly with the purpose of giving them to the community through the organization.

In less than 10 days after the site launched, OSRS was large enough that it was divided into subgroups to better focus on the various aspects of the wide range of remote sensing and image-processing topics. Then, on February 15th, the organization got its first third-party exposure: Wired.com called Mark Lucas for an interview. On February 16th, the article was published. The Web site was flooded with interest, and the participants on the OSRS mailing list reached 181, with a new member joining almost every five minutes.

As you might imagine, excitement was growing at ImageLinks. Publicity grew and hits on the site continued to increase. Then, on February 17th, at 11:30 a.m., someone--perhaps a phone worker, cable worker, or construction worker digging a foundation--accidentally cut the fiber-optic data-transmission trunk in Atlanta, causing many on the East Coast to lose complete or partial access to the Internet. The OSRS Web site--and many others--was out of commission for seven and a half hours while the phone company scrambled to repair the damage. Anyone who tried to access the OSRS Web site at this time wouldn't have been able to do so. However, since the audience for this project is fairly technologically sophisticated, many of them probably heard the news that much of the East coast was inaccessible and tried back again the next day.

Once everything was back up, the OSRS organization got coverage in the magazine Wired.com. As a result, Mark received the following email (edited so the recipient's information is unavailable):

    From: XXX@XXXX.gov  
    X-Lotus-Fromdomain: EOP  
    To: info@imagelinks.com  
    Date: Tue, 16 Feb 1999 18:41:29 -0500  
    Subject: Mark Lucas  
    Mime-Version: 1.0  
    Status: RO
      
    I'd like to talk to Mark Lucas when  
    he gets a chance about a White House  
    initiative in geospatial information.
      
    XXXXXX  
    The White House  
    (v) 202-XXX-XXXX

Mark contacted the government official directly by phone and began a set of discussions and negotiations that would have widespread consequences for the entire remote sensing industry.

Growth has continued since then. Not only has full participation in www.remotesensing.org grown to at least 900 members, the organization has also received coverage in the Linux Journal and local coverage in Florida Today. Government interest has also continued. U.S. government agencies such as the National Imagery and Mapping Agency (NIMA) and the National Aeronautics and Space Administration (NASA) are following the progress made by OSRS.

The many hours of work put in by ImageLinks and all the others who have volunteered time and donated materials to OSRS was done for more than just the benefit of government agencies. This site has become the home of a remote sensing industry discussion board, an archive of images provided by Core Software, and a central point to find all the daily news events in the field. Along with the source code already collected and stored by the community, volunteer development is going on as well.

Some of the Open Source projects run through the OSRS site are listed in Table 2.

Table 2 Open source remote sensing and image processing projects.

Name

Head

Purpose

Digital Elevation Model Tools (DEMTools)

Brian Maddox, of the United States Geological Survey (USGS) Mid-Continent Mapping Center

To develop a collection of conversion programs, libraries, and tools capable of working with topographic elevation data.

GeoTIFF

Frank Warmerdam, Independent Consultant

To create the commercial standard for geospatial (digital map) TIFF files. This project actually began before OSRS was formed and then was moved by Warmerdam to this site.

Large Image Manipulation Program (LIMP)

Valient Gough, formerly of StellaCore Corporation and the author of OrthoVista (currently employed by Amazon.com)

This was one of the first projects started on www.remotesensing.org. "Large" refers to image files that are so large that they cannot fully fit into RAM. LIMP is a set of libraries used to build applications that need to process large image files. This type of library set is constantly needed in the remote sensing and GIS fields.

What Mark points out as one of the biggest benefits to the remote sensing community, however, is that the OSRS site draws together four different groups that historically have had a difficult time maintaining a dialog: academics, businesses, entrepreneur developers, and government agencies. The Web site gives these four groups a central place to discuss issues of mutual interest and to network among themselves, along with giving them all a central library for the tools they need.

If you think that you will never intersect with this aspect of graphic technology, keep in mind that companies in this field produce the maps and terrain for flight-simulation programs; images of fields for farmers showing not just a human-visible analysis, but on multiple light spectrums so the farmer can see details of what areas need water, pest treatment, and so on; images of miles and miles of land for oil companies to look for natural oil slicks; images of cities for city planners to see the reality of what the plans only suggest; and more!

Realities of Open Source

There is much talk these days about the wonders of Open Source and how it is going to change the world. Many of us truly believe that returning to the process of peer review and allowing people access to source code so they can contribute to it shows much promise. What remains to be proven in many people's eyes is the viability of Open Source in the business world. Some still see it as the domain of hobbyists who must otherwise have a "real job" in order to eat and pay the rent.

Mark Lucas was kind enough to share his thoughts and insights with me since he and ImageLinks began their Open Source initiatives. In the beginning, Mark felt that the Open Source movement signaled a healthy paradigm shift for the software industry. He believed that his company's placement as a service provider rather than a software-development house would make it less painful to blend into the movement without losing serious revenue in the process.

Certainly ImageLinks's initial entry into Open Source gave it more exposure than anyone there ever dreamed possible. This result saved ImageLinks some serious money in marketing alone. Even as a service company, however, integrating Open Source into the future of ImageLinks required an adjustment of the entire business model.

Part of the new model Mark had in mind involved trying to find a way to financially reward the key contributors to the OSRS project. Otherwise, there was always the risk that these people might have to move on to other things in order to pay the rent, put food on the table, put their kids through school, or even put themselves through school. Like it or not, money is still necessary in this world. A summary of his thoughts on how this was done specifically for ImageLinks is included in the section "Making Open Source Profits," later in this chapter.

In his 1999 writing, Mark identified a list of models to consider following along with how each might apply to ImageLinks. Table 3 details his thoughts on this issue.

Table 3 Summation of approaches ImageLinks might take in the Open Source movement.

Model Name

Model Description

ImageLinks' Version

Internet portal

An Internet portal is a Web site that, through a combination of quality, hard work, and sheer luck, is considered the best place to go for its topic. This position can be leveraged into advertising revenue or used in other creative ways, and it's an excellent marketing tool.

Become the single, best point of access for finding any software and source code or obtaining services related to remote sensing and geographic information systems (GIS). Advocate for these groups' interests with the U.S. government.

Red Hat

This model involves packaging Open Source tools for download or convenient purchase, providing user-friendly installation programs, building user-friendly manuals and training programs, selling service contracts, and building new Open Source tools to share with the community.

Package and sell Open Source remote sensing and GIS tools, public domain data sets, user-friendly documentation and training, and customer support. When charging a price low enough that it is an affordable alternative to having to download and package--and perhaps burn onto a CD-ROM--the tools themselves generate sales.

Virtual development company

A virtual development company is a company that is not really a company. There may be a small company at the core, but the rest involves subcontracting work out so that the company's capabilities are never limited by its core size.

Contributors to OSRS would have the option of being on-call contractors, brought in for jobs that suit their talents and time availability. This would even be possible by subcontracting through the company the person works for if that person is not his or her own boss, which would increase that company's support of the OSRS initiative as well.

Today, Mark Lucas believes more strongly than ever that a paradigm shift--a fundamental change in the way things are approached and done--toward the Open Source way of doing things is occurring. However, he has run into two different problems in implementing his own solutions:

  • Many managers and investors cannot seem to make the shift away from a focus on intellectual property and patents.

  • Many Open Source advocates have a bias against business and profit-seeking.

Mark, along with many others, believes that it is inevitable that a way around these two issues will be found. Either that or the issues may slowly evaporate over time as implementations of Open Source in business demonstrate to both sides that this is not only a viable way of doing business, but it's still true to the philosophies of Open Source. Until then, folks such as Mark have to constantly pause and analyze their approach from, at a minimum, the two angles mentioned.

One model that does work for ImageLinks--one that was not listed in the table mentioned earlier--is bringing customers into the Open Source fold. As I mentioned earlier, some of the biggest attention OSRS received from the very beginning (in early 1999) was from the U.S. government. Throughout 10 or 12 meetings, Mark had to address issues around this weird phenomenon of allowing people to see the source code. Some of these concerns revolved around configuration control, security, and development methods.

The government contacts had the impression that Linux and Open Source projects were handled somewhat randomly, with people just editing things as they saw fit. They wanted to ensure that the documentation and source code were tracked in a controlled fashion. Considering that Open Source teams quite often use management tools such as the Concurrent Versions System (CVS), this concern was actually not a problem at all. This has become one of the greatest Internet benefits for the Open Source movement--instantaneous, global version tracking and bug reporting engines such as Bugzilla.

Another worry revolved around security. The primary worry was that a malicious programmer might add a Trojan horse or a harmful bug into the mix. However, it is difficult to pass something like that past the entire Open Source community or project team. The entire team would have to be behind the effort. ImageLinks was not likely to comply with allowing someone to sabotage their work in such a manner. In fact, because many people participate in Open Source projects for the reputation and pride of doing so, it is more likely that a troublemaker who tries to introduce problems into the package will be banned from the project than it is that the code will be negatively tampered with.

People often pour over Open Source code to make sure it is elegant, as programmers like to call it. It is actually easier to hide Easter eggs, Trojan horses, and other such things in "closed source" code because so few people lay their eyes on the code.

There was also a concern about how to incorporate traditional software-development requirements into the mix. Proprietary software-development methods often begin with a series of meetings that involve drawing up the specifications that the software needs to meet and then doing all the development at once. An Open Source project tends to work a little more freeform, involving smaller steps. Development is done in phases, with each phase's specifications laid out. Also, rather than having a series of large, long meetings, Open Source development teams tend to be in constant communication through email discussion lists. Email has become more popular in the corporate culture as well as in the Open Source culture because of a number of factors.

People can choose when in the day they want to read their email, so they tend to keep it for moments when they are not heavily concentrating on some task. Also, people tend to think out a bit more what they are going to say when composing email. Not only is the concern there that your words are written in stone, but it is hard to take something back when you sent it out there for all to read and save. Also, folks simply have more time to think when writing email than they do when talking in person or on the phone. They can develop their thoughts into a coherent fashion, laying them out so they make good sense.

Finally, in October 1999, the government invited ImageLinks to make a presentation at the Multi-Modal Image Fusion conference--hosted by Kodak for the U.S. government--and show off what it is capable of. The presentation consisted of two key parts. The first was about ImageLinks, itself, and where it saw the commercial end of its business headed, and the second was about ImageLinks's initial experiments with Open Source and how it was hooked on both the idea and results they gained with Open Source--and how, in turn, it expected the government to benefit as well. In December, after further meetings, the decision was made to put ImageLinks under contract.

In April 2000 the details were finally hashed out. The government contracted ImageLinks to begin the Open Source Image Map Experiment (OSSIM), which Mark claims is pronounced awesome. The goals for this project were considered impossible to accomplish with the resources allocated and traditional software-development methods. ImageLinks was to build a library of applications with the following capabilities:

  • Importing a variety of digital map file formats

  • Exporting a variety of digital map file formats

  • Containing algorithms for processing images

  • Building map projections

  • Implementing geodesy technology, which allows for locating things accurately on the earth

  • Building three-dimensional models

  • Implementing photogrammetry, which allows for properly projecting digital images onto the earth's surface

  • Recognizing geological features

  • Checking and balances for making sure assessments are accurate

  • Running statistical analyses

Certainly all these tasks could be done using any software-development method. The twist was this: The entire project had to be completed within nine months, with enough funding for only three full-time programmers. Notes were to be kept about which Open Source methodologies worked and which did not, especially in the area of utilizing Open Source while following strict government requirements. Mark confesses that he felt that they may have bitten off more than they could chew, but ImageLinks plunged ahead.

Six months into the project, Mark is still confident but feels they have definitely learned some important lessons along the way. For one thing, he notes that Open Source projects often do not gain momentum until there is something in place to work with--some kind of alpha version of a product or key component. OSSIM requires starting from scratch. ImageLinks addressed this problem by using its internal resources to pull together the infrastructure of a base product that the rest of the project could be built off of. The concern was that with only three allocated programmers, this would be by far the hardest part of the project.

However, it turned out that a lot of Open Source code was already available to speed their efforts. ImageLinks started by drawing from OSRS and other locations with LIMP, PROJ4, NIMAMUSE, TIFF, GeoTIFF, and mathematics and science libraries. According to Mark, Open Source creates a whole new problem! "In fact, there is so much code available, it is often difficult just selecting which major project we want to use to implement a given function." For the GUI, ImageLinks stuck with wxWindows, partially because it enables platform independence.

So far, things are chugging along smoothly. Even though programmers are not beating down the door at ImageLinks to help specifically on OSSIM, there are Open Source teams working on many projects that benefit OSSIM anyway. This is one of the immense benefits of Open Source. ImageLinks can concentrate on its core competencies while utilizing the results of other people working on what they do best. So, OSSIM pulls from other Open Source efforts, and its own code goes back out to the Open Source community, furthering things for everyone. The programmers that did show up are all experts in the remote sensing and GIS fields. Are you seeing the trend here? It comes down to core competencies. Everyone works on what they enjoy, are good at, and are qualified for. The programmers all draw from the same base of top-notch tools and libraries. Because every tool here is interrelated, the entire set works well together

. Also, because everyone is doing what they're good at, the product is ultimately better.

NOTE: Some might point out that this can also be a weakness of Open Source. If no one is interested in a project because it's perceived as too boring, it can be hard to get things done, even if it involves building a core tool that other more interesting items will be able to use.

After just three months of work, ImageLinks was just about ready to release the first practical tool build through the OSSIM project. The next phase involved seeing whether this project would really attract the critical mass necessary to get the next six months of work done. Fortunately, the answer is an unequivocal yes. After another three months, ImageLinks accomplished all of its original goals. ImageLinks was able to release the first set of tools utilizing the OSSIM libraries:

  • The beta for the OSSIM library.

  • An image viewer capable of taking in multiple map projections, in multiple image formats, and displaying them properly together. You can then run your mouse over the images and see the exact latitude and longitude you are looking at relative to the Earth's surface.

  • A dozen command-line tools that utilize the libraries.

  • The company also added a full time technical writer to the OSSIM staff. This person's job is to not only document the libraries, but also write manuals for both the users who need to work with these tools and the programmers who might want to alter or add to them. Part of the job now is to get the word out that the OSSIM tools and libraries are available, and teach developers what these items are capable of. The government has also added another list of tasks. The OSSIM team now needs to:* Take 2-Dimensional images and map them onto the Earth's surface, including getting the 3-Dimensional elevations correct.

  • Model the lens distortion and focal length properties of a satellite or aerial camera.

  • Take 2-Dimensional images and build 3-Dimensional models from them.

  • Build an interface that allows the Geographic Resources Analysis Support System (GRASS, www.baylor.edu/~grass/) to utilize the OSSIM libraries.

One of the largest benefits for ImageLinks, itself, is that the many hours of time the programmers spent explaining Open Source to managers and investors--time that would have been better spent doing "real work"--has been replaced with solid productivity. Because the customer is footing the bill, the programmers simply do their job. The members of the programming team are so motivated that they even spend some of their evenings and weekends working from home. As Mark points out, it is not because they are feeling "under the gun," but it's that they are enjoying the project.Who knows where things will go from here. At the very least, OSSIM will leave behind world-class remote sensing and GIS tools for the community. ImageLinks will be able to leverage its experience in this matter in its support and services. The hope, however, is that as the project succeeds, it will lead to further projects and experiments. As the U.S. government gets more excited about Open Source possibilities, the success of OSSIM has the potential of benefiting the rest of the Open Source community as well.

Summary of The Case

What's Missing?

ImageLinks has not been able to fully move to Linux as yet. Not only does ImageLinks have to keep SGI machines around for some legacy software situations, it also uses PCs running Microsoft Windows for administrative and financial tasks. Although ImageLinks is able to use Samba to share files across the network with the Windows PCs, it would like to be able to move these operations to Linux as well.

Recommended Links and Resources

ImageLinks recommends the following online resources:

  • www.remotesensing.org

  • www.beowulf.org

  • www.php.net

  • www.mysql.com

  • www.wxwindows.org

  • www.mozilla.org/bugs/source.html

It also recommends these books:

  • The Cathedral and the Bazaar, by Eric Raymond. Cambridge, Massachusetts: O'Reilly, 1999. ISBN: 1-56592-724-9 (if you are interested in learning more about the Open Source revolution).

  • Crossing the Chasm, by Geoffrey A. Moore and Regis McKenna. New York: Harperbusiness, 1999. ISBN: 0-06662-002-3. This is an excellent book for dealing with paradigm shifts.

In Next Week's Article...

The case for ImageLink's conversion has been laid out and examined. In Part 2 next week, we'll examine some of the technical solutions ImageLink found when it implemented its plan, including:

  • Porting the Application

  • Implementing the Beowulf Cluster at ImageLinks

  • Implementing an LVS Cluster with Red Hat Linux

  • Making Open Source Profits

  • Creating a Kickstart Automated Installation File for a Cluster

  • Creating an NFS Installation Server

  • Implementing LVS Clusters with ssh

Copyright Jupitermedia Corp. All Rights Reserved.