Storage Networking , Part 1 eBook: A storage network is any network that's designed to transport block-level storage protocols. But understanding the ins and outs of networked storage takes you deep into several of protocols. This guide covers SANs, Fibre Channels, Disk Arrays, Fabric, and IP Storage. » Storage Networking 2, Configuration and Planning eBook: Picking up where Part 1 left off, Part 2 of our look at storage networking examines configurations for SAN-attached servers and disk arrays, and also includes a look at the future of IP storage. » Storage Management Costs in the Enterprise: A Comparison of Mid-Range Array Solutions Whitepaper: Many factors contribute to the ownership cost for enterprise storage. These include (but are not limited to): physical capacity relative to physical space requirements, performance capacity for data transfer and system reaction time, software maintenance and updates, expandability and flexibility, and much more. » Storage Is Changing Fast  Be Ready or Be Left Behind PDF: The storage landscape is headed for dramatic change, thanks to new technologies like Fibre Channel over Ethernet (FCoE), pNFS, object-based storage and SAS that will affect everything from NAS and SANs to disk drives. Get the knowledge you need to make the most of your storage environment, now and in the future. » HP StorageWorks EVA4400 Demo: Dont settle for an expensive and complex array that lacks functionality. The HP StorageWorks EVA4400 delivers virtual storage with enterprise class functionality at an affordable price. »

When NetLedger went shopping for an operating system to host its online-accounting ASP, it was looking at building a system from the ground up, with the only requirements that the end result support Oracle database servers in a robust operating environment.

The end result was a server installation incorporating Linux servers from VA Linux, encryption accelerators from Rainbow Technologies, and network file servers from Network Appliance. The data center is the largest deployment of Oracle database servers on Linux servers in the world, incorporating off-the-shelf technology in an unusual manner.

That Linux should have survived the rugged evaluation process devised by NetLedger CIO Dave Durkee isn't surprising to Linux aficionados, but may come as a surprise to UNIX gurus who still are unconvinced of the power of Linux when compared to high-performance UNIX servers. However, after a series of hands-on evaluations that generated real-world benchmarks, the combination of Linux on a Intel box best served the needs of NetLedger.

"We didn't have a preconceived notion about what operating system to use," says Dave Durkee, CIO of NetLedger. "We were looking for something with an attractive cost/performance ratio, and we needed a lot of computing power."

NetLedger is an ASP that offers online accounting tools. There are three levels of service offered, ranging from free accounting tools for individuals to high-level tools for professional accountants, although the focus at all times is toward small businesses that don't require the services of an Arthur Andersen or Price Waterhouse.

The initial process began with evaluations of Intel-based operating systems (Windows NT, Linux, Solaris) and big and small Hewlett-Packard and Sun Microsystems UNIX servers. "We evaluated all of these and decided that the UNIX servers just weren't going to fill our needs: when there were a few users online the performance was spectacular, but when you increased the number of users the performance seriously lagged," Durkee says. "Plus, you just can't scale those machine as well as PCs--you can add a jillion small machines if you want at a lower cost and scale that way."

Comparing a Netra T1 (running an UltraSPARC 440) with an 500 MHz Pentium III Intel box yielded a surprising solution: the Intel box outperformed the Netra, even though Sun deployed a staff of UNIX gurus to fine-tune the UNIX box. "The Intel box was much more robust than the Sun architecture," Durkee says. "It set our direction on Intel."

The final decision, then, was the operating system to deploy on the Intel architecture: Solaris, Windows NT, or Linux. A key requirement was that the operating system robustly support Oracle (Oracle's Larry Ellison is a key investor in NetLedger, and the accounting system used by end users is based on Oracle). Oracle Corp. was an early supporter of Linux, so it's not a surprise that the testing showed that Linux excelled at Oracle support and performance.

"Quite honestly, Oracle didn't run as well on Solaris as on Linux," Durkee says. "And we decided early on that we didn't need the graphical layer used in Windows NT--for us, managing the servers can be done by telnetting in, so there is a lot of waste associated with NT."

With the elimination of NT and Solaris, NetLedger worked with VA Linux Systems, Rainbow Technologies, and Network Appliance to deploy a 500-machine network and launch the ASP. This network can handle "millions" of users, Durkee estimates, and is very scalable: as most capacity is needed, more PCs (from any vendor, Durkee adds) can be added.

The end result, Durkee says, is perhaps the most untraditional architecture found in any Oracle/Linux deployment, as several components are deployed in different ways. "It's off-the-shelf stuff configured in a weird way," Durkee says.

For starters, Oracle servers like to have data files stored on local servers, rather than deployed across the network, but Durkee and the NetLedger staff chose to deploy data on separate Network Appliance NFS servers, with much of the active data stored in cache. This decision both increased performance (the caches are actually faster than local storage when it comes to data retrieval, increasing system performance by 20 to 25 percent, according to Durkee) and introduced system redundancy in the system. "If you have a single point of failure on an Oracle server, you can lose a lot of data," Durkee says. "With the NFS servers, they mirror one another, so you have a high level of system redundancy."

The introduction of Rainbow Technologies of CryptoSwift EN 1000 e-commerce accelerator cards for data encryption also boosted system performance, decreasing CPU load on each PC by 50 percent and actually cutting down on the number of Intel servers needed to serve NetLedger users.

Finally, Linux itself needed to be reconfigured to work on this scale. First, Linux itself usually chokes on large files, while it's quite normal to have two-gigabyte files on an Oracle server, they are rare in the Linux world. Secondly, "NFS support is weak, especially so on Red Hat Linux," Durkee says. "I told VA that I wanted Linux to be as good as Solaris in these areas, and they took care of us with enhancements to the Linux kernel," enhancements that will be released in future kernel releases.

"Using an Open Source solution was really key to this," he says. "We could draw on the talents of a large group of people. With Sun or Microsoft, you run into problems and are told that they'll be addressed in the next release of the operating system. When you ask them when this next release will appear, they say in a year or two, and until then you have no solution or recourse."

Positioning for future growth is essential to this server configuration, as more NetLedger services are planned.

The end cost of the data center, when all associated costs are factored in (Cisco routers, Oracle database-server licenses), was somewhere between $3 million and $5 million--a relatively small sum for a 500-PC ASP network installation.