Technical Perspective: Disk Array Models For Automating Storage Management

Large disk arrays are everywhere, even if we, as end users of computer services, rarely notice them. When we shop at an Internet retailer, the product and account data come from a disk array in a data center. Our email, banking, payroll, insurance, and tax data all reside on disk arrays. The hardware used is typically diverse, obtained from multiple vendors at different times. Depending upon an application’s requirements for throughput, response time, availability, and reliability, its data may be distributed across disk arrays and even across multiple data centers and made resilient to failures through replication or the addition of error correction codes.

Managing disk array storage is extremely complex, involving tasks such as initially placing the data, arranging for data backup, prioritizing data access so that each application can receive the performance it requires, periodically migrating data from one disk array to another, monitoring performance, and diagnosing any performance problems found. Some partially automated tools can assist the operator, but in the end, storage management is a manually intensive process. As a result, management decisions are geared toward simplifying implementation, rather than optimizing application performance.

The key to reducing costs and improving the performance and dependability of storage is to automate the management tasks because computers can keep track of complex environments and intricate decisions better than human beings. However, the management system must understand the behavior of the storage system it manages. For example, when a new database server is to be installed, where should its data be placed? Would one of the existing disk arrays in the data center suffice, or is a new one needed? An automated management system can consider numerous options to make an informed decision, but it must be able to predict the performance impact of each option. In other words, the management system needs a model to answer the question "How will my applications’ storage performance change if I take this option?"

Building accurate performance models of storage systems has long been a stumbling block to designing automated storage management systems, because one needs to be able to build models, quickly and easily, for the multitude of disk arrays in use, and for a wide variety of workloads. While models of basic disk drives for simple workloads are known, most data centers use disk arrays, which are much more complex because they aggregate a number of disks with cache and control firmware. Earlier disk array performance models either were hand-built for each disk array model and required extensive tuning for good accuracy, or were based on benchmark measurements of a few workloads on the device. In either case, the models were only accurate for workloads similar to those used to build the models.

To address this problem, Michael Mesnier, Matthew Wachs, Raja Sambasivan, Alice Zheng, and Gregory Ganger have proposed a new approach, called relative fitness modeling. Rather than directly building a performance model for each disk array, the authors suggest it is easier to characterize the difference in performance between disk arrays. These models, built by measuring the differences in performance between a given pair of arrays for a representative set of workloads, are shown empirically to apply to a larger set of workloads. Then, if we have a relative fitness model for the differences between two arrays, and we know how a given workload performs on the first array, we can predict the performance of the workload on the second. This scenario is common. For example, a user may have measured the average I/O response time of an application on an existing array; if the disk array vendor can provide a relative fitness model of the differences between the user’s existing disk array and a newer one, the I/O response time of the application on the new array can be predicted.

The relative fitness method is an important step in modeling the performance of disk arrays, but many challenges persist. In particular, disk array models must be able to predict accurately the performance of an arbitrary combination of workloads, given the increasing trend of storage consolidation in the data center (that is, storing multiple application data sets on the same disk array), and this problem remains open. The success of the relative fitness method gives us hope that similar techniques can be used to predict the performance of workload combinations; this is an active area of work for storage systems researchers.