What's the Big Deal with Small Files? Scaling Internet Applications for the Cloud

Session Date and Time: Day 3 March 25, 12:30pm EST (9:30AM West Coast) (45min)
Keywords: small files, storage, file serving infrastructure, storage software, scalable infrastructure.
Authors: Gary Orenstein, MaxiScale.
Abstract: Cloud applications such as social networks, photo- and video-sharing websites and blogs have radically changed the dynamics of file sharing and storage. Millions of people now access and post billions of small files such as photos and video thumbnails in unpredictable patterns, in turn creating big problems for storage and data center managers trying to manage these applications with inadequate infrastructures. For example, YouTube had 1 billion views a day, while Apple's iPhone App Store experienced 2 billion downloads and AdMob served 127 billion ads. In situations such as this conventional file systems fall short: they rely on a single metadata server and they cannot economically scale for such an unpredictable workload.

In response to growing demand, industry giants such as Amazon.com (Dynamo), Google (GFS), Yahoo! (MObStor) and Facebook (Haystack) built their own infrastructures. These select few were forced to develop "home-grown" solutions since there were no compelling technology options readily available. Plus, these companies had the resources, million-dollar R&D budgets and large engineering teams to support this new paradigm. But now, all companies can benefit from new file serving and storage solutions that address these issues.

Join Gary Orenstein, vice president of technical solutions for MaxiScale, as he explores the infrastructure needs for modern cloud-scale applications and the inherent challenges with legacy systems. Additionally, Gary will show storage and data center managers how to find the right file system, integrate it within existing cloud infrastructures and develop an infrastructure that can handle the most optimistic expectations for rapid growth and scale.