Enterprise Storage Unleashed: Why We Invested in SwiftStack

We’re excited to welcome SwiftStack, a leading software-defined object storage platform leveraging OpenStack Swift, into the OpenView family after leading their $16m Series B. You might be asking yourself, what exactly does being a storage platform that is software defined, object based, and leveraging OpenStack Swift really mean? The answer is enterprise storage, but with limitless scalability.

Exponential Growth of Unstructured Data is Transforming Enterprise Storage Requirements

According to IDC, there were 4.4 zettabytes of digital data at the end of 2013, set to grow to 44 zettabytes by 2020. To put this into perspective, that’s equivalent to 472 million copies of the Library of Congress or 562 trillion high resolution photos growing by 10x in the next six years — 85% of which is managed by enterprises.

So, what is all this data?

The majority is unstructured data — the videos, documents, music, engineering designs, medical images, log files, and other digital content that we and the growing number of internet-connected devices like sensors and cameras create and consume in our personal and professional lives.

Enterprises have been approaching unstructured data storage using traditional array-based storage technologies, characterized by tightly integrated software and specialized hardware components, but when it comes to maintaining an organization’s digital content at this scale, these systems become cost prohibitive.

Software Defined Storage Decouples Storage Management from Hardware

Being software defined means that the responsibility for the system to receive, maintain, and transmit data is held in its software rather than specialized hardware components. For example, software-defined storage systems accept failures as inevitable and work through and around those failures using techniques such as data replication and erasure coding, instead of trying to prevent failures in the first place.

This fundamental abstraction of storage management from hardware allows organizations to run production storage on low-cost, commodity boxes that occasionally fail, but are horizontally scalable at very low cost. The modularity of these systems allows IT managers to incrementally add components to scale as needed, and can even simplify the process of migration to a new hardware vendor as whole systems can be composed of mix and match parts.

These characteristics allow software defined storage systems to drive the operational efficiencies necessary for enterprises to operate at petabyte scale.

Object Storage is the Natural Form of Unstructured Data

The vast majority of electronic files accessed on a daily basis are stored in file systems — much like you might arrange your documents and applications on your own computer. The way traditional enterprise storage arrays find and access data is quite similar to you clicking through tiers of files to find the document you are looking for and then saving the latest version of that in the same spot — the system traverses a hierarchical tree structure, or file system, to store and retrieve data.

The internet, with its decentralized architecture and massive scale, introduced a new way to store and retrieve data — as objects. An “object” is simply a file along with all its associated metadata and a globally unique identifier so that is can be easily retrieved without being stored in a file system — unstructured data in its natural form.

For example, imagine having to click through a hierarchical tree structure of IP addresses every time you wanted to access a website or web application. Not only would it be very slow process, but simply categorizing and inserting new data logically into the tree would be a Herculean task. Instead, you search for websites using meta-data about the site such as its name, topics that are likely to appear as content on the site, or a myriad of other defining characteristics. Similarly, object storage systems identify and retrieve data based on their meta-data and scale, and share the horizontal scaling properties of internet itself.

Taken together, the cost-effective redundant and highly scalable natures of software defined object storage allows enterprises to create a self-contained storage systems that address the exponential growth of unstructured data and span beyond a single rack, networking switch, or datacenter.

OpenStack Swift is the Open-Source Standard for Object Storage

Swift was originally developed as the engine behind Rackspace Cloud Files in 2010 and open-sourced as part of the OpenStack project. Since then, Swift has become the most popular object store in the world after Amazon S3 and powers the storage clouds of IBM (SoftLayer), HP, Oracle, AT&T, and many others in addition to that of Rackspace.

Swift’s popularity comes from the fact that is delivers on the promise of a scalable software defined object storage system designed to store large amounts of unstructured data at low cost, running on clusters of commodity servers with no central point of control, and offering built-in object replication for data durability. This means that it can scale from a few nodes and a handful of drives to thousands of machines with tens of petabytes of storage while exhibiting no single points of failure. By being open-source, it also prevents dreaded vendor lock-in to the likes of EMC or NetApp.

While these characteristics make Swift and ideal choice for enterprises looking to manage the deluge of unstructured data heading their way, as with many open-source technologies, standing up a production system at scale is beyond the reach of most enterprise IT departments.

SwiftStack Brings the Limitless Scalability of OpenStack Swift to the Enterprise

SwiftStack brings the limitless scalability of open-source Swift to the enterprise by providing seamless onboarding, management, and security through a software defined storage controller. Rather than corralling open-source code, or even waiting weeks to be able to rack and test traditional array-based systems, storage administrators can set up a trial cluster on a few commodity boxes — or even their own laptop — by simply installing the SwiftStack controller and be up and running in minutes.

The enterprise reliability of SwiftStack’s controller and the proven cost effective, redundant, and highly scalable nature of Swift are a perfect match for the needs of enterprise storage in the face of exponentially growing unstructured data today. Moreover, the SwiftStack team is the same group that wrote Swift inside Rackspace, and who continue to lead the open-source project today. We couldn’t be more excited to partner with them in their mission to revolutionize enterprise storage to meet the demands of data growth.

Have questions about SwiftStack? You can learn more at SwiftStack.com.