2 comments on “Making Object Storage do more
  1. Tim Wessels says:

    Well, SwiftStack has been weak when it comes to supporting the AWS S3 API. This is probably because OpenStack Swift prefers its own API. Nothing wrong with SwiftStack supporting the Swift API, but the S3 API is the de facto standard for object storage.

    SwiftStack 4.0 with Elasticsearch is a nice add. Caringo already does the same thing using Elasticsearch. Cloudian will include Elasticsearch and Kibana in its next release later this year. Cloudian did show its use of Elasticsearch and Kibana in a demo at the most recent Storage Field Day. Having Elasticsearch integration will become table stakes for object storage vendors going forward.

    SwiftStack Drive is optional with SwiftStack 4.0, and you can do the same thing using CloudBerry Lab Drive with practically every object storage vendor or service out there.

    A built-in load balancer is a nice add in SwiftStack 4.0. Including a load balancer represents a nice savings for SwiftStack customers. Other object storage software vendors who recommend using load balancers when operating at scale should do the same.

    Here are some real questions about SwiftStack 4.0, which were not addressed in this post by Mr. Crump. Has SwiftStack addressed its scalability issues related to the use of SQLite for metadata storage and rsync for replication? Does SwiftStack 4.0 still rely on a non-P2P architecture which makes Swift harder and more costly to scale? Is erasure coding finally working in SwiftStack 4.0?

    Inquiring minds want to know.

  2. Thanks for the questions. I work for SwiftStack and would be glad to address these topics.

    SwiftStack, along with others in the community, have been focused over the last 18-24 months, on ensuring that all of the necessary S3 features are covered for developers and are now moving on to the more niche functions. Since I work with 3rd party software vendors on supporting both the S3 and Swift API, I can tell you that today, I can hardly think of any commercial software implementing more than just basic CRUD (create, read, update, delete). I’ve given several talks at industry events to try to expand developers use of the rich API that Object Storage allows. It will come, but for now, SwiftStack and our competition support 99% of the commercial applications on the market using S3, because these applications are just not using some of the really cool stuff.

    SwiftStack Drive to your point does the same thing as Cloudberry Lab’s Drive, Mountain Duck and other 3rd party applications. I fully endorse these products as well. You will see that Cloudberry is a SwiftStack partner, we appear in their dropdowns, and we work with their entire suite of products, supporting both S3 and Swift API. SwiftStack Drive gives value to customers that want a single throat to choke.

    When you talk about SQLite for Metadata storage I believe you are confusing another issue. SwiftStack has always stored metadata with the object in a completely durable fashion, and not relied on SQLite. SQLite is simply used for listing of objects in containers. SQLite has presented issues in the past with the number of objects that a bucket can handle. The number of objects that can be held in a bucket / container has steadily risen with each release. Rackspace spoke at OpenStack Tokyo about users with a billion objects in a bucket, while I gave a talk in Austin detailing performance numbers up to 50 Million objects. SwiftStack allows for millions of user accounts each having millions of buckets, with each bucket containing millions of objects.

    Below is a link to my talk:
    https://www.openstack.org/videos/video/swift-102-beyond-crud-more-real-demos

    SwiftStack still utilizes rsync to move data between nodes. This is done in a P2P fashion and always has been. If you watch my talk above, you’ll see that new improvements now allow a drive to drive P2P movement of data. While rsync is a tried and tested utility, we are continuously investigating ways to increase efficiency of how data is replicated between nodes in SwiftStack.

    Erasure coding in SwiftStack is GA and shipped in our 3.0 release in October 2015. Several of our customers have deployed it in production. Again I think you might be confusing the community launch of Erasure Coding in OpenStack Swift with the release in SwiftStack. The community release was labeled as beta and thus SwiftStack did not release a commercial version for several months after the community release. SwiftStack licensed the Intel ISA-L Erasure Coding library and we do not use the open source EC Libraries that the Openstack Swift deploys.

Comments are closed.

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 22,047 other followers

Blog Stats
  • 1,444,791 views
%d bloggers like this: