Cellar v2: even more S3 compatibility
In November 2018, we released the Cellar C2 cluster. Every new cellar account since then uses the new cluster. Previously created add-ons remain on the old one.
Cellar?
Cellar is our S3 add-on. You get the exact same API than S3, but the service is hosted by Clever Cloud. So you can use it with any S3 client. Even the official Amazon ones.
To achieve that, we use an open source implementation.
The differences between the old and the new Cellar
That's a good question. See, at the beginning, we created the first Cellar add-on using Riak CS (then called Riak S2). Riak CS is an S3 API implementation on top of the Riak distributed NoSQL database by Basho.
We used it for quite some time. Then Amazon changed its authentication model, deprecating its Signature v2 algorithm in favor of the new Signature v4. We waited for Basho to update its implementation. Then Basho decided to quit maintaining Riak CS. So no new fancy Signature v4 for us.
With time, Amazon killed Signature v2 by rolling out new AWS APIs clients. It became clear we had to do something.
We had a look at different OSS project and decided to use Ceph.
Ceph is a distributed storage platform that comes with many interfaces:
- Object Store, supporting the S3 and Swift APIs.
- Network Block Device, something we were already interested in.
- Ceph FileSystem, a feature we used at the very beginning of Clever Cloud. It left us
with some mixed feeling. We replaced it quickly.
Cellar C2 is now running on Ceph Object Store.
What does that mean, feature-wise? Well, let's compare it together:
| Feature | Riak CS | Ceph Object Store |
|---|---|---|
| List Buckets | ✓ | ✓ |
| Delete Bucket | ✓ | ✓ |
| Create Bucket | ✓ | ✓ |
| Bucket Lifecycle | ✓ | |
| Policy (Buckets, Objects) | • | ✓ |
| Bucket Website | ✓ | |
| Bucket ACLs (GET, PUT) | ✓ | ✓ |
| Bucket Location | ✓ | |
| Bucket Notification | ||
| Bucket Object Versions | ✓ | |
| GET Bucket Info (HEAD) | ✓ | ✓ |
| Bucket Request Payment | ✓ | |
| PUT Object | ✓ | ✓ |
| PUT Object (Copy) | ✓ | ✓ |
| DELETE Object | ✓ | ✓ |
| GET Object | ✓ | ✓ |
| Object ACLs (GET, PUT) | ✓ | ✓ |
| HEAD Object | ✓ | ✓ |
| POST Object | ✓ | |
| Copy Object | ✓ | ✓ |
| Multipart Uploads | ✓ | ✓ |
| Object Tagging | ✓ | |
| Signature v2 | ✓ | ✓ |
| Signature v4 | ✓ |
The most important feature we gained was obviously the Signature v4 one.
My Cellar add-on is old. Do you plan to migrate it?
For the moment, no. There is no easy way to move from Riak to Ceph. We plan to do it when we get the time. Until then, here is the way to do it:
s3cmd -c old_s3cfg sync s3://bucket/ tmp/
s3cmd -c new_s3cfg sync tmp/ s3://bucket/
Feel free to ask the support (by email or via the console chat) for help and advice about moving your Cellar Bucket!
Julien Durillon
Dev & ops @ Clever Cloud since 2010. I'm responsible for the invoices your receive 😬
