We were fortunate to get early access to $COMPANY’s internal Object Storage which provides an AWS S3 compatible interface and allows uploading and download files over web services. It’s was cheap (blue dollars) and you pay for actual usage, not for reserved space! This price includes Disaster Recovery and High Availability as the implementation makes use of erasure encoding and stores the files across 3 data centres.
The great thing about it being AWS S3 compatible is that you can use the standard AWS libraries in your code for integration. It also moves the integration up to the Application layer so no issues with getting NFS mounts setup on your servers.
Looking ahead, this would also better position us for running on OpenShift. which was in an early stage at that time.
At the time, we had a lot of data in Mongo DB and a lot of data archived to LTFS so we could not go with a big bang switch over. Management also had some concerns over the performance of this new storage.
There was also the development hurdle that our current implementation was tightly coupled to MongoDB.
After some refactoring I had a solution which allowed us to switch between Mongo or Object Storage for active write using a configuration setting. For retrieval of files it supported reading from Mongo, Object Storage and LTFS which meant that we did not need to do any data migrations prior to release.
As a bonus, during the refactoring I was able to remove a Mutex Lock from our Node.JS code, which improved performance of file uploads.
Performance of Object Storage was actually quicker than Mongo!
Once we were satisfied that Object Storage was working as expected we began the migration of files from Mongo DB and LTFS. This was achieved through automated scripts and took a number of weeks to churn through all the data. Due to the cheap cost of Object Storage there was no longer a need to archive data to LTFS so this was decommissioned.
