Data Center into AWS Virtual Private Cloud

SoftNAS^® allows The Street to Move Out of Own Data Center into AWS Virtual Private Cloud

After suffering numerous outages and business impacts due to shortcomings of clustered file systems, The Street now have a commercial-grade NAS storage and disaster recovery solution in place for their AWS-based IT infrastructure. Applications like Oracle, custom Java, video streaming, Drupal, and another line of business applications are now running fast and reliably. In addition, near real-time block replication to a different availability zone provides the disaster recovery and redundancy required.

TheStreet.com and the company’s other websites and applications are now fully operational in the AWS cloud using SoftNAS.

Results

• Applications like Oracle, custom Java, video streaming, Drupal, and other line of business applications are now running fast and reliably.
• In addition, near real-time block replication to a different availability zone provides the disaster recovery and redundancy required.
• TheStreet.com and the its’ other websites and applications are now fully operational in AWS using SoftNAS.

Clustered file systems do not work or scale as expected in the cloud.

When the decision was made to move out of its own data center, where the traditional EMC appliance was no longer available, IT needed something else… The Street originally chose a popular clustered file system to deploy on Amazon EC2. In practice, the promise of the “scale out” clustered file system architectures just did not work as expected.

The Street began with three EC2 instances running the clustered file system. Within weeks, they were down to one instance, due to “split-brain” issues, where the different instances would get out of sync with each other, then self-healing didn’t work, and it would eventually take the Apache webservers down and then everything went offline. Worse yet, since it was community-supported software, “when something breaks, nobody cares…,” support was a major issue. And because The Street has millions of files, file replication can’t keep up – it just chugs along trying to compare and replicate all the files, and can’t finish – sort of like painting the Golden Gate Bridge – by the time you eventually finish, you need to start over again.

At this point, the businesses, website, end-user, and customer impacts became unbearable, so a better storage solution for Amazon Web Services was demanded.

The Solution – An enterprise-grade Cloud NAS designed for Amazon EC2.

After suffering numerous outages caused by the clustered file system failures and shortcomings, The Street’s CTO found SoftNAS and recommended it to IT. An IT consultant was tasked with installing and testing SoftNAS. It was installed and configured on EC2 with elastic block storage (EBS) configured as RAID 10 for best IOPS, throughput, and redundancy. A second SoftNAS instance was configured in a different availability zone (AZ) as a failover target, in case of any outage or equipment failure in the primary AZ.

Then, RSYNC was used to migrate data off the remaining clustered file system node to SoftNAS.

Next, SnapReplicate™ was configured. Snap Replicate uses “block replication” instead of file replication. This means the only data transmitted from the primary SoftNAS controller node to the secondary in the other AZ are those data blocks that changed.

Block replication scales to handle millions of files. Once per minute, snapshots pick up the data blocks that changed during the last minute, then replication sends the changed data blocks. Scheduled file system snapshots were also configured to provide several weeks of recovery points, plus hourly snapshots, to quickly recover any corruption or other potential data integrity issues from lost or deleted files.

Annual Cost – $1,990 for SoftNAS, $9,095 for AWS

For 1 terabyte of usable, replicated storage, the redundant controller SoftNAS solution on Amazon EC2 costs $1,990 per year (or $198 per month), including support and maintenance. The Amazon Web Services costs for such a system is approximately $757.96 per month ($9,095.52 per year) for 2 Large EC2 instances running SoftNAS, including 4 TB of EBS volume storage.

This does not include the customer’s EC2 workload instances, elastic load balancing, bandwidth, volume snapshots for backups, or other AWS service charges that may be included in an overall solution.