Buurst SoftNAS Shares: A Use Case of How we Consolidated EMC VNX File Servers in the Cloud for Easy Access & Sync
Since our early days in 2013, SoftNAS® has seen hundreds of customers move out of their on-premises and colocation datacenters into the cloud. Today, we see an even sharper increase in the number of customers leaving their EMC VNX and other traditional NAS file servers behind, choosing to consolidate and replace aging hardware storage systems with cloud-based NAS file shares. The business impetus to make the change often begins with an upcoming hardware maintenance refresh cycle or a corporate decision to move some or all of its applications into the cloud.
Of course, the users continue to require access to their file shares – over the LAN/WAN from the office and via VPN connections while traveling and working remotely.
How to consolidate file servers for on-premises users into the cloud – a use case
One of the first issues that comes up is how do we seed tens to hundreds of terabytes of live production data from VNX file shares, where it’s actively used today, into the cloud? And then how do we maintain synchronization of file changes during the migration and transition phase until we’re ready to flip the DNS and/or Active Directory policies to point to the cloud-based shares instead?
In this use case, we’re showcasing the implementation of a solution for a well-known media and entertainment company, with dozens of corporate file shares.
A hybrid cloud solution
The initial Seeding Phase involves synchronizing the data from many VNX-based file shares into the cloud. As shown in Figure 1 below, the customer chose a 1 Gbps Direct Connect from the corporate data center to the AWS® VPC for dedicated bandwidth.
The AWS Direct Connect link was used initially for the migration phase, and now provides the high-speed interconnect from the corporate WAN for site-to-site access to the corporate file shares. Later, it became the primary data path connecting the corporate WAN with AWS and the file shares (and other applications hosted in AWS).
As shown above, a SoftNAS Platinum VM was created from a VMware OVA file and operated locally on VMware in the corporate data center. SoftNAS Platinum supports an Apache NiFi-based facility known as FlexFiles.
First, the CIFS shares on the VNX were mounted from SoftNAS. Another copy of SoftNAS Platinum was then launched on AWS® as the VNX NAS replacement. A storage pool was created, backed by four 5-terabyte EBS disk devices, configured in a RAID array to increase the IOPS and performance, and provide the necessary storage space.
A thin-provisioned SoftNAS® Volume was created with compression enabled. Data compression reduced the 20 TB of VNX data down to 12 TB. This left more headroom for growth and since the volume is thin-provisioned, the storage pool’s space was also available for other volumes and file shares that came later.
A SnapReplicate® relationship was created from the on-premise SoftNAS VM running on VMware® to the SoftNAS running in AWS. SnapReplicate performs snapshot-based block replication from the source node to the target. Once per minute, it accumulates all changes accumulated since the last snapshot, then replicates just those block changes to the target system. This is very efficient, and it includes data compression and SSH encryption.
Next, the SoftNAS team created several NiFi data flows which continuously replicated and synchronized the VNX CIFS share contents directly onto a local SoftNAS-based ZFS on Linux filesystem running in the same datacenter on VMware. These flows ran continuously, along with SnapReplicate, to actively seed and sync the VNX files share with the new SoftNAS Cloud NAS filer running in AWS.
After this phase was completed, the on-premises SoftNAS node was no longer required, so the SnapReplicate connection was deleted, leaving a copy of the VNX file share data in the cloud. Then the SoftNAS node was removed from VMware.
During the final phase of the migration, various user communities had to be migrated across dozens of file shares. To maintain synchronization during this phase, the FlexFiles/NiFi flow was moved to SoftNAS Platinum running in AWS, as shown in Figure 2 below.
During a several week period, different departments’ file shares were cut over to use the new consolidated cloud file server. Throughout that period, any straggling changes still arriving on the VNX were picked up and replicated over to SoftNAS in AWS. After all the file shares were verified to be operating correctly in AWS, the VNX was decommissioned as part of the overall datacenter shutdown project.
Successful consolidation of EMC VNX file servers
This project was on a tight timetable from the start. The entire project had to be developed, tested and then used to migrate live corporate file shares from on-premises to AWS in a matter of 45 days in order to stay on schedule for the datacenter closedown project. The project was completed without impacting the user community, who didn’t see any differences in their workflow and business due to where their file share data is hosted.
Find the right fit
Corporations are increasingly choosing cloud hosting for both data and applications. As maintenance contracts come up for renewals of popular EMC VNX, Isilon®, NetApp® and many others, customers are increasingly choosing the cloud over continuing to be in the data center and hardware business. The customer is faced with a fork in the road – stay on the hardware treadmill of endless maintenance, capacity upgrades and periodic forklift replacement – or – move it to the cloud and let someone else worry about it for a change.
Buurst SoftNAS Platinum provides multiple avenues for data migration projects like this one, including the strategy used for this project. In addition to FlexFiles/NiFi and SnapReplicate, there’s also an end-to-end Lift and Shift feature that can be used to migrate both NFS and CIFS from virtually anywhere the data sits today into the cloud. SoftNAS also operates in conjunction with Snowball in several configurations for situations involving hundreds of terabytes to petabytes of data.
Request a free consultation with our cloud experts to identify the best way forward for your business to migrate from hardware storage to cloud-based NAS.
