[Micronet] UCBackup Infrastructure Changes

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[Micronet] UCBackup Infrastructure Changes

Jack Shnell

All,

This message is to inform UCBackup customers about recent changes in the infrastructure supporting their backup and recovery service.  Since April, the UCBackup team has been in the process of replacing the physical tape library that has provided the primary storage for backup data since 2003, with disk array based Virtual Tape Libraries (VTL). 

As backup data storage onsite expanded beyond 250 terabytes (TBs) and peak daily volume to more than 7 TBs, the physical tape library became increasingly inadequate to support the load, even with extensive front-end disk caching.   Recovery of more than a few files at one time also became an extremely time consuming process in many instances.

VTL technology allows provisioning many more virtual tape drives for the backup infrastructure than would be practical or affordable with physical tape drives.   This greatly increases the number of client backup sessions that can run in parallel, thus improving overall throughput, while applying hardware-based compression to data stored on the virtual tapes, reducing overall storage costs.  The ability to mount many virtual tape volumes simultaneously also vastly accelerates multiple-file data recoveries under TSM.

Migration from the old physical tape library onto the new permanent disk array required assembling a temporary VTL environment from storage distributed across the IST Storage Area Network (SAN).  Maintenance and expansion of this temporary environment during the two-month conversion process unfortunately caused accessibility issues on several occasions for many clients, both for backups and recoveries.

The UCBackup team acknowledges the concern that a number of customers communicated during these service interruptions.  We expect and encourage this kind of input to help us determine the scope of problems and the most effective response.  We regret that we were unable to make the necessary infrastructure conversion process transparent for UCBackup customers as we hoped.

The team completed preparation of the permanent backup storage infrastructure the last week of May.  At that time, daily backup processes were all switched over to the new VTL, and migration of active data remaining in the temporary VTL environment and in the physical tape library commenced.   This last phase of the conversion is proceeding well, and we expect completion by early July.  We’ve seen a substantial increase in overall throughput, with just a few residual connectivity issues that have been resolved.  We already have reports of folder-scale recoveries completing successfully in minutes, rather than the hours that might have been expected with the old infrastructure.  In addition to service improvements, we anticipate that the efficiencies of the new infrastructure will lead to a reduction in the recharge rate.

Please continue to send your questions and observations regarding the UCBackup service to [hidden email], we depend on this to focus our efforts for improvement.  You can also send email concerning specific node issues to [hidden email].

 

Jack Shnell

Supervisor, Storage and Backup Group

IST Infrastructure Services


 
-------------------------------------------------------------------------
The following was automatically added to this message by the list server:

To learn more about Micronet, including how to subscribe to or unsubscribe from its mailing list and how to find out about upcoming meetings, please visit the Micronet Web site:

http://micronet.berkeley.edu

Messages you send to this mailing list are public and world-viewable, and the list's archives can be browsed and searched on the Internet.  This means these messages can be viewed by (among others) your bosses, prospective employers, and people who have known you in the past.