Any ideas of the best offsite backup method for our company's data

[greavette]

Hello,

This isn't a comment against OpenVPN at all which is why I didn't post this in the OpenVPN forum. I'm hoping I can get some ideas on the best method to backup our companies data.

I've been using Untangle in our small company for 5 months now...truly a wonderful product! I'm now in the process of setting up a method for off-site backup for the company's important accounting files and whatever else they want to backup. I've setup an OpenVPN client connection on their Untangle Router and I can connect from my location to their router. I also run Untangle at my home where I will backup the data, but I haven't setup either Untangle as a VPN Server.

The data I'm transporting is sometimes quite large...hundreds of thousands of Meg in size which take around 2 hours to transfer. I've noticed with OpenVPN that at some point during the transfer the connection is broken and I have to reconnect. Instead of a VPN connection to transfer, I've now switched to connecting through SSH and using Remote Desktop to connect and transfer files. This is good for now, but I would like to automate a process to run overnight instead and not have to worry about the connection being broken.

Does anyone have ideas on a method I should use? One method I thought of was SSH (SFTP) with rysync.

I would prefer a free method to keep down costs, but this is critical business data and we are not against spending money either.

Thanks in advance for any advice you may give.

[chuckjuhl]

Jungledisk is a backup program that uses Amazon's S3 for storage. Very inexpensive. We've been using a version since it was Beta and have had no problems. http://www.jungledisk.com/

[chuckjuhl]

I generally don't recommend doing remote backups to an entity's own network. IMO, remote backup is something that is best hosted by an outside vendor with multiple redundant storage facilities. However, a quick, free and secure way to backup or move data from a remote location to another location is to set up a virtual VPN using LeafNetworks or Hamachi, map the remote data location as a drive, then use a free program like Karen's Replicator to do scheduled replications. Karen's Replicator will only copy over the changed files, so once the initial replication is completed, the scheduled replications will usually take much less time to complete. The benefit of using a virtual VPN is ease of installation at both the remote location and the central location. You can Google Leafnetworks, Hamachi, and Karen's Replicator to learn more about each. All three are free.

[sky-knight]

I haven't tried Karen's Replicator, but it sounds like Pathsync. I've got several scripts with Pathsync in the middle to do unidirectional directory syncs.

http://www.cockos.com/pathsync/

Beyond that I agree, this is a service to pay for. Also, make sure you read the docs carefully. Most of these systems are simply normal backup applications buried in an SSL internet delivery. What I have seen is most people tend to just stuff incremental backups in there... doesn't do you any good if you don't have a full backup somewhere to increment from.

[Silver Bullet]

Just cron a rsync script and make sure you use "--partial" so that if the connection is broken, you can resume the transfer.

Maybe something like so with a log file:
rsync -r -v --progress --partial --stats -e ssh user@1.2.3.4:/dir/to/backup /dir/of/stored/backups >/dir/of/stored/backups/backup.log

then you could tar directory in the same script with a date in the name with something like:
tar czf backup_$(date '+%F_%I%M%P').tar.gz /dir/of/stored/backups

then you should probably just move them to an archive directory to keep from tarring other tars in your working directory
mv /dir/of/stored/backups/backup* /backup/archives/

I guess you could also tar the stuff you want to backup before transferring the data with rsync.

[chuckjuhl]

I haven't tried Karen's Replicator, but it sounds like Pathsync. I've got several scripts with Pathsync in the middle to do unidirectional directory syncs.

http://www.cockos.com/pathsync/

Karen's Replicator is a simple windows-based unidirectional replication program; not as complex as pathsync, nor as difficult to set up rsync. I was assuming the original poster was looking for a simple, easy to configure solution that would work with a VPN connection and that did not require any scripting or even any additional configuration to his untangle installation. Using a virtual VPN like Leafnetworks or Hamachi also eliminates issues with OpenVPN.

Of course, I'm also making an assumption that the data servers and workstations are running a Windows OS, which may not be an accurate assumption. But if it is, I believe the combination of using a virtual VPN with Karen's Replicator would do the trick well in this situation. LeafNetworks (http://www.leafnetworks.net/)also has a GUI client for Linux. If the data to be backed up is on a Linux server, then this solution will still work if the remote computer is Windows-based. If the remote computer is running a Linux OS, then the virtual VPN is still useable, but the backup software will need to be a Linux program - something like the Simple Backup Configuration Program (http://simplelinuxbkup.sourceforge.net/) would probably be an easy and flexible Linux solution to pair with a virtual VPN.

As a consultant, I'm not inclined to recommend to clients solutions that require any significant scripting, or command-line interaction where a suitable GUI application is available that they themselves can (hopefully) understand.

eyond that I agree, this is a service to pay for. Also, make sure you read the docs carefully. Most of these systems are simply normal backup applications buried in an SSL internet delivery. What I have seen is most people tend to just stuff incremental backups in there... doesn't do you any good if you don't have a full backup somewhere to increment from.

Valid point. However, most offsite backup services (Mozypro, Jungledisk, etc.) don't really work like traditional backup programs. They work more like Karen's Replicator or pathsync. Once you configure your source files, the first run will be a "full" backup of the source files. Subsequent scheduled runs will only copy over changes, with the option of archiving older versions for a specified period of time or number of archive copies. I particularly like Jungledisk because it has clients for Windows, Mac and linux, and the low cost per gb for storage.

[PROACTIVENS]

This is a program that I have been playing with recently. I am launching an online backup program for my clients as part of my MSP offerings. This product seems to work fairly well, and you can point it across the VPN or the internet. I have been using MSP edition, so I am not too well versed on the feature set of the professional edition, which is what you would be working with. It's only $30 per pc, $60 per server, per year, so its not too expensive.

http://www.vembu.com/storegrid/network-backup.html

It would be a good idea to have a site to site connection with openVPN rather than using a client/site VPN.

Hope it helps.

[SirBC]

I use 2 QNAP NAS boxes (509 pro) that do encrypted Rsync backups nightly. You can do the initial backup onsite so that if it is multi-gigs it won't take all that long, then move one box offsite and do nightly incrementals. It's a set it and forget approach, works for me.

[lucidtek]

And interestingly enough, I found an S3 variant: Secobackup.

Apparently, this is free, except that you go through them for the Amazon service, and you pay $0.20/GB instead of 15, but it's free, and it even has a reseller version. The GUI leaves much to be desired, but the features are interesting enough. I signed up this evening, and I'm testing it. It looks promising, especially if it ever fixes that GUI.

It allows encryption as well as local storage:

Network Backup & Versioning
In addition to Amazon S3, you may also setup a network drive (share) for a backup on your local network. In addition to keeping a backup of the latest version, network backup also features versioning. Previous versions of files are retained for 7 days.

Some of you other guys need to try this out so I'm not the only one. :-P

Oh, you're gonna have to enter your CCard info and create an Amazon account just to set it up...and it charged me $0.23 right off the bat, but I figure it's worth 10 bucks or so moving data around this month to test it.

http://www.secobackup.com/

*update*

The GUI is killing it for me. Also, Jungledisk has a reseller program as well, and is planning on implementing VSS as well as 64bit support.

[greavette]

Thanks to everyone for the responses!

The Server where their data is stored has Raid 10 configuration on it with a battery backup on the raid controller and a Battery backup unit as well. I (as their offsite location) have a D-Link 323 NAS device setup as Raid 1. My plan was to use some kind of method to securely backup their data to the NAS, and from there transfer the data on a semi-regular basis to DVD. I do appreciate the need to sometimes pay for something especially when we are talking about a companies financial data. I think what I'll do is recommend that they use a commercial source for their important financial data and other data be backed up to my house...of course if the commercial solution is not all that expensive then I might as well backup everything they deem important.

Thanks again for all the advice!

[cheaman]

I've been playing with this lately:

http://backuppc.sourceforge.net/

It's open source and does deduplication. There's also a pre-built virtual machine (on Cent OS) of this available.

http://gotitsolutions.org/2007/01/15/open-source-backup-and-data-de-duplication-virtual-appliance-2.html