I discovered Toucan a while ago and think it’s a great program that is very handy when working on the move.
I’m trying to keep a non zip backup of the entire server with over 150GB of data. I would like to keep a true mirror copy of the files. Any files changed should be moved to an archive location before syncing with the main destination.
It would be good if the archive directory have the same structure as the source to prevent files with the same name overwriting.
Can anyone help me with this script, if it can be done!
Visit the Community page
Join our forums
Subscribe to our email newsletter
Subscribe with RSS
Follow us on Facebook
Follow us on LinkedIn
Follow us on Twitter
Follow us on Mastodon
Follow us on BlueSky
for the slow reply on this, I am not exactly sure what you want, could you perhaps explain what you are after in some more detail, using a small number of file for example.
I would like to keep a mirror backup of the project files, but also keep a copy of any old files that have been updated, this will allow me to return to a previously updated file after running a backup
Master
E:\Project\cost\cost.doc
E:\Project\cost\list.doc
E:\Project\cost\Order list.doc
Mirror Backup
X:\ backup\Project\cost\cost.doc (file updated - old file moved to archive folder before updated)
X:\ backup\Project\cost\list.doc (file NOT updated)
X:\ backup\Project\cost\Order list.doc (file updated - old file moved to archive folder before updated)
Archive
X:\ archive\12-11-10\Project\cost\cost.doc (old copy of file)
X:\ archive\12-11-10\Project\cost\Order list.doc (old copy of file)
You should script a double backup plan.
Set up the Mirror and the Archive copies manually initially.
First sync the Mirror to the Archive, and then Master to the Mirror.
The Archive will always be one version old, and the Mirror will match the master and be all new.
Edit: You could set up several generations. Presently, I script rename Backup7 to Backup8, Backup6 to Backup7, etc, etc, Backup1 to Backup2, and then finally perform the new Backup1, and delete Backup8. In this manner, I keep several generations and never have more than 7 onhand.
neutron1132 (at) usa (dot) com
That’s a good plan! The information I’m backing up is over 100GB having 8 versions archived would be massive.
I was thinking only archiving the files that have been changed, cutting the copied data by a massive amount.
The way I see the mirror script working is …
Compares the master with the backup - if different overwrite with new. If only I could move/copy the file to another location before overwriting would be great.
… maybe too complicated to script.
First I told you what to do on your system, then I told you about the system that *I* have to show you a practical application of such an idea.
Obviously, you don't have to set up exactly the system that I have.
And even more obviously, you can use the SYNC feature and not the BACKUP feature.
...definitely not too complicated to script.
neutron1132 (at) usa (dot) com
He could also do a differential backup of the mirror backup and then do the mirror sync could he not? Either way, yours is a smart and viable solution.
He could use differentials, but I personally hate them. I just don't like to hunt around for something if I need to manually restore only one file. That's why I keep all the distinct, entire backups of important files (such as Thunderbird profile/mail, Firefox profile, and my directory of financial spreadsheets). The entire scripted backup cycle takes just over a minute. Why risk any issues with differentials when doing the whole thing is so clean and easy?
neutron1132 (at) usa (dot) com
Well that depends on the OP and how much space he has to play with, what mediums he's dealing with and I suppose to a certain extent how much time he has to play with. 100GB is no small chunk of data. I agree though, that when you start messing around differentials it makes the restoration process a little more challenging, and although it's something that you (hopefully) won't need to do very often, when you do need to do it you need it done as quick as possible. The most important thing is, as you've demonstrated, there are options.
I suppose that comparing 100GB to a flash drive seems like a lot, but if he's handling that amount exclusively on a hard drive or to a second (internal or external) hard drive, it isn't so bad.
With a sync, if only a handful of files changes, then the process is very quick and doesn't stress the available connection, no matter how big the entire collection may be. The only requirement is to have the initial space available in however many versions desired... Master, Mirror and Archive. That's a total of 300GB. With today's hard drives, the price of a pizza dinner will get enough extra space if needed.
Ah, I guess we're getting into theory instead of the mechanics of making Toucan do the trick (although I do enjoy the discussion, thanks).
neutron1132 (at) usa (dot) com
Yeah, I'm probably over thinking it on account of being in "work mode". So I'm thinking it terms of tape loads, RAID and clustered file systems etc. You're right, the average joe with a flash drive and a desktop computer is not going to bat an eye at 100gb and they wouldn't care less if it took all night (mostly).
Hooray for derailment!
All the Master information is stored on a server and moving large amount of data can be slow, the original Backup took over a week running it during the working day.
Thanks for your help will just have to work round our problem
Merry Christmas when I comes
Is the master server being accessed during the day? Why can't you run the backup at night? Is it a database you're trying to backup, or just a plain file system? A week sounds an incredibly long time, is this for mirroring though? It sounds like you need some fundamental architectural changes to facilitate backing up such large volumes of data.
That was for the very first backup from nothing. Our server is on the otherside of the country.
We have about 100 staff accessing random files about 5-10MB each, might only need to archive 1GB of data every week or two so copy the whole 100GB as an archive is a bit over the top.
Thanks