You are here

Home » Forums » Support Forums » Utility Apps Support » Toucan

Toucan non-zip backup

14 posts / 0 new
Log in or register to post comments
Last post
November 23, 2010 - 11:36am
#1
Iain
Offline
Last seen: 12 years 3 months ago
Joined: 2010-11-23 11:07
Toucan non-zip backup

I discovered Toucan a while ago and think it’s a great program that is very handy when working on the move.
I’m trying to keep a non zip backup of the entire server with over 150GB of data. I would like to keep a true mirror copy of the files. Any files changed should be moved to an archive location before syncing with the main destination.
It would be good if the archive directory have the same structure as the source to prevent files with the same name overwriting.

Can anyone help me with this script, if it can be done!

Top
December 7, 2010 - 3:11pm
Steve Lamerton
Steve Lamerton's picture
Offline
Last seen: 9 years 6 months ago
Developer
Joined: 2005-12-10 15:22
Sorry

for the slow reply on this, I am not exactly sure what you want, could you perhaps explain what you are after in some more detail, using a small number of file for example.

Top
December 13, 2010 - 6:18am
(Reply to #2)
Iain
Offline
Last seen: 12 years 3 months ago
Joined: 2010-11-23 11:07
Archive Backup

I would like to keep a mirror backup of the project files, but also keep a copy of any old files that have been updated, this will allow me to return to a previously updated file after running a backup

Master
E:\Project\cost\cost.doc
E:\Project\cost\list.doc
E:\Project\cost\Order list.doc

Mirror Backup
X:\ backup\Project\cost\cost.doc (file updated - old file moved to archive folder before updated)
X:\ backup\Project\cost\list.doc (file NOT updated)
X:\ backup\Project\cost\Order list.doc (file updated - old file moved to archive folder before updated)

Archive
X:\ archive\12-11-10\Project\cost\cost.doc (old copy of file)
X:\ archive\12-11-10\Project\cost\Order list.doc (old copy of file)

Top
December 13, 2010 - 2:11pm
(Reply to #3)
J Neutron
Offline
Last seen: 1 week 6 days ago
Joined: 2008-06-10 19:26
Twice as much fun

You should script a double backup plan.

Set up the Mirror and the Archive copies manually initially.

First sync the Mirror to the Archive, and then Master to the Mirror.

The Archive will always be one version old, and the Mirror will match the master and be all new.

Edit: You could set up several generations. Presently, I script rename Backup7 to Backup8, Backup6 to Backup7, etc, etc, Backup1 to Backup2, and then finally perform the new Backup1, and delete Backup8. In this manner, I keep several generations and never have more than 7 onhand.

neutron1132 (at) usa (dot) com

Top
December 14, 2010 - 4:55am
(Reply to #4)
Iain
Offline
Last seen: 12 years 3 months ago
Joined: 2010-11-23 11:07
too much of a good thing

That’s a good plan! The information I’m backing up is over 100GB having 8 versions archived would be massive.
I was thinking only archiving the files that have been changed, cutting the copied data by a massive amount.

The way I see the mirror script working is …
Compares the master with the backup - if different overwrite with new. If only I could move/copy the file to another location before overwriting would be great.

… maybe too complicated to script.

Top
December 14, 2010 - 7:32am
(Reply to #5)
J Neutron
Offline
Last seen: 1 week 6 days ago
Joined: 2008-06-10 19:26
Please reread the post!

First I told you what to do on your system, then I told you about the system that *I* have to show you a practical application of such an idea.

Obviously, you don't have to set up exactly the system that I have.

And even more obviously, you can use the SYNC feature and not the BACKUP feature.

...definitely not too complicated to script.

neutron1132 (at) usa (dot) com

Top
December 14, 2010 - 7:54am
(Reply to #6)
Darkbee
Darkbee's picture
Offline
Last seen: 2 years 10 months ago
Joined: 2008-04-14 09:41
Two Jobs, One Script

He could also do a differential backup of the mirror backup and then do the mirror sync could he not? Either way, yours is a smart and viable solution.

Top
December 14, 2010 - 9:18am
(Reply to #7)
J Neutron
Offline
Last seen: 1 week 6 days ago
Joined: 2008-06-10 19:26
Yeah

He could use differentials, but I personally hate them. I just don't like to hunt around for something if I need to manually restore only one file. That's why I keep all the distinct, entire backups of important files (such as Thunderbird profile/mail, Firefox profile, and my directory of financial spreadsheets). The entire scripted backup cycle takes just over a minute. Why risk any issues with differentials when doing the whole thing is so clean and easy?

neutron1132 (at) usa (dot) com

Top
December 14, 2010 - 9:49am
(Reply to #8)
Darkbee
Darkbee's picture
Offline
Last seen: 2 years 10 months ago
Joined: 2008-04-14 09:41
Agreed

Well that depends on the OP and how much space he has to play with, what mediums he's dealing with and I suppose to a certain extent how much time he has to play with. 100GB is no small chunk of data. I agree though, that when you start messing around differentials it makes the restoration process a little more challenging, and although it's something that you (hopefully) won't need to do very often, when you do need to do it you need it done as quick as possible. The most important thing is, as you've demonstrated, there are options.

Top
December 14, 2010 - 11:13am
(Reply to #9)
J Neutron
Offline
Last seen: 1 week 6 days ago
Joined: 2008-06-10 19:26
Not that much

I suppose that comparing 100GB to a flash drive seems like a lot, but if he's handling that amount exclusively on a hard drive or to a second (internal or external) hard drive, it isn't so bad.

With a sync, if only a handful of files changes, then the process is very quick and doesn't stress the available connection, no matter how big the entire collection may be. The only requirement is to have the initial space available in however many versions desired... Master, Mirror and Archive. That's a total of 300GB. With today's hard drives, the price of a pizza dinner will get enough extra space if needed.

Ah, I guess we're getting into theory instead of the mechanics of making Toucan do the trick (although I do enjoy the discussion, thanks).

neutron1132 (at) usa (dot) com

Top
December 14, 2010 - 11:29am
(Reply to #10)
Darkbee
Darkbee's picture
Offline
Last seen: 2 years 10 months ago
Joined: 2008-04-14 09:41
Okay, I'm done now!

Yeah, I'm probably over thinking it on account of being in "work mode". So I'm thinking it terms of tape loads, RAID and clustered file systems etc. You're right, the average joe with a flash drive and a desktop computer is not going to bat an eye at 100gb and they wouldn't care less if it took all night (mostly). Smile

Hooray for derailment! Wink

Top
December 15, 2010 - 4:03am
(Reply to #11)
Iain
Offline
Last seen: 12 years 3 months ago
Joined: 2010-11-23 11:07
Thanks to you all

All the Master information is stored on a server and moving large amount of data can be slow, the original Backup took over a week running it during the working day.
Thanks for your help will just have to work round our problem

Merry Christmas when I comes

Top
December 15, 2010 - 7:54am
(Reply to #12)
Darkbee
Darkbee's picture
Offline
Last seen: 2 years 10 months ago
Joined: 2008-04-14 09:41
Ouch

Is the master server being accessed during the day? Why can't you run the backup at night? Is it a database you're trying to backup, or just a plain file system? A week sounds an incredibly long time, is this for mirroring though? It sounds like you need some fundamental architectural changes to facilitate backing up such large volumes of data.

Top
December 16, 2010 - 3:44am
(Reply to #13)
Iain
Offline
Last seen: 12 years 3 months ago
Joined: 2010-11-23 11:07
Over the top

That was for the very first backup from nothing. Our server is on the otherside of the country.
We have about 100 staff accessing random files about 5-10MB each, might only need to archive 1GB of data every week or two so copy the whole 100GB as an archive is a bit over the top.
Thanks

Top
Log in or register to post comments