Toucan Deleting But Not Copying Data !!!

28 posts / 0 new

Last post

September 30, 2010 - 6:54am

Justice

Offline

Last seen: 12 years 7 months ago

Joined: 2006-05-10 06:53

Toucan Deleting But Not Copying Data !!!

Firstly thank you for making Toucan !

I normally use SyncbackPro for my backups but I don’t like the fact it doesn’t make true mirror images of the source drive. Upper and lower case characters are ignored and this is pretty annoying when you have re-named thousands of MP3 files !

I was very pleased to learn that Toucan was able to differentiate the character case so I thought I would make the switch. However I have to say I am disappointed and a little worried with my first encounter with Toucan.

I had downloaded Toucan V 3.0.2 some time ago and never got round to testing it until last night. I set it all up and used the “Mirror” option. I am fairly computer literate so I will skip the minor details.

The first thing I noticed was that it took a few attempts to start Toucan. I then discovered that there were a few instances of Toucan running in the process list so I tried to disable them there. I then assumed this was a one off and re-started my computer. I tried again to start Toucan and after a very long pause it started.

I used Toucan to “Mirror” my large backup drives. I have 2 Sata 500GB drives connected to my computer and they are encrypted with Truecrypt. I mounted both containers and Toucan saw both pseudo drive letters straight away. Everything seemed ok up to now.

I then set Toucan running and it quickly found discrepancies in character cases between the 2 drives. I noticed that the progress list showed that it had “Removed” many files from my destination drive. I knew these files were named wrongly so I was pleased that Toucan had found them and I assumed that everything was going well.

What I did notice was that Toucan was not copying across the new versions of these files. As this was my first run with Toucan I wondered if these files were all copied over at the end of the run so I waited patiently. Toucan took a very long time to progress through the files and then to my surprise it just crashed.

I then looked on the site and noticed that a newer release was available so I downloaded that in the hope it would solve all my problems.

I installed and set up Toucan V 3.0.3 and started it. It still took a couple of attempts to get going but it did seem better than V 3.0.2. It went though my files and then started to copy across some of the files the first V 3.0.2 had deleted from my destination drive.

I assumed everything was ok now and let it run its course. It took several times longer to make a complete run compared to SyncBackPro but I put this down to the fact it was making a better job of things which I am prepared to wait for.

However, again Toucan V 3.0.3 was deleting files from the destination drive and not copying the new versions across. If I hadn’t been watching it I would have wrongly assumed everything had been mirrored and have been none the wiser as this pass didn’t crash and there were no error messages.

I worry that there may be other people assuming Toucan is copying changes across and thinking they have backups when they don’t !

So, my questions are.

Is there a drive size / amount of files to process limit Toucan can’t handle and did I go over it ?

If Toucan is buggy, has it done any damage to my source drive ??

Just to re-cap and help bug fix.

I have….

XP Pro SP3
1 IDE C drive
2 x 500GB SATA drives encrypted with Truecrypt.
I mount the 2 SATA drives and then point Toucan to the new pseudo dives.

Please don’t be disheartened by my report, I am very grateful to free software developers its just that when I start to think I may loose valuable data I felt the need to warn you and other users that V 3.0.2 and V 3.0.2 at least, are not working properly.

Thanks again and I hope you are able to fix Toucan as it looks like an interesting alternative.

September 30, 2010 - 9:31am

Steve Lamerton

Offline

Last seen: 9 years 8 months ago

Joined: 2005-12-10 15:22

Thankyou

for the report, unfortunately nothing like this came up in the pre-release testing. Firstly I am not sure why it is starting so slowly for you, very little actually happens when Toucan starts up, it checks to see if it is on a read-only drive, updates and out of date settings files and then launches the user interface.

As for the main problem you are having with mirroring files I am a little stumped I am sorry to say! They should be no real limit to the number of files or their size, certainly I have heard of people using Toucan with data sets larger than yours in the past. I just ran a quick test myself with ~0.5GB of data to test and that worked fine, I will check again with ~45GB later and see how that gets on. My first suggestion would be to check how the preview looks, do you see the files being copied there? The complete lack of a copy error message seems to suggest Toucan didn't think the files needed copying for some reason, hopefully the preview will help to clarify this.

September 30, 2010 - 1:02pm

(Reply to #2) #3

Justice

Offline

Last seen: 12 years 7 months ago

Joined: 2006-05-10 06:53

Hi Steve Thank you for your

Hi Steve

Thank you for your reply. Before you read any further please understand that I am not being critical or angry about this. I am grateful for your free software but it is hard to convey mood in text based messages. So in an effort to allow me to write freely about the problems I am having with Toucan please bear this in mind when reading the rest of my post.

The starting problem.

I have to say that V 3.0.3 is considerably better at starting than V 3.0.2. I notice it is better if I reboot and don’t do anything else other than open Toucan.

If I open Toucan use it, shut it down then open it sometimes it takes a long time.

(Just returned to this point to mention that I have tested this again with V 3.0.2 and I can confirm V 3.0.3 is much better at loading than V 3.0.2.)

Mirroring Problem.

I would say that I may be asking too much from Toucan, I have just checked and I am checking 442GB of data ! There are 68,430 files in 6,046 folders.

I have been testing and I can report the following.

Toucan Settings:

Sync
Function = Mirror
File Check = Short Comparison
Everything else unchecked.

XP Pro SP3
C drive IDE
V and W are 500GB SATA Drives
V and W are the pseudo drive letters assigned by Truecrypt. If you are unfamiliar with Truecrypt I can explain this in more detail if you like.

V is Source and W is Destination.

I press Run and watch the display screen.

There is nothing on the screen for quite some time (I guess Toucan hasn’t found any discrepancies during this time.) Which is probably correct.

Occasionally the progress / preview screen shows that Toucan is removing files from W. These are indeed files that need replacing. What I have noticed is that Toucan does sometimes actually copy the files from V to W (as it should) when overwriting changed files. This seems / appears to happen when there aren’t many files to copy less than 5 or so. If there are more than 5 - 7 then I seem to get the problem. These files are only mp3’s so there isn’t really much actual data there to give you an idea of file size. I would estimate 20 – 30 MB batches.

The preview / progress screen has never shown that it is copying data from source to destination for me to later discover that it hasn’t. It only displays that it has removed data and because I don’t see any copying notification I then check to find that it has not been done. So I would suggest that Toucan knows there is a difference between the files, removes them but then forgets to copy the replacements across.

Toucan has again today removed files but then failed to copy the replacements across. Interestingly if I cancel the job Toucan will notice the files are missing from W (destination) and copy them from V (source) on a separate and new run.

Going only by filename I seemed to get about 25% through the files when I noticed the first “non” copy. After Toucan has started to miss or forget to copy across from source what it has deleted from destination it seems to always forget to do it from then on unless I stop it and restart the process again.

Also the progress bar can sometimes go to the end of the screen and stop there well before Toucan has actually finished. This of course is only a cosmetic problem to me but it may be of interest to you.

Another point to mention is that I have a pitiful amount of RAM, 512MB actually, if that makes any difference to anything. However SyncBackPro has no trouble with these drives, in fact I used it to repair the files deleted by Toucan.

Not to belittle Toucan but more to prove my system is ok and also my abilities I have actually tried a few backup programs today and I didn’t experience any problems with them, same drives, same procedure.

Thanks again for your help.

September 30, 2010 - 2:05pm

(Reply to #3) #4

Darkbee

Offline

Last seen: 3 years 1 month ago

Joined: 2008-04-14 09:41

Not much

This might come off as overly sarcastic but it isn't meant to be...

How much data? That's a lot! You don't expect this to run in any reasonable amount of time do you? We have database at work smaller that this that get backed up nightly and they can take a couple of hours to run!

Anyway, I'm sure your detailed analysis is appreciated. Anything that helps Toucan improve is a good thing. I have been looking for Toucan to replace SyncBack(free edition) and so far it's almost up to the job. I think a few more iterations and I'll have complete confidence in Toucan.

September 30, 2010 - 3:52pm

(Reply to #4) #5

Justice

Offline

Last seen: 12 years 7 months ago

Joined: 2006-05-10 06:53

Hi Darkbee I am not

Hi Darkbee

I am not complaining about the time taken at all and I hope no one got that impression from my post. I understand that Toucan is checking for upper / lowercase which I can imagine massively increases the time taken.

You do realise that I was talking about “Mirroring” and not “Backing Up” don’t you ? There is a significant difference in time taken for the two methods. If you are mirroring your backups at work that are smaller than mine and it is taking 2 hours there may be something wrong.

SyncBackPro does the scan for me in 4-5 minutes. Then time added depending on how much data there is to copy across. Toucan was taking around 50 minutes (I think) but admittedly it did copy some data. However, the data it actually moved was less than 40MB.

My entire reason for posting is that I want to prevent anyone from assuming that Toucan has backed up all their data when it might not have actually done that. Removing files from the destination that are supposed to be replaced from source during the same session and then failing to do so is a little worrying.

I hope Steve manages to work it out for us.

September 30, 2010 - 4:07pm

(Reply to #5) #6

Darkbee

Offline

Last seen: 3 years 1 month ago

Joined: 2008-04-14 09:41

My Mistake

You're right, I was considering the worst case scenario for mirroring... I must be an eternal pessimist.

I agree that a backup/synchronization program had better do what the user expects it to do otherwise the consequences could be dire.

September 30, 2010 - 6:11pm

(Reply to #6) #7

Justice

Offline

Last seen: 12 years 7 months ago

Joined: 2006-05-10 06:53

Hi Darkbee Yeah, at least

Hi Darkbee

Yeah, at least Toucan had the progress screen so I was able to notice the problem.

I wonder how many users haven’t even thought to check whether Toucan (or any backup software) has actually mirrored or not ? I must admit to using SyncBackPro for nearly an entire week before I noticed it wasn’t mirroring properly.

As you have access to a large drive with the need to back it up, would you be prepared to help test Toucan ?

September 30, 2010 - 7:19pm

(Reply to #7) #8

Darkbee

Offline

Last seen: 3 years 1 month ago

Joined: 2008-04-14 09:41

I test already... sort of

Actually, I use Toucan already but I have to confess I've been overly casual about assuming it's doing its job thoroughly. I do have some large jobs for home use though nowhere near as large as yours. My largest job would be about 80gbs worth of data. I was actually preparing a drive for use last night, although it's to backup a NAS which has a built-in backup feature that I want to try first.

I can try the Toucan mirror afterwards if the backup is successful (I'm asuming the NAS backup feature just copies the files).

September 30, 2010 - 8:13pm

(Reply to #8) #9

Justice

Offline

Last seen: 12 years 7 months ago

Joined: 2006-05-10 06:53

That would be useful if you

That would be useful if you would try even at 80GB, as I said before I think Toucan started to fail about 25% the way in so I don’t think it has to be as large as my total drive. Please remember it is mirroring that fails, not copying or backup.

I am wondering if it is just because I am looking for alpha case changes and not actually missing or updated files.

When you have done your copying would you mind changing the case of a bunch of files on your source drive and then use Toucan ?

I have access to two 2 TB drives that are nearly full, one being a mirror of the other. When Steve has fixed Toucan and it is safe to use I will test on that if you like ! It might take a while though !!

October 3, 2010 - 7:29pm

(Reply to #9) #10

Darkbee

Offline

Last seen: 3 years 1 month ago

Joined: 2008-04-14 09:41

Good news and Bad news

The good news is (kind of) my built-in NAS backup feature doesn't work. I suspect this is because my destination drive is smaller than my source drive and so the feature simply won't permit the backup to start.

The bad news is my current external hard drive enclosure seems to be giving up the ghost. It was never the most reliable to begin with (an eBay cheapy) but now it's more unreliable than ever. I just tried to do a full mirror of the NAS (starting with an empty destination drive) and it cr@pped out about 10% of the way into the mirror.

I guess this is all kind of moot since Steve seems to have figured out the problem, still it might be nice to put Toucan through it's paces. I guess I'll have to buy a better quality HD enclosure.

October 1, 2010 - 6:03am

(Reply to #10) #11

Steve Lamerton

Offline

Last seen: 9 years 8 months ago

Joined: 2005-12-10 15:22

Don't

worry, I really appreciate you providing so much detail, it should really help us track this down! Firstly I apologise in advance if my replies are a bit fragmented at the moment as I am in the process of moving up to university to start the new year so things are a bit frantic

In terms of the size of data it is fairly high, but I regularly mirror ~50GB in 46,000 files and 4,000 folder without problems so I don't think that should be the problem.

Just to clarify none of the files that should be copied exist in the destination drive? If they do then it sounds the short check thinks they are the same and so skips them, if you try it without any of the checks the files will always be copied so that would be a good thing to try.

Thanks in advance!

October 1, 2010 - 8:29am

(Reply to #11) #12

Justice

Offline

Last seen: 12 years 7 months ago

Joined: 2006-05-10 06:53

Don’t worry, I really

Don’t worry, I really appreciate you providing so much detail

OK, I have actually bug tested a lot of programs and I am almost an expert at it now !! Ha ha ! Some professional (paid for) programs don’t escape my fault finding !! I have bug tested for PGP, Jetico, Truecrypt, ULM, SyncBackPro, Comodo Firewall, AVG, NOD32..etc so don’t feel bad, you are not alone !

Just to clarify none of the files that should be copied exist in the destination drive?

Some do get copied, it is intermittent (the worst kind of bug). Ones that never existed on destination do get copied, ones that were already there and require removal then re-copying often get removed then their replacement (updated version) doesn’t get copied across.

A very simplified explanation of what is happening.

Example.

Source Contains

File 1
File 2
File 3
File 4

Destination Contains

File 1
file 2
File 3
file 4

When Toucan scans using the settings I mentioned above, it correctly notices that File 2 and File 4 on source is in upper case (the only difference between the files) and that file 2 and file 4 are in lower case on the destination drive.

Everything ok up to this point.

Toucan then updates the progress screen and tells me that file 1 and file 2 have been removed.

Everything ok up to this point. I expect Toucan to remove these files as it should then replace them with File 1 and File 2.

(Sometimes this does actually happen at this stage and everything works great using the same settings on the same drives and data).

However…

Quite often Toucan continues to scan trough the drive and doesn’t replace (copy from source the files it deleted from destination).

So at the end of a failed pass I am left with …

Source Contains

File 1
File 2
File 3
File 4

Destination Contains

File 1
File 3

When I should be left with….

Source Contains

File 1
File 2
File 3
File 4

Destination Contains

File 1
File 2
File 3
File 4

Toucan then appears to the user to have completed successfully. No error messages or crashes at this point. The user is unaware that file 2 and file 4 have been deleted from the back up destination drive. This is why I think many people haven’t noticed it.

If Toucan is run again using the exact same settings it will notice that File 2 and File 4 are on the source drive but don’t exist on the destination drive and correctly copy them across.

I believe that it is something like Toucan correctly identifies mismatching files, knows to remove them from the destination drive and then simply forgets to copy the new replacement files from source.

Can I mention that I don’t think Truecrypt is an issue here, although it is a difference between our setups I did extensive tests with Truecrypt and it has absolutely no effect on any software running in the windows environment. It is a transparent driver that I have yet to find fault with. Although I am prepared to admit there is always a first time for everything ! Ha ha !

Thanks.

October 3, 2010 - 4:24pm

#13

Steve Lamerton

Offline

Last seen: 9 years 8 months ago

Joined: 2005-12-10 15:22

Just

to say here the problem is specifically with files that are in both the source and destination folders with the same name but different cases. I have committed a partial fix to the Toucan source control, but I plan on refining it a little more. Should get a pre-release out with this fixed this week hopefully.

October 3, 2010 - 6:53pm

(Reply to #13) #14

Justice

Offline

Last seen: 12 years 7 months ago

Joined: 2006-05-10 06:53

Hi Steve Thank you very much

Hi Steve

Thank you very much for letting us know you have sorted it, or at least that you know what it is.

Is there any chance that you could provide a “not too technical” explanation as to the problem ?

Also now you know what the problem is could you put my mind at rest that I haven’t lost / damaged any files on my source drive ? I appreciate this is unlikely but as there is a bug, however small, it does make my mind wonder !!!

Thanks again Steve, great work !

October 4, 2010 - 9:08am

(Reply to #14) #15

Justice

Offline

Last seen: 12 years 7 months ago

Joined: 2006-05-10 06:53

Hi Is the forum software

Is the forum software working as it should ?

I can see in the main page for Toucan that Darkbee has replied to this thread but I am unable to see his post. Quite often I log in here and I cannot find my own posts if I return within an hour or so.

However I notice that Darkbee’s post was over 13 hours ago and yet it still isn’t there for me.

Main page states last reply was…

13 hours 34 min ago
by Darkbee

October 4, 2010 - 9:12am

(Reply to #15) #16

Darkbee

Offline

Last seen: 3 years 1 month ago

Joined: 2008-04-14 09:41

Nothing to see here

It's there, but this forum is not very user-friendly IMHO. Whenever you see "new" click on the word "new" literally and it will jump to the exact new post.

My post is further up the page because I replied to one of your earlier comments.

Nothing exciting really. Move along.

October 4, 2010 - 9:56am

(Reply to #16) #17

Justice

Offline

Last seen: 12 years 7 months ago

Joined: 2006-05-10 06:53

Hi there Darkbee I can see it

Hi there Darkbee

I can see it now, although it looks as if you edited a post rather than made a new one !

As you say, this forum software is a little strange, why is it still being used ?

Anyway it looks as if Steve has sorted the problem with Toucan.

October 4, 2010 - 10:19am

(Reply to #17) #18

Jimbo

Offline

Last seen: 3 years 4 months ago

Joined: 2007-12-17 05:43

not strange, just different

The forum system here is part of Drupal, the CMS that runs the entire site.

It is a little different from the usual phpBB, in that it supports fully threaded comments. In other words, when you reply to a comment from someone, your reply is actually stored as a reply to that comment, instead of being blindly and meaninglessly tacked on at the bottom of the list. This means that people reading the thread, actually see a threaded discussion, with the individual streams of conversation.

New posts are highlighted with a "new" in read in the title of them, and if you click on the "new" text on the index page it will take you straight to the first of them on the thread page.

Sorry if you find this form confusing, but it makes the archives of the discussions significantly easier to use.

October 4, 2010 - 11:41am

(Reply to #18) #19

Darkbee

Offline

Last seen: 3 years 1 month ago

Joined: 2008-04-14 09:41

Adequate

I don't want to get off on an anti-drupal-forum rant but I have to disagree. It is useful sometimes, but for long threads, multiple pages long with increasingly decreasing widths of postings it's a hot mess. It's not useful at all. Frequently you see people not actually replying to the thread they are really replying to, in order to avoid the "helpful" formatting.

I addition, when you have one new post, it's fine, you can jump to that post, but when you have multiple new postings all over the place it doesn't work. It's often the case that I can't even find where the new posts were so I just give up.

However, as I stated above, the administrators of this site have decided it works for them and so it is what it is. I understand the concept and it's a logical choice but in reality it doesn't always quite work out either. The forums are adequate. That's all I have to say.

October 4, 2010 - 10:22am

(Reply to #19) #20

Darkbee

Offline

Last seen: 3 years 1 month ago

Joined: 2008-04-14 09:41

It works

It's being used because I suppose it works, and the administrators see no need to change it. The whole site is running using Drupal and making such changes would not be trivial. I don't know if there are multiple forum choices as modules for Drupal but this is the one that has been chosen. I suppose as the saying goes "if it ain't broke, don't fix it". Technically, it's not broken albeit rather unfriendly.

FYI, you can't edit your post once someone replies to it (with the exception of the original thread posting).

October 4, 2010 - 10:57am

(Reply to #20) #21

Justice

Offline

Last seen: 12 years 7 months ago

Joined: 2006-05-10 06:53

Hi Jimbo and Darkbee Hmm… I

Hi Jimbo and Darkbee

Hmm…

I think I understand what you mean about reading through the entire thing at a later date, but as it is so rare to use this type of forum it really does seem either broken or at least confusing to the average humble user.

I must admit if I wasn’t so concerned about people loosing their backups before Toucan was fixed I don’t think I would have stuck with it for much longer.

Has Portable apps ever considered using Trac to let people report bugs etc ?

October 8, 2010 - 3:12pm

(Reply to #21) #22

Justice

Offline

Last seen: 12 years 7 months ago

Joined: 2006-05-10 06:53

Hi Steve I just wondered how

Hi Steve

I just wondered how you were getting on with this.

Has it turned out to be a bigger problem than you first thought ?

October 9, 2010 - 5:43am

#23

Steve Lamerton

Offline

Last seen: 9 years 8 months ago

Joined: 2005-12-10 15:22

Sorry

for the slow response, I posted a pre-release here a couple of days ago that I think should solve things pretty well. It looks like it doesn't run for some / all people so I shall have a look tomorrow when I get home.

October 9, 2010 - 8:54am

(Reply to #23) #24

Justice

Offline

Last seen: 12 years 7 months ago

Joined: 2006-05-10 06:53

Hi Steve Thanks for the

Hi Steve

Thanks for the update, I look forward to 3.0.4 !

Good work !

October 20, 2010 - 6:58am

(Reply to #24) #25

Justice

Offline

Last seen: 12 years 7 months ago

Joined: 2006-05-10 06:53

Hi Steve It has been a few

Hi Steve

It has been a few weeks now since I reported the case sensitive bug. I appreciate you got onto it straight away and say you fixed it.

I see that version 3.0.3 is the main exe available which still contains the case bug. I just wonder is it fair to new or inexperienced users to allow them to download that version when it “may” result in them loosing data should they ever need to restore from a backup made by 3.0.3 ?

I understand Toucan is free, you do all your work on it for free and in your spare time, which I and probably everyone else who uses Toucan is extremely grateful for.

I tried the “fixed” version of Toucan 3.0.4 which didn’t even run for me at all. If you are very busy and don’t have time for Toucan at the moment could you perhaps please put a warning message on the 3.0.3 version ?

As far as I could tell if Toucan 3.0.3 is run several times on the same backup it does eventually work and copy the data it deleted. This would be a workaround for new users and help them protect their data.

As I mentioned before, I am not having a go at you or being ungrateful for your work, it’s just that we were all noobs once and I can foresee someone loosing data with 3.0.3 which isn’t fair really, especially when they are making backups.

Thanks.

October 24, 2010 - 5:13am

(Reply to #25) #26

Steve Lamerton

Offline

Last seen: 9 years 8 months ago

Joined: 2005-12-10 15:22

will upload a (hopefully) fixed copy of the 3.0.4 test later on. I am just not sure where my time has gone recently. I will add it to the known issues list but I think it is important to note that if you use Toucan to copy this files originally then there won't be a problem.

I think this bug may well have been around for a long time, certainly since 3.0 and no-one has ever reported it before, now this might be because no-one has noticed, but I suspect really it is because it just isn't a common case to already have files in both a source and destination but with differing cases, normally one would have been copied to the other and then they would be the same.

I will post again when I have hopefully uploaded a fix and then hopefully we can get a release out this week which fixes it Thanks for your help on this one and for reminding me to get on with it!

October 28, 2010 - 10:41am

(Reply to #26) #27

Justice

Offline

Last seen: 12 years 7 months ago

Joined: 2006-05-10 06:53

Hi Steve Thanks for your

Hi Steve

Thanks for your reply.

It’s nice to know what’s happening and I am grateful for your progress report. I am only nagging at you because I want to use Toucan as my main back up software mainly because of the case sensitive feature and I am just getting a bit impatient !! Ha ha !

Thanks again and I look forward to seeing the new release being made available.

October 29, 2010 - 8:13am

(Reply to #27) #28

Steve Lamerton

Offline

Last seen: 9 years 8 months ago

Joined: 2005-12-10 15:22

It's

already up here:
https://portableapps.com/node/25171

Please Help Support Us

Please consider making a donation so we can keep bringing you great software.

You are here

Toucan Deleting But Not Copying Data !!!

Please Help Support Us

Translate Page

Latest Releases & News

Join Our Community

Partner with PortableApps.com

About PortableApps.com

You are here

Toucan Deleting But Not Copying Data !!!

Please Help Support Us

Translate Page

User login

Latest Releases & News

Join Our Community

Partner with PortableApps.com

About PortableApps.com