You are here

[Fixed] Cache files left behind due to a bug in RawTherapee

4 posts / 0 new
Last post
ajax
Offline
Last seen: 2 months 2 weeks ago
Joined: 2016-01-25 11:14
[Fixed] Cache files left behind due to a bug in RawTherapee

RawTherapee (RT) is a portableapp used for processing the raw files produced by many digital cameras. It has been discovered that large numbers of files that are cached for the purpose of aiding transition from one invocation of RT to the next are being transferred/stored on the local machine. This data may be very personal. It is precisely to avoid such a phenomenom that is one of the primary reasons for desiring to use portable applications in the first place.

Based on observation it appears as though these files are being stored on the portable drive in folder named “\PortableApps\RawTherapeePortable\Data\RawTherapee\cache” but are transferred to the folder named “C:\Users\David\AppData\Local\RawTherapee\cache” while the program is executing and then removed from there when RT terminates normally. It is suspected that they may be transferred back to the portable drive upon termination as the means for updating them. There are 2 problems with this concept as follows:

1. For RT these files may consume an enormous amount of space which might not be available on the local machines C: drive. This problem might be considered minor in that a consequence could be that RT simply fails to run on such a system. However, it is also possible that consumption of this space could have a significant consequential negative impact on the local system where such is completely unexpected.
2. A huge problem is that when RT fails to terminate normally these files would be left on the local machine without any but the most savvy end users aware of such a result. This data is likely to be of a personal nature and could end up being inadvertently left on numerous machines that are not under the control of data’s owner and who does not expect use of the portable application to be spreading their data around in this manner..

On Windows it looks like RT intends to use environment variables for locating where these files should be stored. Based on RT documentation there is a hard coded variable name of “%LOCALAPPDATA% which is used by default and that appears to be the location to which RT Portable is transferring the files on startup. However, RT provides the ability for the user to customize where such data is stored by setting a variable named “RT_CACHE”.

NOTE: There is also a variable named "RT_SETTINGS" that functions is a similar manner.

I’m not very knowledgeable about the mechanisms used to make applications portable but a quick glance at some of the ini files (e.g., particularly \PortableApps\RawTherapeePortable\App\AppInfo\Launcher\RawTherapeePortable.ini) there doesn’t seem to be any attempt to facilitate this aspect of the design of RT by defining a variable named “RT_CACHE”.

Absent some resolution for this problem this portable application ought to either be withdrawn or else a very prominent disclaimer given to anyone who might choose to use it.

John T. Haller
John T. Haller's picture
Offline
Last seen: 1 hour 28 min ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
Incorrect: Cache isn't moved locally, No Security Issue

You're saying there's a possibility of some cache files being left behind if a crash occurs. That's not what you'd classify a "major security issue". It's also not what the app does.

The last time I updated the launcher, the cache was hard coded to RawTherapeePortable\App\RawTherapee64\cache (or RawTherapeePortable\App\RawTherapee\cache) while running and moved back and forth to RawTherapeePortable\Data\Cache which usually only takes a few milliseconds. The environment variable FULLAPPDIR is a custom one used by this launcher to distinguish between the 32 and 64 bit versions of the app (see previous 2 directories mentioned). Please see the custom.nsh file for the code. At no time does the launcher move the cache to or from the local PC. This is accomplished via the setting MultiUser=false which is set in the options file which is moved to/from the appropriate Rawtherapee directory on start. When this is set, the cache is kept within RawTherapee's directory as mentioned in the options file's comments from the publisher. Is this not the behavior you are seeing? If not, then that's a bug in the current version of RawTherapee we package that we'll need to work around.

As to the settings, I was unaware of the RT_SETTINGS environment variable, but I think I'll be using that going forward if it works. I don't recall that this used to exist previously. From their documenation, it looks like they added this feature in 4.0.12.33. We started our bundle about 2 years before that. So, those env vars didn't exist at the time, which is why our launcher works the way it does.

I'll test to ensure these settings work as expected and incorporate them into RTP 5.2.

Sometimes, the impossible can become possible, if you're awesome!

ajax
Offline
Last seen: 2 months 2 weeks ago
Joined: 2016-01-25 11:14
Findings

It’s nice to know that these files are not being copied by the portableapps system.

What is happening on my 32bit system (i.e., RT 5.0) is that when RawTherapeePortable (RTP) is closed there is an empty directory named “C:\Users\David\AppData\Local\RawTherapee\cache”. Immediately after you launch RTP (i.e., have done nothing in RTP) this directory contains what appears to be the same subdirectories and files that are stored on my thumb drive in “\PortableApps\RawTherapeePortable\Data\RawTherapee\cache”. My reading of the RawPedia documentation (http://rawpedia.rawtherapee.com/Making_a_Portable_Installation), where it says “Let's assume that you've unzipped your archive into E:\RawTherapee, where E:\ is the drive letter of your USB flash drive. Now open the E:\RawTherapee\options file, and set the MultiUser option to false. That way, the cache directory will be located in a subdirectory of the installation path.” is consistent with yours. In that, it appears as though “MultiUser=false” is supposed to be how you make RTP portable which includes storing these files within the installation path (i.e., on thumb drive). It would seem to me that a very plausible interpretation is that in this regard RawTherapee is failing to function as prescribed.

Is there a means within the portableapps design, that I could exercise, for setting the designated environment variables (i.e., RT_CACHE and RT_SETTINGS)? If I wrote my own javascript to set them prior to invoking the portableapps startup program (i.e., start.exe) would you expect that to work? Possibly you have your own suggestion about a relatively simple way to test whether or not setting these variables solves the problem.

My contention regarding the security exposure pertains to other information in the RawPedia documentation where it says “By default, RawTherapee keeps up to 20 000 cached sets. Keep an eye on the "cache" folder as over time it may grow considerably in size! This is mostly due to the cached thumbnails which are stored in the "images" sub-folder. Deleting the "images" sub-folder is safe, you will not lose any image settings, RawTherapee will just have to regenerate the thumbnails.”. These are images from the camera which I regard as personal data.

Upon further examination it looks like these images are retained in files with an extension of .rtti. From just a little research I did into .rtti it appears as though this is some convention used within C++ programming which may suggest that compromising this data requires some specific C++ skills but I’m still inclined to think that portableapps users ought to know about this aspect of the way their data is being handled. I merely stumbled over this finding and was quite surprised. I now know that I need to exercise some care when it comes to what computers I might choose to run RTP on.

John T. Haller
John T. Haller's picture
Offline
Last seen: 1 hour 28 min ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
Fixed in 5.3-5.0

This is fixed with today's release of Rawtherapee Portable 5.3-5.0. I had to play around with things a bit to get them working properly. It seems setting the options file within the Rawtherapee to single user breaks the environment variables. It's working with both of them now to save some moving and keep things cleaner. The cache is only stored within the Data\cache directory and not on the local machine in my testing.

Sometimes, the impossible can become possible, if you're awesome!

Log in or register to post comments