nsis help: redirect APPDATA

I just checked the Firefox launcher and it doesn't redirect APPDATA at all - it backs up, then restores, FF's APPDATA folder.

You can probably adapt my TopOCR launcher to handle this; just change the backup & restore locations.

Here's how I handled saving settings the old fashioned PortableApps way (two years ago). The code may not work nowdays, but the comments could help you git-r-done. Scroll over to the right to read them.

    SkipSplash:
        IfFileExists `$PROFILE\Tick5.ini` "" MoveIni                            ; Check if local Tick5.ini exists --  No: Goto MoveIni
            Rename `$PROFILE\Tick5.ini` `$EXEDIR\Data\settings\Tick5.ini.bak`   ; Backup the local machine's Tick5.ini file 

    MoveIni:
        IfFileExists `$EXEDIR\Data\settings\Tick5.ini` "" Execute               ; If settings saved from last time -- No: Run program to generate a new ini
            Rename `$EXEDIR\Data\settings\Tick5.ini` `$PROFILE\Tick5.ini`       ; Copy ini back to profile dir

    Execute:
        ExecWait `"$EXEDIR\App\Tick5\tick5gui.exe"`
        Rename `$PROFILE\Tick5.ini` `$EXEDIR\Data\settings\Tick5.ini`           ; Save settings back to portable
        IfFileExists `$EXEDIR\Data\settings\Tick5.ini.bak` "" TheEnd            ; Check if local Tick5.ini file backed up? 
            Rename `$EXEDIR\Data\settings\Tick5.ini.bak` `$PROFILE\Tick5.ini`   ; Return original ini file to the local machine

Here's the relevant stuff from my TopOCR launcher. I actually copied some of this (the Registry backup/restore code) from the 7-Zip launcher, then modified it to work with TopOCR. You can probably do the same for Handbrake.

Anyway, here you go:

Includes:

!include LogicLib.nsh
!include Registry.nsh

TopOCR file handling

       INIBackup:
       		${If} ${FileExists} "$WINDIR\topocr.ini"
                      ReName $WINDIR\topocr.ini $WINDIR\topocr-BackupByTopOCRPortable.ini
                ${EndIf}

        RestoreTheINI:
                      IfFileExists "$SETTINGSDIRECTORY\topocr.ini" "" RegistryBackup
                      ${If} ${FileExists} "$WINDIR\topocr.ini"
                            Delete $WINDIR\topocr.ini
                        ${EndIf}
                      CopyFiles $SETTINGSDIRECTORY\topocr.ini $WINDIR\topocr.ini

	SetOriginalINIBack:
                        ${If} ${FileExists} "$WINDIR\topocr.ini"
                            Delete $WINDIR\topocr.ini
                        ${EndIf}
                        
                        ${If} ${FileExists} $WINDIR\topocr-BackupByTopOCRPortable.ini
                              ReName $WINDIR\topocr-BackupByTopOCRPortable.ini $WINDIR\topocr.ini
                        ${EndIf}

TopOCR Registry handling

	RegistryBackup:
		StrCmp $SECONDARYLAUNCH "true" LaunchAndExit
		;=== Backup the registry
		${registry::KeyExists} "HKEY_CURRENT_USER\Software\SubSystems-BackupByTopOCRPortable" $R0
		StrCmp $R0 "0" UpdatePaths
		${registry::KeyExists} "HKEY_CURRENT_USER\Software\SubSystems" $R0
		StrCmp $R0 "-1" UpdatePaths
		${registry::MoveKey} "HKEY_CURRENT_USER\Software\SubSystems" "HKEY_CURRENT_USER\Software\SubSystems-BackupByTopOCRPortable" $R0
		Sleep 100
	
	UpdatePaths:
		ReadINIStr $0 "$SETTINGSDIRECTORY\topocr_portable.reg" "HKEY_CURRENT_USER\Software\SubSystems" `"TopOCR"`
		StrCmp $0 "" RestoreTheKey
			StrCpy $1 $0 "" 3 
			StrCpy $2 $EXEDIR 1 ;current drive letter
			StrCpy $0 `"$2:\$1"`
			WriteINIStr "$SETTINGSDIRECTORY\topocr_portable.reg" "HKEY_CURRENT_USER\Software\SubSystems" `"TopOCR"` `$0`
	
 	RestoreTheKey:
		IfFileExists "$SETTINGSDIRECTORY\topocr_portable.reg" "" LaunchNow
	
		IfFileExists "$WINDIR\system32\reg.exe" "" RestoreTheKey9x
			nsExec::ExecToStack `"$WINDIR\system32\reg.exe" import "$SETTINGSDIRECTORY\topocr_portable.reg"`
			Pop $R0
			StrCmp $R0 '0' LaunchNow ;successfully restored key

	RestoreTheKey9x:
		${registry::RestoreKey} "$SETTINGSDIRECTORY\topocr_portable.reg" $R0
		StrCmp $R0 '0' LaunchNow ;successfully restored key
		StrCpy $FAILEDTORESTOREKEY "true"

	SetOriginalKeyBack:
		${registry::DeleteKey} "HKEY_CURRENT_USER\Software\SubSystems" $R0
		Sleep 100
		${registry::KeyExists} "HKEY_CURRENT_USER\Software\SubSystems-BackupByTopOCRPortable" $R0
		StrCmp $R0 "-1" TheEnd
		${registry::MoveKey} "HKEY_CURRENT_USER\Software\SubSystems-BackupByTopOCRPortable" "HKEY_CURRENT_USER\Software\SubSystems" $R0
		Sleep 100
		Goto TheEnd

Hint: Use the NISEdit search function to locate those section labels in TopOCR.nsi. The code will be much easier to read & understand when taken in context (and also with proper NSIS formatting).

HTH!

Wait, what's wrong with modifying the environment variables?

OK, it's time for me to try and clarify everything.

1. There are four main ways used of finding out where %APPDATA% is.
   1. Through a DLL call, SHGetFolderPath (there are a few variants of it like SHGetSpecialFolder in various stages of deprecation and OS support but they all do the same thing). This is the way you're "supposed" to do it, and how it's normally done.
   2. The environment variable %APPDATA%. This is common, but not the way it "should" be done.
   3. In the registry, in HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders\AppData. (This should not be used. There's actually a value in that key !Do not use this registry key with the value "Use the SHGetFolderPath or SHGetKnownFolderPath function instead"!)
   4. %USERPROFILE%\Application Data. Yes, I have actually seen this in, I think, three applications. (Don't use this one ever.)
2. A call with SHGetFolderPath can only be handled in one way; for an old completely working example, see my AppData redirection script.
   1. Note that %APPDATA% is %USERPROFILE%\[something]. That [something] is "Application Data" on Windows XP (English), varies between translations, and on Vista/7 is "AppData\Roaming" (note that extra subdirectory, it's a real trouble with handling this).
   2. Set %USERPROFILE% to your new location.
   3. Set %APPDATA% to the new %USERPROFILE%\[something] - it must be the same subdirectory layout as it was before!
3. Changing %APPDATA% and/or %USERPROFILE% via Kernel32::SetEnvironmentVariable applies to the current process and all spawned sub-processes.
4. If you change %APPDATA% other than in the way described two points up (in parallel with %USERPROFILE%, as the correct subdirector[y|ies]), the SHGetFolderPath call will return an empty string (which can break some applications, and can make others look for a different location which may even be useful, though inadvisable).
5. The native file selection dialogues (open/save) don't work with redirected USERPROFILE, as they look for the location of the buttons on the side and can't find them - and so you'll never be able to select a file (the main window loses focus and then gets it again and the file chooser never comes up).
6. Because sub-processes get the same things, the problems in the above two points apply to any processes that may be spawned. And so, for that reason, an application like Firefox couldn't possibly use an APPDATA redirect as it can start arbitrary processes which might break horribly.
7. I think I've explained all in my mind just now... if you have any more questions, just post them and I should be able to answer them. (I did a lot of research into the workings of %APPDATA% about a year ago when I was trying to work with Inkscape.)
8. My final piece of advice is, look at the source code and see if you can find an alternative settings location switch (maybe it searches for an environment variable first or something like that). If not, request it as a feature or better still offer a patch to do it (I did that last with Inkscape which is why Inkscape Portable 0.47 works by setting the INKSCAPE_PORTABLE_PROFILE_DIR environment variable). A quick look seems to indicate that HandBrake is using the C# Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData) which is SHGetFolderPath, so either move data back and forth or try to patch it or ask them to.

I don't have Handbrake installed on my system, so I can't test this, but it looks like you're not doing it quite the way MSDN would like...

MSDNExample 2

Altering the environment variables of a child process during process creation is the only way one process can directly change the environment variables of another process. A process can never directly change the environment variables of another process that is not a child of that process.

If you want the child process to inherit most of the parent's environment with only a few changes, retrieve the current values using GetEnvironmentVariable, save these values, create an updated block for the child process to inherit, create the child process, and then restore the saved values using SetEnvironmentVariable

Unfortunately, I don't know how to do this in NSIS.