You are here

PortableApps.com App Compactor 1.0 Pre-Release 4

126 posts / 0 new
Last post
John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
PortableApps.com App Compactor 1.0 Pre-Release 4

This release is OUTDATED. Please see the newer release here:
https://portableapps.com/node/16097

Application: PortableApps.com App Compactor
Category: Utilities
Description: PortableApps.com App Compactor allows you to easily compress application to decrease size on disk and increase performance when running from a slow media (like flash drives). It packages UPX and 7-Zip within an easy to use wizard allowing you to easily compress an app's EXEs, DLLs and other binaries using UPX in either NRV2E, NRV2D, LZMA or BRUTE mode as well as recompressing ZIP and JAR files to their maximum level using 7-Zip. It also has an option to DECOMPRESS an app's binary files that were previously compressed with UPX. Just select the directory and all the DLL, EXE, ZIP, JAR, IRC, PYD and BIN files within that directory will be compressed. Depending on the files, the PortableApps.com App Compactor can decrease an app's size by 50% or more without affecting functionality. This same tool has been used on PortableApps.com's most popular apps including OpenOffice.org Portable, Pidgin Portable, 7-Zip Portable and more.

Download PortableApps.com App Compress 1.0 Pre-Release 4 [0.7MB download / 0.8MB installed]
(MD5: 0c853b7180fa925ed7f8ca03bcb465b9)

View a Screenshot of Compression Options

Helpful tips:

  • Don't compress NSIS launchers with CRC enabled (which they all should), they won't run
  • It's often easiest to just select the AppName\App directory in a portable apps
  • It's best to have the app you're compressing reside on a hard drive (at least temporarily) rather than a flash drive to improve performance and decrease writes to your flash drive
  • You can disable the Welcome screen or have the console window stay by editing the settings.ini in the Data directory
  • ONLY USE NRV2E for development tests and pre-releases of stuff that will be officially released unless it's a small app (one exe and a dll or two) and very easy for you to test every feature and function on every Windows OS.

I know this one has been a long time coming. Please try it out and post your feedback.

And if you're using this on your apps for release, please mention it Smile

Thanks!
John

Release Notes:

1.0 Pre-Release 4 (2008-04-10): Fixes and Enhancements:

  • ENHANCEMENT: Now accepts path on the command line, skips welcome and prefills path to compress making it easy to drag and drop a path onto the EXE from Explorer or set it up with a Sent To in Windows
  • BUGFIX: Paths with spaces now properly recompress Zips
  • BUGFIX: Now recompresses Zips and Jars in the base directory selected
  • OTHER: Name changed to PortableApps.com App Compactor to avoid possible trademark confusion

1.0 Pre-Release 3 (2008-03-25): BUGFIX: Now works on paths with spaces again

1.0 Pre-Release 2 (2008-03-25): Fixes and Enhancements:

  • BUGFIX: ZIP/JAR recompression no longer loops on FAT32 drives
  • ENHANCEMENT: UPXed files are tested to ensure they decompress (thanks Shawn!)
  • ENHANCEMENT: Directory size before and after and space saved shown on finish screen
  • ENHANCEMENT: Checkbox to compress another app added to final screen
  • ENHANCEMENT: Console window closed by default to speed process
  • ENHANCEMENT: ZIP/JAR compression set to maximum instead of Ultra to improve compatibility and runtime performance
  • MISC: Spelling errors and inconsistencies fixed

1.0 Pre-Release 1 (2008-03-19): Initial Release

Tim Clark
Tim Clark's picture
Offline
Last seen: 10 years 3 months ago
Joined: 2006-06-18 13:55
For ??

Am I correct in assuming that this is for our personal use to compress apps that we [PortableApps.com] can not distribute compressed for licensing reasons, e.g. FFP and TBP?

Tim

Things have got to get better, they can't get worse, or can they?

ZachHudock
ZachHudock's picture
Offline
Last seen: 3 years 6 months ago
Developer
Joined: 2006-12-06 18:07
It could also be of use to

It could also be of use to developers here at PortableApps.com.

The developer formerly known as ZGitRDun8705

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
Development

This is primarily a tool for developers to compress apps for release on PortableApps.com. But it can also be used by individuals for personal use on any software they'd like (provided it isn't redistributed).

Sometimes, the impossible can become possible, if you're awesome!

OliverK
OliverK's picture
Offline
Last seen: 1 month 6 days ago
Developer
Joined: 2007-03-27 15:21
why?

why?

Too many lonely hearts in the real world
Too many bridges you can burn
Too many tables you can't turn
Don't wanna live my life in the real world

Tim Clark
Tim Clark's picture
Offline
Last seen: 10 years 3 months ago
Joined: 2006-06-18 13:55
Why ???

Why is This is primarily a tool for developers?

Why can [it] also be used by individuals for personal use?

Why provided it isn't redistributed?

What is your question?

Tim

Things have got to get better, they can't get worse, or can they?

OliverK
OliverK's picture
Offline
Last seen: 1 month 6 days ago
Developer
Joined: 2007-03-27 15:21
Point taken. Why can't it be

Point taken.

Why can't it be used for things other then those developed for PortableApps.com?

Too many lonely hearts in the real world
Too many bridges you can burn
Too many tables you can't turn
Don't wanna live my life in the real world

Tim Clark
Tim Clark's picture
Offline
Last seen: 10 years 3 months ago
Joined: 2006-06-18 13:55
He didn't say that.

He didn't say that.

This is primarily a tool for developers to compress apps for release on PortableApps.com

As opposed to my question he was answering.

provided it isn't redistributed is referring to apps created using the PortableApps.com App Compactor, not the Compactor itself.

e.g., if I used it to compact FF I would not be allowed to distribute it as FF because of Mozilla's rules.

Tim

Things have got to get better, they can't get worse, or can they?

paulf
Offline
Last seen: 1 month 2 weeks ago
Joined: 2007-10-17 13:28
I'll Give It A Test

I've been thinking about doing a Portable application and this compressor will be a good opportunity to get started on it.
I'll report back about its efficiency and whether I see any "evidence" left behind.

ZachHudock
ZachHudock's picture
Offline
Last seen: 3 years 6 months ago
Developer
Joined: 2006-12-06 18:07
Great! Thanks John, I'll

Great!

Thanks John, I'll test it out tonight one I get home.

The developer formerly known as ZGitRDun8705

LOGAN-Portable
LOGAN-Portable's picture
Offline
Last seen: 8 years 6 months ago
Developer
Joined: 2007-09-11 12:24
Thanks

Nice tool! Can I suggest including support for OptiPNG? (Available at sourceforge)

When I tried OptiPNG it took ages (because it tried for each image to find the best compression) but eventually it saved some MB's. It supports TIFF, PNG and I think also GIF files. Although the compressor supposidly also supports BMP I dont know how that can be.

John Bentley
John Bentley's picture
Offline
Last seen: 11 years 11 months ago
Developer
Joined: 2006-01-24 13:26
Looks good. Is there any

Looks good. Is there any chance of getting support for png compression?

cowsay Moo
cowthink 'Dude, why are you staring at me.'

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
Not Yet Recommended

PNG recompression is not yet recommended and should be handled on an app by app basis at this time. The features in the app compressor I've been using on all apps (except Mozilla) for well over a year, so they're tried and true at this point. PNG recompression often doesn't save that much space per file (unlike UPX or ZIP/JAR recompression which can save 50% or more per file) or that much space overall in an app (most space in apps is taken up by DLLs, EXEs, JARs, etc) and can introduce problems in some apps which may not support certain PNG types (which we've learned from experience).

BuddhaCha was working on PNG compression and was interested in enhancing this (are you still, B?) so perhaps we'll start with some manual scripts and start testing them with the various apps... see what kind of space they save, what kind of compatiblity issues arise, etc.

If you're thinking for your own apps... like Xenon... you should compress the PNGs as you save them. All good image editors handle this... in Paint Shop Pro, it's Export - PNG Optimizer. In Photoshop it's Save For Web. In GIMP there are options in the Save Dialog. I'm actually going to be compressing the images I give you as part of the icon set.

Sometimes, the impossible can become possible, if you're awesome!

Travis Carrico
Offline
Last seen: 12 years 8 months ago
Developer
Joined: 2006-10-22 00:30
yeah

i agree, PNG optimization saved me a lot of space on SMC but when i tried it on Neverball it made the app crash as soon as you tried to start it. it's best to leave it off of this.

LOGAN-Portable
LOGAN-Portable's picture
Offline
Last seen: 8 years 6 months ago
Developer
Joined: 2007-09-11 12:24
What compressor did you use?

What compressor did you use? Maybe some PNG compressor gives different results than other compressors.

Although PNG compression can be handled manually, maybe we can eventually find a compression setting that is compatible with virtually every app.

I used OptiPNG to compress Dreamchess a bit more and on first glance it worked and saved another MB. It took very very long to compress the png files though.

DreamChess v0.2.0.577
Original: 10.70MB
UPX: 07.32MB
OptiPNG: 06.26MB

Travis Carrico
Offline
Last seen: 12 years 8 months ago
Developer
Joined: 2006-10-22 00:30
PNG Optimizer

I used PNG Optimizer. Is there one that is less likely to cause problems?

LOGAN-Portable
LOGAN-Portable's picture
Offline
Last seen: 8 years 6 months ago
Developer
Joined: 2007-09-11 12:24
I used OptiPNG (from

I used OptiPNG (from SourceForge)

i used the -o7 setting, it will test many settings to determine the best one.

ZachHudock
ZachHudock's picture
Offline
Last seen: 3 years 6 months ago
Developer
Joined: 2006-12-06 18:07
That sounds fair. Some test

That sounds fair. Some test scripts for PNG compression would be good to have so we can test out different options and find something that works.

The developer formerly known as ZGitRDun8705

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
Nothing

The problem is that nothing will work for all programs due to the different PNG libraries around. So, even if we add it, it won't be a default or recommended option.

Sometimes, the impossible can become possible, if you're awesome!

LOGAN-Portable
LOGAN-Portable's picture
Offline
Last seen: 8 years 6 months ago
Developer
Joined: 2007-09-11 12:24
I don't know how many

I don't know how many libraries there are, or if they can be autodetected. Is we could search for a specific DLL (and maybe version) we could make a small list of libraries. And based on these we could see what compression settings - if any - could be used.

This would require some testing with different PNG libraries and different compression tools and settings though. But maybe we could at least figure out what libraries would work.

Having said that, it's always required to test the resulting test, and having a backup of the files to restore the app to working order if compression causes problem.

I found that SCUMMVM didn't like having the zip re-compressed long ago, so also the zip libraries used could give us a hint what the application can handle automatically and optionally use other settings or no compression at all.

And perhaps overkill, but maybe its possible to determine what generated the exe file and have some exclusion based on that. The same for DLL's. Maybe there are some librarie dll's who dont like being compressed so a small list of known dll's that cause problems could be added as a kind of exclusion list. Heck, maybe the UPX compression with a certain setting could cause the re-compressed zip or png dll to not function correct. I don't know.

Eventually as with the UPX and the ZIP, the app should be tested also with the PNG compression. So users always can compress and test each step of the compression.

BTW, I recently used the upx batch and changed it to do the png compression Smile

wraithdu
Offline
Last seen: 8 years 1 month ago
Developer
Joined: 2007-06-27 20:22
Looks good. Only thing I

Looks good. Only thing I noticed is in the Uncompress .bat file, your status lines (ECHO...) still say Compressing DLLs, Compressing ZIP....etc.

millarrp
Offline
Last seen: 7 years 9 months ago
Joined: 2007-03-02 13:43
no obvious problems so far

I've tried it on a couple personal apps that I use and haven't had any problems with it.

The MAZZTer
The MAZZTer's picture
Offline
Last seen: 3 years 2 months ago
Developer
Joined: 2006-11-17 15:31
Woah

I just finished making a program just like this, except I support more archive types for 7-zip (every archive type 7-zip can compress) and have a few more options for UPX and 7-zip. In addition it can handle archives and binaries INSIDE archives, for infinite recursion (well until the stack overflows).

I also included automatic backup and before/after size comparisons, and a post-compression screen which allows you the opportunity to test your newly compressed apps to be sure they still work and automated restore of the backup for each app if they don't.

I should post it maybe... it's not entirely portable itself since it requires .NET, but ignoring that exception it's made to be entirely portable.

http://x.mzzt.net/0117.png
http://x.mzzt.net/0119.jpg
http://x.mzzt.net/0120.png
http://x.mzzt.net/0122.jpg
http://x.mzzt.net/0123.jpg
http://x.mzzt.net/0124.png
http://x.mzzt.net/0129.png

Signature automatically removed for being too awesome.

wraithdu
Offline
Last seen: 8 years 1 month ago
Developer
Joined: 2007-06-27 20:22
Even if it isn't portable,

Even if it isn't portable, this would be a great help for developers. Can you post it somewhere anyway?

mecki77
Offline
Last seen: 2 years 6 months ago
Joined: 2008-03-03 02:54
jep

it looks nice on the screenshots, could be worth a try - even if it is not portable like John's app,...

@John: By the way: it's nice, and the the hints at the options give a good help which compression to choose - it seems to work without problems...
gives it the same results, as the Developper Toolbox Batch files from the cubegames.net compression wiki at http://cubegames.net/development/compression ?

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
By Design

Yeah, by design it's supposed to be dead easy to use. I think the batch files over there were based on my original recommendations here, so it should be similar.

Sometimes, the impossible can become possible, if you're awesome!

Ryan McCue
Ryan McCue's picture
Offline
Last seen: 11 years 10 months ago
Joined: 2006-01-06 21:27
Correct.

They were modelled off what you noted on support pages and in the forums.

I'll try this in the morning. Nice to see you used the app release template Smile

"If you're not part of the solution, you're part of the precipitate."

BuddhaChu
BuddhaChu's picture
Offline
Last seen: 4 years 9 months ago
Joined: 2006-11-18 10:26
The batch files from the

The batch files from the Cubegames wiki were written by me and do use the "recommended" UPX switches and whatnot.

Cancer Survivors -- Remember the fight, celebrate the victory!
Help control the rugrat population -- have yourself spayed or neutered!

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
Some Issues

I was originally going with more options as well... but it's too complex with them and folks are more likely to encounter an issue.

Like with 7-Zip, for instance: anything less than maximum is pretty pointless as all the others just make the file bigger with no real trade off in speed and Ultra has compatibility issues with some files and some applications and takes much longer to compress without saving more than 0.5% so should not be used. As far as the types besides JAR and ZIP, we don't include those in any current (or dev test to my knowledge) apps as they're generally source files that we bundle separately.

And with UPX, we're only compressing known file types that don't have issues. You can't compress *.* or you'll end up with corrupted files in several apps (like OpenOffice.org where .py files get corrupted by UPX). We'll add to the safe list as we go.

Options that shouldn't be used and lots of extra options that aren't of much use (like 7-Zip compression level) just make it more complicated than it needs to be.

Plus, the .NET requirement is a deal breaker for any portable app. Sad

Sometimes, the impossible can become possible, if you're awesome!

The MAZZTer
The MAZZTer's picture
Offline
Last seen: 3 years 2 months ago
Developer
Joined: 2006-11-17 15:31
Wait, you can't compress the

Wait, you can't compress the .py files in OpenOffice.org?

... Oops. >_>

I have to go now... and run some apps off my flash drive... for a completely unrelated reason.

I chose to support multiple archive types because it was rather trivial to do so in my setup. All I had to do was write a short routine to open the file and check the header to determine the file type, then all that would change is the -t flag I pass back to 7-zip to recompress.

I believe the actual UPX default file types are a safe list of EXE, DLL, OCX, CPL, SCR, and maybe a couple others. My preferred choice is to compress everything, but obviously that is not good all the time (ex: it's probably not a good idea to compress NSIS' stubs it uses to build installers from).

I have made a couple other apps which are also portable but require .NET (what can I say, it's easy to make apps in) and as more time goes on we will see more and more computers with the .NET framework (right now if you accept all Windows Updates on an XP machine you get it; Vista has it installed from the get go) so it will become less of an issue.

Signature automatically removed for being too awesome.

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
Yeah

The .py issue nailed the last official release of OpenOffice.org. Mail merge and some other python-related stuff won't work as a result. I'm packaging a revision but had been hoping we'd get a new point release. And .py files are text files. I think UPX misinterpreted them as PYDs perhaps... so, essentially, UPX can mess up... so you should never feed it *.* to compress, just the files you know will work properly.

The .NET issue is likely to persist as it's an optional update on XP (majority of the world's PCs) -- and it definitely *should* be optional due to the security issues it's had over the years -- and hardly any users ever do any updates except the high-priority ones. I have .NET 2.x on my main box because I game on it and the ATI Catalyst Control Center annoyingly needed it. My laptop and virtual machines don't have any .NET runtimes on them.

Sometimes, the impossible can become possible, if you're awesome!

The MAZZTer
The MAZZTer's picture
Offline
Last seen: 3 years 2 months ago
Developer
Joined: 2006-11-17 15:31
Thanks for all the replies,

Thanks for all the replies, I have to tweak the dialog in screenshot 119 (might turn those buttons into "Add" and "Remove"... "

Signature automatically removed for being too awesome.

Patrick Patience
Offline
Last seen: 1 year 7 months ago
DeveloperModerator
Joined: 2007-02-20 19:26
Nice!

Thanks for releasing this, you did an excellent job! It's been working fine so far. Note, the read Source directory readme, you accidentally replaced with 'Comoressor'.

The MAZZTer
The MAZZTer's picture
Offline
Last seen: 3 years 2 months ago
Developer
Joined: 2006-11-17 15:31
Haha, in my own app I

Haha, in my own app I frequently accidentally typed "Compator". I think I got them all though.

Signature automatically removed for being too awesome.

BuddhaChu
BuddhaChu's picture
Offline
Last seen: 4 years 9 months ago
Joined: 2006-11-18 10:26
I agree with PP

MAZZTer: I haven't downloaded or used your app yet, but I can just tell from the screenshots it at least looks great Wink so I bet it works pretty good too.

n1

What language is it coded in?
Is it Open Source?
Can we get a copy of the source?

Cancer Survivors -- Remember the fight, celebrate the victory!
Help control the rugrat population -- have yourself spayed or neutered!

Jacob Mastel
Offline
Last seen: 9 months 4 weeks ago
Developer
Joined: 2007-06-13 19:36
Seems to work well

Nice app John. One slight issue. One option is to decompress stuff. So maybe instead of saying Compress> it should say start or next so we don't confuse some less experienced users.

Release Team Member

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
Try

Yeah, I thought of that. I'm gonna try in the next release. Few small refinements I'm gonna make for a PR2 tomorrow or this weekend and that is one I'm gonna try and do.

Sometimes, the impossible can become possible, if you're awesome!

Jacob Mastel
Offline
Last seen: 9 months 4 weeks ago
Developer
Joined: 2007-06-13 19:36
Awsome glad to hear.

It works awsome. I just compressed a bunch of installers I'd been saving and I saved about 60 mb's

Release Team Member

BuddhaChu
BuddhaChu's picture
Offline
Last seen: 4 years 9 months ago
Joined: 2006-11-18 10:26
I haven't downloaded John's

I haven't downloaded John's app or the one posted The MAZZTer above (yet), but I thought I post on a few things.

John: Yes, I'm interested in trying to add PNG "optimization" to your app. Whether or not I have enough NSIS ski11z yet, is the question. Sorta like PAM, I think it should eventually be a "group effort" (that may be painful however with communication and everything else needed to code an app as a group). I will definitely give it a go myself first.

Another feature I can envision John's app having in a future version is whitelisting. A text file with filenames with/without filepaths listed by a method so that they can be parsed and "skipped" by the UPX compressor. This would prevent files that have known problems with UPX compression to not be touched by UPX (discovered during the alpha/beta test process for example and added to the whitelist). Put the whitelist in \AppInfo or \Other directory and have the program look there for it (add official location to PortableApp spec).

If anyone is interested in seeing multiple PNG optimizers in action, I've made a version of my PortableApps™ Developer Toolbox availble on my webserver. (Yes, I know, I need to work on getting a new name since PortableApps is now a ™ of John Haller).

To use the four optimizers, put the PNG files or folder structure in the SqueezeBox directory and run the batch fi9le in that directory. The reason for the folder structure is I also use that dir as the place I do all file ops on portable apps to test them. If the UPX, 7-zip, and all the PNG optimizer executables were in that folder, UPX would error out on them all the time since they're already compressed and it wastes time.

Get the four PNG optimzers and the batch file here. If you'd like to discuss PNG optimization further, please make a new thread in the developer Forum so this one doesn't get cluttered up.

More reading on PNG Optimization: http://en.wikipedia.org/wiki/Portable_Network_Graphics#File_size_and_opt...

Cancer Survivors -- Remember the fight, celebrate the victory!
Help control the rugrat population -- have yourself spayed or neutered!

BuddhaChu
BuddhaChu's picture
Offline
Last seen: 4 years 9 months ago
Joined: 2006-11-18 10:26
Feedback

I still haven't run the program yet, but looked in the batch files first for some reason. Here's some feedback:

Separate UPX command lines can be combined into one. I don't know if they're separate to make it simpler for people to view/understand. If not, then for example the five lines of code in PortableAppsAppCompressor.bat could be combined into:

FOR /R %1 %%F IN (*.dl~,*.ex~,*.bi~,*.by~,*.ir~) DO del "%%F"

Same goes for the same kind of lines in the other batch files.

PortableAppsAppCompressorUncompress.bat has a typo on line 10: "Binraries"

Suggestion: Add "*.sfx" filetype for binary compression which is used in 7-zip. Also add "*.pk3" to rezip which are Quake 3 engine map/mod packs (renamed zip files). With the Q3 engine being Open Source, it's now in more widespread use and used in games released on PortableApps.com (still dev releases I think).

Cancer Survivors -- Remember the fight, celebrate the victory!
Help control the rugrat population -- have yourself spayed or neutered!

wraithdu
Offline
Last seen: 8 years 1 month ago
Developer
Joined: 2007-06-27 20:22
I wouldn't add .pk3. Yes

I wouldn't add .pk3. Yes they are zip files, but if I remember correctly Quake will fail if they're actually compressed zip files. I've always had to make my pak files for Q1-Q4 with no compression.

BuddhaChu
BuddhaChu's picture
Offline
Last seen: 4 years 9 months ago
Joined: 2006-11-18 10:26
I recompressed my Q3 .pk3

I recompressed my Q3 .pk3 files for the IOQuake portable app and they work fine.

Cancer Survivors -- Remember the fight, celebrate the victory!
Help control the rugrat population -- have yourself spayed or neutered!

LOGAN-Portable
LOGAN-Portable's picture
Offline
Last seen: 8 years 6 months ago
Developer
Joined: 2007-09-11 12:24
does that work?

I didn't know pk3 files were actually Zip files. But how is the performance? The pk3 files can be rather big. Doesn't compressing such big files cause problems?

And how much space is saved on them after re-compressing?

Also other games use pk3 files (Cube, World of Padman, OpenArena, UFO Alien Invasion, Alien Arena 2008, Warsow, etc) Would they all keep running fine with compressed pk3 files?

What settings did you use? (I assume you did not use solid as for that to get one file from the archive it would need to decompress the whole archive...

BuddhaChu
BuddhaChu's picture
Offline
Last seen: 4 years 9 months ago
Joined: 2006-11-18 10:26
They all work fine for me.

They all work fine for me. I've done it for Quake 3, Warsow, etc. Where you may have a problem is if Punkbuster (or a program like that) does something like an MD5 sum on the .pk3 to see if it's been tampered with. In that case, recompressing them is NOT a good idea since the file's signature will change. I don't play online so Punkbuster isn't a problem for me.

There is no "solid" compression for Zips. That's 7-zip's LZMA compression you're thinking of. 7-zip's "Ultra" compression works great at squishing the .pk3 files a little bit more.

To test if a .pk3 file is actually a zip, either attempt to open the file with a zipping program or copy the file and change the extension to .zip and see if it opens. Personally, I haven't encountered any .pk3 files that were NOT renamed zip files.

Cancer Survivors -- Remember the fight, celebrate the victory!
Help control the rugrat population -- have yourself spayed or neutered!

Shawn Faucher
Shawn Faucher's picture
Offline
Last seen: 11 years 11 months ago
Developer
Joined: 2007-10-23 22:14
UPX -t

Looks great! Unfortunately I have the same problem with it that I have had with all the other scripts etc out there.. it blindly compresses every binary and doesn't test the files afterward. UPX has a '-t' option to test a compressed file and make sure it decompresses correctly. If a file fails decompression it is extremely likely, if not certain, it will not work correctly since the first step of running a compressed binary is to decompress it.

By adding a test step and reverting to backup if it fails I've managed to run my UPX script that does this on any binaries (including compressed NSIS executables) and it has caused no errors. It correctly detects the problem binaries and reverts to backup.

As far as I have been able to determine this seems foolproof.. never had a crash or other failure with several apps I have tried this on.

formerly rayven01

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
Good Point

I'll add this in to PR2

Sometimes, the impossible can become possible, if you're awesome!

digitxp
digitxp's picture
Offline
Last seen: 9 years 10 months ago
Joined: 2007-11-03 18:33
*Bug*

It ends up extracting and compressing 1 zip file in an infinite loop Sad

Insert original signature here with Greasemonkey Script.

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
Details

You mean if there's only a zip file in the selected directory and nothing else? Specifically named zip file? Or does this happen always?

Sometimes, the impossible can become possible, if you're awesome!

digitxp
digitxp's picture
Offline
Last seen: 9 years 10 months ago
Joined: 2007-11-03 18:33
No

I was doing SMPlayer, the ratio of the space to the space on drive for the themes was so large I could save 20 megs just putting it in a .tar :P. It was named themes.zip.
I don't want to do it again, upx hogs my PC Power :(...

Insert original signature here with Greasemonkey Script.

BuddhaChu
BuddhaChu's picture
Offline
Last seen: 4 years 9 months ago
Joined: 2006-11-18 10:26
Your posts are confusing

Your posts are confusing since you mention a bug with zipping then mention not using UPX. 7-zip handles the zips, UPX handles the executables.

What the dealie-o?

Cancer Survivors -- Remember the fight, celebrate the victory!
Help control the rugrat population -- have yourself spayed or neutered!

digitxp
digitxp's picture
Offline
Last seen: 9 years 10 months ago
Joined: 2007-11-03 18:33
Okay, I'm not good at explaining.
  1. The compressor kept recompressing in an infinite loop
  2. UPX hogs my pc power, so I don't wanna redo it.

Jee, Vivia can't even edit a video Sad

Insert original signature here with Greasemonkey Script.

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
Can't Reproduce

Please give more details including the full directory structure to the path that had the infinite loop, file name, etc.

As stated, I can't reproduce this at all and I've been using a variant of these scripts for well over a year on every app here without incident. And from looking at the code itself, it's impossible for it to be stuck in an infinite loop unless you're dealing with corrupted directory structures at the filesystem level.

Sometimes, the impossible can become possible, if you're awesome!

digitxp
digitxp's picture
Offline
Last seen: 9 years 10 months ago
Joined: 2007-11-03 18:33
I stink at detail, too

But it kept extracting, compressing, and recompressing.
The path was G:\Documents\Development\SMPlayer\SMPlayer Portable\App\smplayer\
The path to the compressor was G:\PortableApps\PortableAppsCompressor\PortableAppsCompresser.exe (Or something like that)
I chose brute on upx, and checked compress jars/zips.

Insert original signature here with Greasemonkey Script.

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
Don't Use Brute

Don't use brute on any PortableApps.com dev tests or pre-releases that aren't dead simple apps (one exe and one dll, perhaps) and easy to test ALL functions. You'll wind up with some oddly compressed stuff that will bork on some systems.

Sometimes, the impossible can become possible, if you're awesome!

reta
Offline
Last seen: 13 years 4 months ago
Joined: 2007-02-01 17:45
*Bug*

I wanted compress application "Firefox_Portable_2.0.0.12_en-us.paf.exe" dir "FirefoxPortable\App", but it finished in loop (STEP 5: Compress JARs/ZIPs...) in dir "FirefoxPortable\App\firefox\chrome\".

Try to I'm with correct this mistake:

Modification: PortableAppsAppCompressor.bat
.
.
.
echo STEP 5: Compress JARs/ZIPs...
FOR /R %1 %%F IN (*.jar) DO CALL PortableAppsAppCompressorRezip.bat "%%F"
FOR /R %1 %%F IN (*.zip) DO CALL PortableAppsAppCompressorRezip.bat "%%F"
rem Fix bug loop. Delete temporary files.
FOR /R %1 %%F IN (*.OK~) DO del "%%F"

.
.
.

Modification: PortableAppsAppCompressorRezip.bat

@echo off
rem CompressZip.bat
rem recompresses a zip file to use maximum compression
rem usage: CompressZip.bat [file]
rem Fix bug loop. Check processing set.
if exist "%1.OK~" goto :eof
echo OK>"%1.OK~"
rem Fix bug loop.

echo Uncompressing file...
.
.
.

BuddhaChu
BuddhaChu's picture
Offline
Last seen: 4 years 9 months ago
Joined: 2006-11-18 10:26
uhhhh...what?

I don't see the reason you had to do that workaround in the first place.

1) Only .zip and .jar file extensions are rezipped and only the UPX program creates backup files (-k "keep" switch)
2) Only .dll, .exe, .irc, .bin, and .pyd files are compressed by UPX
3) In order for that extension to be created by UPX, you'd have to add a line in step 1, 2, or 3 in the PortableAppsAppCompressor.bat or PortableAppsAppCompressorNoZips.bat batch files.
4) There are no files that end in .ok* in the firefox directory to recompress anyways (* = some character)
5) If there are files in your firefox directory that end in .ok~, they weren't created by the rezipping or UPX batch file.
6) I just compressed/rezipped the \App\firefox folder and it didn't create any .OK~ files. Seeing as how there are no file extension beginning with .ok, there are no backup files with the .OK~ extension to create.

Be advised: it's not a good idea to compress firefox since it breaks in-place updating (firefox updating itself) if that's a feature you want to keep.

Cancer Survivors -- Remember the fight, celebrate the victory!
Help control the rugrat population -- have yourself spayed or neutered!

reta
Offline
Last seen: 13 years 4 months ago
Joined: 2007-02-01 17:45
Responded be on loop in step

Responded be on loop in step 5 in at run "PortableApps.com App Compressor 1.0 pre- Release 1"

*Bug*
digitxp's picture
digitxp (Homepage) - March 21, 2008 - 11:49am
It ends up extracting and compressing 1 zip file in an infinite loop Sad

or

Can't Reproduce
John T. Haller's picture
John T. Haller (Homepage) - March 22, 2008 - 10:18am
Please give more details including the full directory structure to the path that had the infinite loop, file name, etc.
As stated, I can't reproduce this at all and I've been using a variant of these scripts for well over a year on every app here without incident. And from looking at the code itself, it's impossible for it to be stuck in an infinite loop unless you're dealing with corrupted directory structures at the filesystem level.

UPX is OK
Recompress ZIP and JAR to be stuck in an infinite loop. Original code looks OK. I don't know why it to be stuck in an infinite loop.
File *.OK~ is secondary file, whether went ahead cycle compression ZIP. Maybe it isn't good solution.

Yes isn't necessary compress "Firefox", if is free space to USB disk. Smile
Isn't that necessary. Was it only example to test. Smile

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
Hmmm

I just ran 1.0 PR1 on Firefox Portable 2.0.0.12's FirefoxPortable\App\ directory and it finished exactly as expected. No issues. No loops. Just compressed.

So, the question is... what's wrong on these specific PCs that have this problem? No access to %TEMP%? Odd path or something? I'm at a loss.

Like I said, I've been using this script for well over a year with 0 issues.

Sometimes, the impossible can become possible, if you're awesome!

reta
Offline
Last seen: 13 years 4 months ago
Joined: 2007-02-01 17:45
Test ... NTFS and FAT32

Test: compress application "Firefox_Portable_2.0.0.12"
OS: Windows XP home sp2 czech, Windows XP pro sp2 english, Windows 2k sp4 czech

File system NTFS ... finished OK
File system FAT32 ... finished infinite loop

BuddhaChu
BuddhaChu's picture
Offline
Last seen: 4 years 9 months ago
Joined: 2006-11-18 10:26
I tried using the App

I tried using the App Compressor on my portable FAT32 fornmatted drive and now see the looping error.

UPXing worked fine. 7-zip is continuously extracting/compressing the following file

\App\firefox\extensions\inspector@mozilla.org\chrome\inspector.jar

Which just so happens to be the very first .jar file it encounters.

I deleted that whole folder to see if the batch or 7-zip loop on any of the .jar files in \App\firefox\chrome and 7-zip extracted/compressed all 7 .jar files then started all over doing the same thing with the first .jar file. i though at first maybe 7-zip didn't like the @ character for some reason. the second set of files have all alphabetical characters in the directory tree structure so I'm at a loss for the moment.

I'm going to see if my re-jarring script has an issue on a FAT32 drive.

Ok, my script loops too. WTF?

Temp solution is: Don't use the Jar/Zip option in the PortableApp App Compressor on FAT32 formatted media until this gets sorted (if ever).

Cancer Survivors -- Remember the fight, celebrate the victory!
Help control the rugrat population -- have yourself spayed or neutered!

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
Confirmed

I can confirm that this is FAT32-specific. I'm adding a note to the thread about this and have a possible fix.

Sometimes, the impossible can become possible, if you're awesome!

gmatyola
Offline
Last seen: 4 years 7 months ago
Joined: 2007-01-08 12:29
A possibly silly idea...

Call it the PortableApps Application Compression Kit. (PaACK)?

Thanks for the work, John!

digitxp
digitxp's picture
Offline
Last seen: 9 years 10 months ago
Joined: 2007-11-03 18:33
Yeah

that sounds nice Smile

Insert original signature here with Greasemonkey Script.

LOGAN-Portable
LOGAN-Portable's picture
Offline
Last seen: 8 years 6 months ago
Developer
Joined: 2007-09-11 12:24
Or just PortableApps

Or just PortableApps Compression Kit. (PACK)

PortableApps Application is like saying it twice.

Smile

BuddhaChu
BuddhaChu's picture
Offline
Last seen: 4 years 9 months ago
Joined: 2006-11-18 10:26
I agree

"PACK" is better. Nice acronym too that actually describes it's function. Smile

Cancer Survivors -- Remember the fight, celebrate the victory!
Help control the rugrat population -- have yourself spayed or neutered!

rab040ma
Offline
Last seen: 12 months 2 days ago
Joined: 2007-08-27 13:35
Bullets

Your third bullet might be a bit ambiguous. "Reside", to me, suggests a permanent location, rather than a place to work on it.

MC

BuddhaChu
BuddhaChu's picture
Offline
Last seen: 4 years 9 months ago
Joined: 2006-11-18 10:26
Feedback for PR2

The lack of the RealProgress plugin prevented me from compiling the app. If you need it, get it here: http://nsis.sourceforge.net/File:RealProgress.zip

Question: Is Registry.nsh really required on line 54? The app doesn't use the registry AFAIK.

EDIT: I just compiled the app with Registry.nsh commented out and it worked fine. I now assume it's not needed. (Launcher is 1k smaller now...Wheee!!!! Wink )

Cancer Survivors -- Remember the fight, celebrate the victory!
Help control the rugrat population -- have yourself spayed or neutered!

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
1.0 Pre-Release 2 Posted

I just posted 1.0 Pre-Release 2. It fixes the issue with ZIP/JAR recompression causing an endless loop on FAT32 drives. It also adds in UPX testing thanks to Shawn, the ability to relaunch the app compressor from the finish screen (easy to compress another app) and a display of the directory size before and after as well as the space saved by compression. The console is now hidden by default (upgraded installs will still leave it shown) and the ZIP/JAR compression was dialed down to 7 aka Maximum as it works faster and has fewer compatibility issues with applications than Ultra does.

Please give it a try and post your feedback!

Sometimes, the impossible can become possible, if you're awesome!

BrianAll
Offline
Last seen: 3 years 1 month ago
Joined: 2008-02-13 13:44
Great!

I'll try this right away...
(As soon as my 700mb Linux distro finishes downloading Wink )

Shawn Faucher
Shawn Faucher's picture
Offline
Last seen: 11 years 11 months ago
Developer
Joined: 2007-10-23 22:14
Worked on GnuCash

Lots of error messages flying around during the process (file not found errors) but it appeared to work correctly on GnuCash this time around.

formerly rayven01

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
Yeah

There will show an error on a file that is already UPX compressed (yours will, too, actually).

And there are several errors at the end due to the fact that I have to rename file extensions back and forth to get around the way FAT32 directories are handled (it gets around the looping error, but it isn't pretty).

Sometimes, the impossible can become possible, if you're awesome!

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
1.0 Pre-Release 3 Posted

Just a quick bug fix to fix the issue introduced in Pre-Release 2 with handling paths with spaces.

Sometimes, the impossible can become possible, if you're awesome!

LOGAN-Portable
LOGAN-Portable's picture
Offline
Last seen: 8 years 6 months ago
Developer
Joined: 2007-09-11 12:24
Oops, I seem to have replied

Oops, I seem to have replied to another thread my message of successfully compressed an app... Sorry..

BrianAll
Offline
Last seen: 3 years 1 month ago
Joined: 2008-02-13 13:44
Works great

I tested this with no problems.
Hope to save a lot of space on my drive with this. Biggrin
Good job & thanks. Smile

Nathan9222
Nathan9222's picture
Offline
Last seen: 9 years 2 months ago
Developer
Joined: 2007-12-06 22:35
John i think to make things easier......

You could always add a feature that says "Compress Portable Apps" and when you click on it, you click on the "Portable Apps" Folder that has all your apps, then if there was a way to add a variable that would only compress Files in the "App" folders, then when you hit compress it will compress everything in the "App" folders. plus if u do it like this it would be faster than having to go through every app. Other than that, great app, I have used it a lot.

An eye for an eye makes the whole world blind.
Mahatma Gandhi,
Indian political and spiritual leader (1869 - 1948)

ace_dent
Offline
Last seen: 12 years 2 months ago
Joined: 2005-12-13 10:08
pngslim - png optimizer

Hi,

I've continued to develop my script (first developed for portable OpenOffice). You can see the thread here:
http://www.jonof.id.au/forum/index.php?topic=846.45

My focus has been working with FOSS projects to get the compressed artwork upstream to source. My methods have little regard for crunching time- just getting tiny files. However, I'd be happy to contribute to a safe-generic-fast png optimizer (if you need any help that is!). One suggestion is that you stop using advpng- it is kinda buggy. Advdef is a safer bet.

Cheers,
Andrew

reta
Offline
Last seen: 13 years 4 months ago
Joined: 2007-02-01 17:45
ZIP/JAR don't work in select directory

Compression ZIP\JAR don't work in select directory. Work in only of other subdirectories.

Start in directory:
X:\PortableApps\FirefoxPortable\App\

X:\PortableApps\XXXPortable\App\test.zip ... NO compression
X:\PortableApps\XXXPortable\App\XXX\test.zip ... OK compression

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
Got it

I have a fix prepped for PR3 that fixes this.

Sometimes, the impossible can become possible, if you're awesome!

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
Prepping PR3 / Rename?

I'm prepping PR3 which fixes another space in path bug.

I'm also thinking about changing the name to PortableApps.com AppShrinker (or AppShrink). It's a few chars shorter and I think it's catchier. I was gonna call it AppCompress, but that's the name of something else.

Sometimes, the impossible can become possible, if you're awesome!

digitxp
digitxp's picture
Offline
Last seen: 9 years 10 months ago
Joined: 2007-11-03 18:33
Hmm

P Double A C
UPPA (Ultimate Packer for Portable Apps)

Insert original signature here with Greasemonkey Script.

Shawn Faucher
Shawn Faucher's picture
Offline
Last seen: 11 years 11 months ago
Developer
Joined: 2007-10-23 22:14
Prefer Compressor

Not a fan of Shrinker/Shrink, personally.. I like the original name. This is a tool for devs after all; better to have it say plainly what it does rather than get cutesy.

formerly rayven01

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
Compressor

Also debating just calling it PortableApps.com Compressor.

Sometimes, the impossible can become possible, if you're awesome!

Patrick Patience
Offline
Last seen: 1 year 7 months ago
DeveloperModerator
Joined: 2007-02-20 19:26
I like that

Although AppShrink is catchy, I think just PortableApps.com Compressor works well, too.

LOGAN-Portable
LOGAN-Portable's picture
Offline
Last seen: 8 years 6 months ago
Developer
Joined: 2007-09-11 12:24
I would opt for this. Why? -

I would opt for this. Why?

- Using Portable Apps Apps Compressor sounds weird due to uveruse of the App.
- Make it just PortableAppsCompressor. The Installer will make clear its from PortableApps.com
- Or PortableAppsCompact or PortableAppsCompactor
- PortableAppsSchrink almost sounds like DVD Shrink, shrink any DVD (App?)
- Use PACK would also be nice Smile

PortableAppsCompressor would be my name of choice. (Keeps it a bit in line with PortableAppsMenu, PortableAppsBackup and PortableAppsUpdate (Who also omit the .com)

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
.com

Actually, everything has the .com going forward. PortableApps.com Platform (which consists of the PortableApps.com Menu and PortableApps.com Backup, etc), PortableApps.com Format, PortableApps.com Themes, etc. That's one reason I was considering just PortableApps.com Compressor.

Sometimes, the impossible can become possible, if you're awesome!

LOGAN-Portable
LOGAN-Portable's picture
Offline
Last seen: 8 years 6 months ago
Developer
Joined: 2007-09-11 12:24
If you like, its

If you like, its ok.

Personally I think the .com is overkill and stating the obvious. Personally I would choose for PortableApps as it's more pleasing i think.

Ryan McCue
Ryan McCue's picture
Offline
Last seen: 11 years 10 months ago
Joined: 2006-01-06 21:27
Yes, but

The trademarked name is PortableApps.com, not PortableApps (correct me if I'm wrong John).

"If you're not part of the solution, you're part of the precipitate."

dark_yux
Offline
Last seen: 7 years 6 months ago
Joined: 2007-10-23 11:23
Well...

Just call it Compressor or better yet Compress, one really cool word like Toucan.

self.path = path if self.path == None else self.path

John T. Haller
John T. Haller's picture
Online
Last seen: 10 min 37 sec ago
AdminDeveloperModerator
Joined: 2005-11-28 22:21
1.0 Pre-Release 4 Posted

I just posted PR4. This fixes a final bug with spaces in paths and recompression of Zip/Jar and now allows you to recompress Zip/Jars in the base directory you select to compress.

A new feature was added to accept a path to compress on the command line. This will automatically skip the welcome screen and pre-fill the path to compress in the options window so you can drag a folder from explorer and drop it on the AppCompactor.exe. It also makes it easy to set up App Compactor with a Send To within Windows.

Finally, the name was changed to App Compactor as AppCompress is a trademark of another company and App Compressor may be confusingly similar. I'm not married to the name and was also considering AppShrink or something similar.

Sometimes, the impossible can become possible, if you're awesome!

Steve Lamerton
Steve Lamerton's picture
Offline
Last seen: 7 years 10 months ago
Developer
Joined: 2005-12-10 15:22
Do

we have to have the extra app in there, is seems a little odd to have it twice? We could just call it the PortableApps.com Compactor.

Patrick Patience
Offline
Last seen: 1 year 7 months ago
DeveloperModerator
Joined: 2007-02-20 19:26
Ditto

I'd have to ditto that. Smile (wow, that is becoming so overused now. :P)

Pages

Topic locked