You are here

PortableApps.com AppCompactor 1.0 Pre-Release 5

35 posts / 0 new
Last post
John T. Haller
John T. Haller's picture
Offline
Last seen: 8 hours 49 min ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
PortableApps.com AppCompactor 1.0 Pre-Release 5

THIS HAS BEEN RELEASED. THANKS FOR TESTING.
https://portableapps.com/apps/utilities/appcompactor

Application: PortableApps.com AppCompactor
Category: Utilities
Description: PortableApps.com AppCompactor allows you to easily compress application to decrease size on disk and increase performance when running from a slow media (like flash drives). It packages UPX and 7-Zip within an easy to use wizard allowing you to easily compress an app's EXEs, DLLs and other binaries using UPX in either NRV2E, NRV2D, LZMA or BRUTE mode as well as recompressing ZIP and JAR files to their maximum level using 7-Zip. It also has an option to DECOMPRESS an app's binary files that were previously compressed with UPX. Just select the directory and all the DLL, EXE, ZIP, JAR, IRC, PYD and BIN files within that directory will be compressed. Depending on the files, the PortableApps.com AppCompactor can decrease an app's size by 50% or more without affecting functionality. This same tool has been used on PortableApps.com's most popular apps including OpenOffice.org Portable, Pidgin Portable, GIMP Portable and more.

Download PortableApps.com AppCompactor 1.0 Pre-Release 5 [0.7MB download / 0.8MB installed]
(MD5: 1e5bf21aad90cb6ed3537b24da780a7d)

View a Screenshot of Compression Options

Helpful tips:

  • Don't compress NSIS launchers with CRC enabled (which they all should), they won't run
  • It's often easiest to just select the AppName\App directory in a portable apps
  • It's best to have the app you're compressing reside on a hard drive (at least temporarily) rather than a flash drive to improve performance and decrease writes to your flash drive
  • You can disable the Welcome screen or have the console window stay by editing the settings.ini in the Data directory
  • ONLY USE NRV2E for PortableApps.com Releases (including development tests and pre-releases) unless it's a small app (one exe and a dll or two) and very easy for you to test every feature and function on every Windows OS.

If you're using this on your apps for release, please mention it Smile

This is just a quick update to fix some bugs before I switch over to the file walking procedure that hea was kind enough to write for us. The old thread is here.

Thanks!
John

Release Notes:

1.0 Pre-Release 5 (2008-10-09): Fixes and Enhancements:

  • ENHANCEMENT: Now checks for changes to directory and file counts to detect files lost or temp files left behind
  • BUGFIX: Properly retains uncompressible files when directories have spaces

1.0 Pre-Release 4 (2008-04-10): Fixes and Enhancements:

  • ENHANCEMENT: Now accepts path on the command line, skips welcome and prefills path to compress making it easy to drag and drop a path onto the EXE from Explorer or set it up with a Sent To in Windows
  • BUGFIX: Paths with spaces now properly recompress Zips
  • BUGFIX: Now recompresses Zips and Jars in the base directory selected
  • OTHER: Name changed to PortableApps.com App Compactor to avoid possible trademark confusion

1.0 Pre-Release 3 (2008-03-25): BUGFIX: Now works on paths with spaces again

1.0 Pre-Release 2 (2008-03-25): Fixes and Enhancements:

  • BUGFIX: ZIP/JAR recompression no longer loops on FAT32 drives
  • ENHANCEMENT: UPXed files are tested to ensure they decompress (thanks Shawn!)
  • ENHANCEMENT: Directory size before and after and space saved shown on finish screen
  • ENHANCEMENT: Checkbox to compress another app added to final screen
  • ENHANCEMENT: Console window closed by default to speed process
  • ENHANCEMENT: ZIP/JAR compression set to maximum instead of Ultra to improve compatibility and runtime performance
  • MISC: Spelling errors and inconsistencies fixed

1.0 Pre-Release 1 (2008-03-19): Initial Release

horusofoz
horusofoz's picture
Offline
Last seen: 1 year 2 months ago
Joined: 2008-04-03 22:45
Thanks John. Though I can't

Thanks John. Though I can't test this with True PAF stuff I'll trial it on some PAC ported apps. Cheers : )

EDIT: Just realised I better ask before testing. Is this safe to do with PACed apps?

PortableApps.com Advocate

John T. Haller
John T. Haller's picture
Offline
Last seen: 8 hours 49 min ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
Don't Know

PAC is unsupported.

Sometimes, the impossible can become possible, if you're awesome!

BuddhaChu
BuddhaChu's picture
Offline
Last seen: 5 months 2 weeks ago
Joined: 2006-11-18 10:26
If this is for PortableApps,

If this is for PortableApps, why is there an option for BRUTE or LZMA? In the past you said those cause problems for some system so only NRV2E should be used.

That is, unless your intent is for this to be used for things other than PortableApps also.

Cancer Survivors -- Remember the fight, celebrate the victory!
Help control the rugrat population -- have yourself spayed or neutered!

John T. Haller
John T. Haller's picture
Offline
Last seen: 8 hours 49 min ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
Anything

You can use them for anything you'd like. But only NRV2E should be used for PortableApps.com releases as I stated right in the post.

Sometimes, the impossible can become possible, if you're awesome!

J Neutron
Offline
Last seen: 7 months 5 days ago
Joined: 2008-06-10 19:26
Hmmm

Link doesn't work. Could you please check it out?

Thanks!

Jim

neutron1132 (at) usa (dot) com

John T. Haller
John T. Haller's picture
Offline
Last seen: 8 hours 49 min ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
Works

It works fine for me. Goes to SF and downloads from your mirror of choice.

Sometimes, the impossible can become possible, if you're awesome!

OliverK
OliverK's picture
Offline
Last seen: 3 years 5 months ago
Developer
Joined: 2007-03-27 15:21
kept crashing firefox this

kept crashing firefox this morning on me. I'm trying again and I'll let you know . . .

*EDIT* Fails on me. But MAZZter's link worked.

Too many lonely hearts in the real world
Too many bridges you can burn
Too many tables you can't turn
Don't wanna live my life in the real world

Simeon
Simeon's picture
Offline
Last seen: 10 years 1 month ago
DeveloperTranslator
Joined: 2006-09-25 15:15
same

here. tried 5 mirrors....

"What about Love?" - "Overrated. Biochemically no different than eating large quantities of chocolate." - Al Pacino in The Devils Advocate

The MAZZTer
The MAZZTer's picture
Offline
Last seen: 1 year 11 months ago
Developer
Joined: 2006-11-17 15:31
http://superb-east.dl.sourcef

http://superb-east.dl.sourceforge.net/sourceforge/portableapps/PortableA...

Here's one of the mirrors that works for me. For some reason it likes auto-selecting a down mirror for me. So I have to go to the portableapps project page and try downloading from there.

Signature automatically removed for being too awesome.

J Neutron
Offline
Last seen: 7 months 5 days ago
Joined: 2008-06-10 19:26
Thanks

Thanks for the link. I finally got it.

Jim

neutron1132 (at) usa (dot) com

BuddhaChu
BuddhaChu's picture
Offline
Last seen: 5 months 2 weeks ago
Joined: 2006-11-18 10:26
Thanx. The Voxel mirror is

Thanx. The Voxel mirror is the only mirror working for me at this time. If I hit the link in the original post, it tries all the mirrors and they all fail. That was happening this morning so I gave it until this afternoon and still the redirection is failing and manually selecting all mirrors also fails. I don't know why the link to the Voxel download works, but it does here.

SourceForge availability/"service level" has gone downhill ever since they started changing things a few months ago.

Cancer Survivors -- Remember the fight, celebrate the victory!
Help control the rugrat population -- have yourself spayed or neutered!

Patrick Patience
Offline
Last seen: 4 years 11 months ago
DeveloperModerator
Joined: 2007-02-20 19:26
Thanks

After like 7 failed attempts, that did the trick.

Bart.S
Offline
Last seen: 6 months 1 week ago
Developer
Joined: 2008-07-23 07:56
Question

Hi John,
I ask here due to no experienced user/dev answered in my thread. Could I compress all EXEs, DLLs without legal issues?
Most of the DLLs in FreeMat are copyrighted by Trolltech ASA, so I'm not sure.

J Neutron
Offline
Last seen: 7 months 5 days ago
Joined: 2008-06-10 19:26
For yourself of for distribution?

What are you going to do with these compressed EXEs and DLLs?

If you keep them for yourself, no problem.

If you want to distribute them, no way.

Jim

neutron1132 (at) usa (dot) com

Bart.S
Offline
Last seen: 6 months 1 week ago
Developer
Joined: 2008-07-23 07:56
Ummm,

follow the link and you will see Blum

J Neutron
Offline
Last seen: 7 months 5 days ago
Joined: 2008-06-10 19:26
So?

Let's see... you say they are copyrighted, and you want to distribute them?

I already gave you my opinion. Following the link helped... how?

Jim

neutron1132 (at) usa (dot) com

Bart.S
Offline
Last seen: 6 months 1 week ago
Developer
Joined: 2008-07-23 07:56
Ok

Following the link:

  • You see what i am going to do with these EXEs and DLLs
  • If you download the Dev Test 1, you will see the DLLs in App\FreeMat\Contents\bin and App\FreeMat\Contents\Plugins\imageformats

Thanks for your opinion! Well, I won't compress the files for Dev Test 2, coming this monday.

App Guy
Offline
Last seen: 11 years 1 month ago
Joined: 2008-03-16 14:24
Virus?

I know this is a false positive but AVG is marking upx.exe as trojan horse. Tried to send it to virustotal.com but couldn't do it. Could someone else try it and see what virustotal comes up with?

May the Shwartz be with you

getco
Offline
Last seen: 3 years 8 months ago
Joined: 2008-08-03 05:31
I'm a rookie here but

if I understand correctly I can further compact PortableApps' applications which leads me to the question why wouldn't they come compressed by default?

BuddhaChu
BuddhaChu's picture
Offline
Last seen: 5 months 2 weeks ago
Joined: 2006-11-18 10:26
1) This is for developers to

1) This is for developers to compact their apps prior to deployment (any apps, not just PortableApps).

2) Some apps (Mozilla) come to you uncompressed due to licensing. You the end-user can then compact them "legally".

Cancer Survivors -- Remember the fight, celebrate the victory!
Help control the rugrat population -- have yourself spayed or neutered!

m2
Offline
Last seen: 13 years 9 months ago
Joined: 2006-12-12 12:00
Some suggestions: - kzip

Some suggestions:
- kzip generates smaller zips than 7zip.
- there are many file extensions that are just renamed zips, not only .jar. All Open Document formats for example.
- How about jpg / gif / png / pdf / chm optimization?
- Advanced options? I always use upx with --best --nrv2e --all-filters --crp-ms=999999

"Those people who think they know everything are a great annoyance to those of us who do." Asimov

wk
wk's picture
Offline
Last seen: 1 year 5 months ago
Joined: 2007-09-05 12:31
for pics you can use png optimizer portable

"Lorem ipsum dolor sit amet, consectetur adipisici elit, sed eiusmod tempor incidunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis .." Friday Next -
"May The Schwartz be with You!" Yogurt the Yoda

m2
Offline
Last seen: 13 years 9 months ago
Joined: 2006-12-12 12:00
Thanks, I missed it. But I

Thanks, I missed it.
But I don't think that possible breaking application is a reason to skip png recompression - upx breaks executables sometimes too, I've seen this happening 2 or 3 times.
The same with jar...
Everything needs to be carefully tested after optimization anyway.

"Those people who think they know everything are a great annoyance to those of us who do." Asimov

John T. Haller
John T. Haller's picture
Offline
Last seen: 8 hours 49 min ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
True

Yes, UPX can cause issues with some executables. The AppCompactor attempts to check for many of these by keeping a backup when compressing and then testing the UPXed executable using UPX's test command. If it fails, the AppCompactor will restore the backup. If the test succeeds, then the AppCompactor will keep the UPX compressed executable.

Sometimes, the impossible can become possible, if you're awesome!

John T. Haller
John T. Haller's picture
Offline
Last seen: 8 hours 49 min ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
Some Issues

7-Zip can actually create even smaller ZIPs than we have it set to. We use -mx=7 instead of -mx=9 as I've found that max compression produces issues with many apps whereas 7 works with most apps. 7-Zip can usually create smaller ZIPs than kzip and kzip is closed source anyway which would cause distribution issues.

Most apps only have ZIPs and JARs within them. I haven't seen any with ODF files, for example. If we have some examples, I'd consider adding them to the list of files.

Compressing of JPG/GIF would be lossy and out of the question. PNG compression can cause issues as some of the apps we've seen with PNG support don't fully support the PNG spec. CHM optimization can also cause issues as it doesn't support the full array of ZIP specs.

The compactor uses --best --?? --crp-ms=999999 where ?? is the compression method selected (nrv2e is the default and recommended). The --all-filters option usually doesn't gain much size benefit when it does make a difference. It often doesn't, as indicated in the UPX manual ("This may improve the compression ratio in some cases, but usually the default filter gives the best results anyway.") And it adds extra time to the compression process.

Sometimes, the impossible can become possible, if you're awesome!

m2
Offline
Last seen: 13 years 9 months ago
Joined: 2006-12-12 12:00
kzip is closed source anyway

kzip is closed source anyway which would cause distribution issues
You don't have to distribute kzip.
I think that sb. who's advanced enough to create portable apps will be able to download it when given a link.

Most apps only have ZIPs and JARs within them. I haven't seen any with ODF files, for example. If we have some examples, I'd consider adding them to the list of files.
Get yourself a free copy of OpenOffice.

Compressing of JPG/GIF would be lossy and out of the question.
Not necessarily. I'm not sure how about gifs, but jpgs often don't have optimal huffman trees.

PNG compression can cause issues as some of the apps we've seen with PNG support don't fully support the PNG spec. CHM optimization can also cause issues as it doesn't support the full array of ZIP specs.
Yes, there may be issues. My take: Quality over quantity, do more tests and you'll be good.

The compactor uses --best --?? --crp-ms=999999 where ?? is the compression method selected (nrv2e is the default and recommended). The --all-filters option usually doesn't gain much size benefit when it does make a difference. It often doesn't, as indicated in the UPX manual ("This may improve the compression ratio in some cases, but usually the default filter gives the best results anyway.") And it adds extra time to the compression process.
I know. I want to get the best and I'm willing to pay with longer time. Maybe some PA.com developers would be willing to spend some more too?

Another idea: multitasking. You can run several copies of upx at once.

"Those people who think they know everything are a great annoyance to those of us who do." Asimov

John T. Haller
John T. Haller's picture
Offline
Last seen: 8 hours 49 min ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
Integrated, Just-Works Solution

The AppCompactor is designed to be an integrated, It Just Works solution. Of course a developer could use kzip, but it won't by them anything over 7-Zip unless you sacrifice compatibility with some apps (which we could do with 7-Zip, too).

OpenOffice.org only has a few ODF files in it. Only one ODT. No ODSes. There are the templates, of course, but I wouldn't want to alter those at all for fear of causing issues. I've found that while you can recompess OPF files (since they are ZIPs) it causes compatibility issues.

I hadn't thought about the non-optimal huffman trees issue in JPGs, that's a good point. Some PNGs can be compressed without sacrificing quality. But the issue is that a certain PNG may only cause issues with a certain app in a certain circumstance. Some are obvious (like compressing PNGs with Brutal Chess just caused the app to crash) but some may not be. I think it'd make more sense for it to be a separate compressor for now... or maybe an optional (off by default) feature of the AppCompactor.

I think you're right about the time thing in terms of -all-filters. Maybe adding it as an option would be handy for devs. I think keeping it off in the first release by default and letting folks try it will help and if most folks agree, we'll make it a recommended option.

On multi-tasking, that's actually an idea I was toying with. Running it asynchronously is definitely possible once I switch to haustin's file-walking routine over the current batch file. (It's not easily implemented with the current batch file.) I'd need to have a way to determine whether the current PC is multi-core/multi-CPU... or just make it an option for the user to select.

Sometimes, the impossible can become possible, if you're awesome!

m2
Offline
Last seen: 13 years 9 months ago
Joined: 2006-12-12 12:00
Kzip gives the smallest

Kzip gives the smallest files, isn't it what it's all about?
Everything that uses zlib or something compatible is going to decompress it's output. Sure these are not all programs, but I wouldn't expect more problems than with any other recompression.

I've found that while you can recompess OPF files (since they are ZIPs) it causes compatibility issues.
Could you tell some more? I've been doing this, but maybe I should stop?

One more:

Run upx on all files. It ususally doesn't hurt.
Actually I've been thinking about doing the same with zips / gzips too, but it might not be the best option...

"Those people who think they know everything are a great annoyance to those of us who do." Asimov

John T. Haller
John T. Haller's picture
Offline
Last seen: 8 hours 49 min ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
Comparison, ODF compression, UPX all

The last couple comparison sites I checked, 7-Zip on max was better than kzip on max. But I've found that max compression of the zips causes issues with JARs and even ZIPs with some apps, so I set it to one setting below that. (7-Zip has 1, 3, 5, 7, 9 as settings, so we use 7) If we could find a commonality, we could switch it on and off for specific apps.

On the ODF file compression issues, I think I'd tried compressing it at mx=9 using 7-Zip and OpenOffice.org 2.something couldn't open it. Same thing happened when I tried it with a DOCX file at Word, I think. And I know it happens with CHM, too. So, it seems that something about the max compression setting causes issues with some implementations of ZIP.

I had been debating running UPX on all files, but we ran into an issue with some. Some python scripts for instance, which aren't binaries, UPX will compress and corrupt them. That affected a test release last year or earlier this year. So, I decided to err on the side of caution and only compress files that we know work right assuming we'd add to it over time as we find other files we should be compressing. Currently, we compress EXE, DLL, PYD, IRC and BIN.

Sometimes, the impossible can become possible, if you're awesome!

m2
Offline
Last seen: 13 years 9 months ago
Joined: 2006-12-12 12:00
Can you post a link to a

Can you post a link to a comparison, where 7zip wins?
All I tested or found shows kzip to be better.

With Deflate 7zip has -8 option.

Thanks for the info about the odf with -mx9, I'll take a deeper look.

"Those people who think they know everything are a great annoyance to those of us who do." Asimov

John T. Haller
John T. Haller's picture
Offline
Last seen: 8 hours 49 min ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
Links

Here is the last one I'd found:
http://molhanec.net/old/archivers.php

Unfortunately, it's older versions. The other comparisons I've seen don't include KZIP. As a quick test, I took OpenOffice.org 3's Program directory which contains 19.6MB of EXEs, DLLs, INIs, a BMP and other files. 7-Zip on max got it down to a 3.44MB zip. KZIP got it down to a 3.51MB zip and took probably 10x as long to run.

I thought -8 when compressing in 7-Zip just becomes -7. At least the manual says only 1/3/5/7/9 are acceptable.

Sometimes, the impossible can become possible, if you're awesome!

m2
Offline
Last seen: 13 years 9 months ago
Joined: 2006-12-12 12:00
The link doesn't work.

The link doesn't work. Pardon

Yep, -mx8 is undocumented. But it's different, really. I tried BZIP, Deflate, Deflate64, LZMA all the way 1..9 and in all but LZMA 8 was different.
2,4,6 were not, in some modes all lower values were the same.

When I tried kzip with both /s1 and /s2, each with all recommended /b values kzip always won by 1-5%. Yes, it is very slow, quick tests are not enough. Blum

"Those people who think they know everything are a great annoyance to those of us who do." Asimov

kai.inouye
kai.inouye's picture
Offline
Last seen: 12 years 11 months ago
Developer
Joined: 2008-02-03 20:12
Thanks, but I cant download

Thanks, but I cant download it... wont download...

m2
Offline
Last seen: 13 years 9 months ago
Joined: 2006-12-12 12:00
http://portableapps.com/node/

"Those people who think they know everything are a great annoyance to those of us who do." Asimov

Topic locked