Revisions
All,
OK, I said I was going to post this after the next "dam" revision, just didn't expect it quite so soon so it might be a little disjointed 
, so here it goes.
I am concerned with the number of "Revisions" [i.e. Bug Fixes] being released.
I think this is a result of John being pushed [to put it kindly] recently to get things out faster.
I think that releasing Apps that need "revisions" so soon after release does not reflect well on our current testing procedures.
At this point I really feel like I need to wait at least a week before downloading the "New" apps and probably a couple of days for the updates.
Now I know that this post is going to make many of you unhappy, I'm sorry about that.
But I really saw this coming. In the past revisions were rare.
[The recent updates with new splash screens and digital signing don't count]
There was a reason for that, "It will be ready when it's ready"
[and a famous paraphrase: "We will release no app before it's time"]
It may be time for us to take a step back,
If the app is not up to par it should not be released.
More Thorough testing is Needed.
The most recent list is:
Task Coach Portable 0.69.1 Revision 3
PeaZip Portable 1.11b [*]
PokerTH Portable 0.6.1 Revision 2
InfraRecorder Portable 0.44.1 Revision 3
GIMP Portable 2.4.5 Revision 2
GIMP Portable 2.4.4 Revision 2
These all since "Black Thursday", March 6th 2008
That's 6 in 12 days !!!
In a recent post someone mentioned that we were past the "24 hour turn around time" for the update of an app. I think the 24 hour turn around time is a bad idea, A Very Bad Idea!
I think we need to start issuing a FINAL-PreRelease for apps that will need a Thorough Vetting for a week before released. That is what The Beta Testers have volunteered for and we should make use of them.
Post the FINAL-PreRelease and Wait for a Good Number of the Testers to report back with what they found and didn't find.
"Works Fine" doesn't tell us anything.
"Leaves Nothing Behind" doesn't tell us anything unless you tell us where you looked and or how you tested.
Please note that I am writing this out of concern for the Reputation of PortableApps.com, the Quality of the Apps, and Love of John, who has taken a lot of shit recently because of his cautious concerns.
[*] By the way PeaZip Portable 1.11 is a special case.
But should we have been releasing it with the licensing questions involved in the first place?
Respectfully submitted,
Tim
Visit the Community page
Join our forums
Subscribe to our email newsletter
Subscribe with RSS
Follow us on Facebook
Follow us on LinkedIn
Follow us on Twitter
Follow us on Mastodon
Follow us on BlueSky
I agree, much to much revisions. Things like Task Coach was just a small thing, and it had been in testing for a while, apparently I just botched it at one point, and should have throughly tested it one last time myself before releasing. Now I know in the future I should test myself one last time, throughly.
In John's case, yet he's rushed and it's not fair. Some of them were honestly really stupid like PuTTY not creating the settings folder, Sumatra handling the wrong file extension, and the wrong GIMP binaries. I applaud John for working so hard on trying to get them all out, but it's best they are delayed, and have some good 15/20 minute testing to catch those things like PuTTY's settings directory not being created, Sumatra handling the wrong extension, and the wrong GIMP binaries.
However, things like Task Coach and InfraRecorder are not as dumb, but I should have caught mine, and Travis can't catch a 'travis' directory on his computer, but someone else may or may not have noticed.
For clarification, by stupid I mean silly.
'Revisions' as they are kindly called, are re-releases to fix issues that slipped past beta quality testing. It's obvious that when an app is released into the mainstream changes are higher some issue surface than having a smaller number of dedicated beta testers. Most people will download the releases obviously.
It might be so less issues turn up with a longer pre-release period, and certainly the "oops, small issue" thingies will mostly surface in time. The less obvious things turn up eventually.
But what is someone forgot to update the appinfo.ini, or leaves references to another application in a readme. These minor issues happen sometimes when using another application as template. These issues can simply be avoided. (I use the installer to write the correct info into appinfo.ini for example. I never need to bother with the file again as long as the main file is up-to-date)
The bigger problems is changed behavior of an app that needs the launcher or installer be adapted. A simple typo may very well cause undesired effects.
A checklist for Portable Apps testers or some way to automatically monitor registry and folders created (and not moved into settings) would help. And actually using the app for a while.
Hopefully the betatester group will grow so more people will be able to do the last checks on an application. The more people that test an application, the fewer issues turn up post release. (Unless all beta testers test if the appinfo.ini is correct
I check AppInfo, Readmes, everything. I don't think Tim ever said anything about updating for those reasons though, the revisions here or for dumb little bugs that could be easily avoided with some quick 15 minutes testing.
How hard would it be to devise a test suite, or at least a check list, that exercises the likely candidates for dumb little bugs?
MC
There's not really any likely candidates, I mean, handling the wrong file extension, not creating a settings directory, wrong binaries...
Automatically generating readme's, appinfo.ini and maybe even the help.html would make at least not needed to check all those files individually and thus more time to check for real issues like you mentioned above.
How to you plan to do that? You write something, and we'll try it.
I was thinking of coding a little script myself. Dunno if I'll ever get around to it...
Cancer Survivors -- Remember the fight, celebrate the victory!
Help control the rugrat population -- have yourself spayed or neutered!
I just let the installer write appinfo.ini and could as well write the other text readme files.
I have created an paf.exe for DreamChess which demonstrated the writing of the ini after installation. I plan to make a !define switch to turn on/off writing the readme files (including the correct appname) as well soon.
The apps that we're portablizing are all too different for any sort of automated test to work very well. A checklist of things that need to be tested, however, is a no-brainer. I can think of many things to add to it offhand:
formerly rayven01
That can almost all be automated. e.g. The format structure can be checked with IfFileExists, ReadINIStr, etc., the virus scans can be automated, etc.
"If you're not part of the solution, you're part of the precipitate."
I've been thinking the same thing. I think a lot of it stems from inadequate testing, pure and simple. I went back over posts in Beta Testing to get some numbers. I checked each new Dev Test or Pre-Release post within a week-long period starting two weeks ago. This was roughly 1 week after the formation of the Beta Testing Team. Fifteen apps were posted for testing, not counting launcher-only releases. The average number of testers reporting in on those apps was 2. Some were as high as 5, some had zero. This counts the posts that consisted of only "it ran fine". This is from a Beta team of 15. Even more sad is the fact that 3 people made up more than 75% of these test result posts, and one of those isn't even on the Beta team.
It seems that most of the people on the team aren't taking it seriously. Volunteering to do this should be considered a commitment. It's the reason I didn't volunteer.. I couldn't make the commitment to be able to test the majority of apps that need it the majority of the time due to my job, a toddler at home, etc.
Beyond actual tester participation I think some other things might help as well. The mailing list or some other way of keeping testers apprised of what needs doing seems to be a priority since apparently the Beta forum isn't enough. Also a checklist of things that need testing.. following the format, registry/files left behind, AV checking, 9x/2K/XP/Vista checks etc. Perhaps this list could be added to the standard test post and filled in by the author and/or mods as testers sign off on each item. That in itself would give John and/or release techs a much better idea of how ready an app is to move on to the next step. It also gives testers more of a direction beyond download, run, click a few buttons and sign off.
Anyway, those are my thoughts, take them or leave them.
formerly rayven01
I was mentioning that point about the Beta Testing Team today. I've tried to test all I could, there's a few apps I didn't yet like Blender, X-Moto, Secret Maryo Chronicles, etc as I am only allowed 200MB's of bandwidth a day for my whole household, so that slows me down. But really, it's kinda sad no one else is testing.
I would, but for the fact that by the time I get home to read the posts, the test is finished and the official one is out!
"If you're not part of the solution, you're part of the precipitate."
thats why I quit the testing team.
But I think the list that was posted earlier is a good starting point.
I also agree with Tim. Once upon a time we had pre-releases for every App released. I can understand that if nothing changes (like the latest Filezilla release from 3.0.8 to 3.0.8.1) these aren't necessary. But other releases should get one week or maybe less in the Forums so we know they are good. I´d go with "Quality over Speed".
The one problem mentioned with the Readme is one that happened to me with PDFTK. I thought I had fixed it but John packed it and now I'm not sure. But it doesn't justify a revision in my eyes.
just my 2 cents.
"What about Love?" - "Overrated. Biochemically no different than eating large quantities of chocolate." - Al Pacino in The Devils Advocate
Perhaps the 24 hour turnaroud should mean get the Pre-Release out in 24 hours.
cowsay Moo
cowthink 'Dude, why are you staring at me.'
and then give it some 3 days till the main release gets out.
"What about Love?" - "Overrated. Biochemically no different than eating large quantities of chocolate." - Al Pacino in The Devils Advocate
I usually test in the development section. Which reminds me, theres an app in the development center called Portable App Creator. It's really good. I helps take non portable apps, and make them portable. Unfortunatly, since the apps it creates arent in perfect Portable Apps Format, he thinks that it wouldn't fit in with the site. I'm trying to tell him that since the app itself is in PA format, he should move it to beta testing. Could you please check it out John? I'll start beta testing more
Simplifying daily life through technology
Portable App Creator DOES NOT WORK, by the author's own statement. If you try to move one of the apps it creates into a new location, it's highly likely that it will not run at all, or just not run portably (leave stuff behind).
The developer formerly known as ZGitRDun8705
It works fine for me. Plus, when you install it into the required area, it works. If you need to move it, open PAC again. Plus, here is a quote from him.
PAC itself is a real PorableApp, but - as it is developed to portablize closed-source apps - it doesn't generate real "PortableApp"s. So it is questionable if it should or shouldn't be in the development apps list. For now, I think it is fine where it is now. In the "Portable App Creator" thread in the "Development Forum" as I stated before. But if mods and/or John think it should be in the "Development Apps" list, I'll folow their guidlines.
I want John to check it out.
Simplifying daily life through technology
is that, in its current form, it is a developer tool, rather than anything aimed at typical end users.
As such, it makes sense for it to be in the dev area, and it can be useful to some, in some circumstances, but, if it were ever to be released as a mainstream application, then it would generate a truly stupendous number of support requests from people who thought that it was a magic solution and wanted it fixed, right now, so it worked for their favourite pet app.
You just said that if the app needs to be moved, PAC must be run again. Sorry if I'm misunderstanding, but isn't this saying that if you need to use the app that you just "portablized" portably, you have to run PAC to re-portablize it.
It sounds to me like PAC doesn't work, because if the drive letter changes, or everything is moved to a different folder, PAC has to be run again, and that is NOT portable at all.
The developer formerly known as ZGitRDun8705
PAC works with different drive letters.
Simplifying daily life through technology