You are here

Updater hangs

25 posts / 0 new
Last post
Ackael
Offline
Last seen: 3 years 11 months ago
Joined: 2013-08-04 13:03
Updater hangs

Hi, since a few weeks the automatic updater doesn't work any more:
After I click to update the apps that are listed it says "Connecting..." but nothing more happens.

So now I can only update from the website.
I already re-installed the PA platform but no success.
On my other computer I have no problems.

Furthermore I cannot open the list of other available apps. The error message says that this may be due to the updater running. In fact is the same program, which hangs too, so I have to kill it with the task manager.

badon
Offline
Last seen: 6 years 9 months ago
Joined: 2013-11-15 22:12
I have this same problem.

I have this same problem. Update doesn't work, and it won't retrieve the list of other available apps. The process of getting that list always takes a long time, though. I suggest some sort of caching to avoid the need to grab a complete list every time I want to add something new. I suspect a timeout is occurring, and PortableAppsUpdater.exe is unaware of it when the timeout happens.

John T. Haller
John T. Haller's picture
Online
Last seen: 16 min 22 sec ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
Same Server

The list of apps is retrieved from the same server that's serving this website and the list is smaller than the homepage. So, it's not something up on the server end. The updater itself has no problems retrieving it on my multiple test machines. So, is there a firewall or anything else running on your machine that could be interfering? Also, is it running on a slower flash drive with lots of apps installed?

Sometimes, the impossible can become possible, if you're awesome!

badon
Offline
Last seen: 6 years 9 months ago
Joined: 2013-11-15 22:12
I just reviewed this

I just reviewed this discussion about possible firewall problems:

https://portableapps.com/node/38584

I disabled my firewall, but the problem remains. I did notice that it tries to connect to 72.32.40.232 via port 80, which is the portableapps.com server, like you said. Unfortunately, my firewall shows that no data being transferred. After a few minutes, the connection disappears from the list of connections, presumably because it has timed out...but PortableAppsUpdater.exe continues running happily.

If there were at least some kind of close button or a status indicator on the updater GUI thing, I would be able to figure out if it's still responsive. This isn't the first time this has happened, but previously whatever wasn't working resolved itself spontaneously (from my user point of view).

Any other ideas for debugging this?

John T. Haller
John T. Haller's picture
Online
Last seen: 16 min 22 sec ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
Unfortunately

Unfortunately, no. The download is done via the NSIS Inetc component (used in tons of live regular installers) which does the download via API calls to the Windows internet components. It's set for a 30 second timeout to allow time for folks with password protected proxies to type the password in. Our code gives up all control to inetc which handles everything via Windows APIs.

If you're using the beta, be sure it's set in Automatic mode (there are bugs in proxy and direct at the moment). Other than that, there are no additional debug steps I can take as inetc provides no additional debugging capabilities. If it's not working on your machine, something is interfering with it, but I'm not sure I can determine what.

Sometimes, the impossible can become possible, if you're awesome!

badon
Offline
Last seen: 6 years 9 months ago
Joined: 2013-11-15 22:12
I'm pretty sure nothing on my

I'm pretty sure nothing on my machine is keeping PortableAppsUpdater.exe running for hours and days after it has obviously failed to do whatever it is trying to do, so clearly there is something wrong with it, if it's supposed to be timing out after 30 seconds. My firewall software shows it connecting to the server, but then it seems to do nothing. I'm not sure of this, but it looks like it doesn't transfer any data, like it would if it were downloading a list of apps. I'm not sure the connection was successful. Incoming data is always 0 bytes, and outgoing data (presumably the connection request) is always around 66 bytes.

I just tried it on 3 other PC's. The first one had the same hanging problem where it does nothing forever. The second one failed too, but it gave the error message:

"Unable to connect to PortableApps.com to retrieve portable apps. Please try again later. [ SendRequestError ]"

The third one succeeded after a short delay, and it is now displaying the list of apps sorted by category.

I should also add that I have always had problems with the PortableApps updater. It has had this problem before, and then it would spontaneously resolve itself for a while. Then, it would try to download an update or a new app, and then it would fail with some vague error message about it not working after a second attempt (if I remember correctly). This is the first time I've had one of them fail ALWAYS for such a long period of time - about a week so far.

It looks to me that the updater is unreliable in general, and another tool should replace it, or the existing tool needs features to aid in debugging so the failure cases can be identified, studied, and resolved. I suspect you might be using a service like CloudFlare that is blocking access seemingly at random. Most of the times I've encountered something strange that fails intermittently like this, a sort of reverse firewall was to blame, along with dynamic IP or otherwise multiple IP addresses, with some of the IP addresses being rejected. All 4 of the PC's I tested are connecting from different geographic locations and different IP addresses, and their IP addresses are probably dynamic. They're very often different, in any case.

Oh, hold on there, it looks like PC #1 finally displayed a list of apps after about 5 to 10 minutes of delay. I have seen the updater being very slow before, but I think that's the slowest I've ever noticed it. It's quite a lot slower than it should be if the 30 second timeout is working properly.

Are any of these clues helpful?

badon
Offline
Last seen: 6 years 9 months ago
Joined: 2013-11-15 22:12
I just noticed the command

I just noticed the command line parameters passed to PortableAppsUpdater.exe:

/MODE=ADD /OPENSOURCEONLY=false /KEYBOARDFRIENDLY=false /ADVANCED=false /SHOWINSTALLEDAPPS=false /BETA=false /ORDER=category

badon
Offline
Last seen: 6 years 9 months ago
Joined: 2013-11-15 22:12
These parameters fail

These parameters fail too:

/STARTUP=true /MODE=UPDATE /KEYBOARDFRIENDLY=false /BETA=false

John T. Haller
John T. Haller's picture
Online
Last seen: 16 min 22 sec ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
Unrelated

This just tells the updater what to do after it gets the app list.

Sometimes, the impossible can become possible, if you're awesome!

John T. Haller
John T. Haller's picture
Online
Last seen: 16 min 22 sec ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
Fresh Install

Have you tried a fresh install, just temporarily, to your Desktop folder or any other location just to confirm whether a completely clean install with absolutely no changes to Options and no apps installed works on your setup? Please do this before continuing.

No additional checks are done on the server. It's a dedicated server serving the updater list via Apache which is served as a single plaintext file (an INI file). And it's served over a standard http connection using inetc. You can even view the file in your browser.

One other thing, be sure that IE is functioning and correctly connect to the internet. The Windows internet components are essentially IE bits. If they are broken or misconfigured, nothing that relies on them will function either.

Finally, please supply additional details to your specific installation including OS version, full install path, platform version, etc.

Sometimes, the impossible can become possible, if you're awesome!

badon
Offline
Last seen: 6 years 9 months ago
Joined: 2013-11-15 22:12
Yes, I tried fresh installs.

Yes, I tried fresh installs. It doesn't matter where it is installed or what PC it is installed on, it has the same problem as all the other instances of the PortableApps.com Platform that I have. Internet works fine on all of the PC's I tested, and this URL loads in mere seconds in my web browsers:

https://portableapps.com/updater/update.ini

Can you be more specific about what "bits" you're referring to? IE is a malicious pest, and I haven't used it since the last time I got hacked through it about 15 years ago.

As far as I can tell, there are no network misconfigurations on any of the PC's I used to replicate these problems. I ran the MS internet troubleshooter, and it claims it couldn't find any problems. None of the PC's are configured to use IE as the default browser, and all them have out-of-date versions, or IE may be proactively disabled in some cases. One one of the PC's I tested that has never been able to update, IE is functional.

I double checked firewall logs, and still nothing is being blocked. Can you tell me if PortableAppsUpdater.exe uses the server IP address, or does it use the domain name? Even more importantly, does it ignore network configurations to forcibly do its own DNS lookups? If so, then maybe whatever DNS server it is querying is unreliable from the network location of the 4 PC's I tested.

Also, maybe PortableAppsUpdater.exe is sensitive to high latency? I doubt that's the problem, but if a packet got dropped here and there, but PortableAppsUpdater.exe could not detect something was wrong, that might explain what's going on. Actually, that's not so far fetched now that I think about it. It already can't tell that it has failed, so there's no reason to think it would be aware of any kind of failure, regardless of the cause. That could be the reason it is so unreliable. If packets are being throttled by the ISP to favor VoIP or something like that, that might be enough to make PortableAppsUpdater.exe fail silently.

In that case, once again, the real problem is that PortableAppsUpdater.exe is unreliable. It has never been as quick as it should be, and considering the fact that it tries to download version information for hundreds of irrelevant software packages, there's no reason to expect snappy efficiency anywhere else in its operation.

It usually doesn't take this long for me to isolate a simple bug, if that's all we're dealing with. That's why I believe PortableAppsUpdater.exe is probably suffering from an overarching unreliable design that goes beyond any single thing that can be blamed as the cause. I do bug hunting for the WikiMedia Foundation (Mediawiki, Wikipedia, etc) and I've never seen anything quite like this before. That's partly why I never reported something sooner. I just guessed that maybe server load or internet "weather" was to blame, and it would work again eventually...which is how it normally goes until now, on 2 of the 4 PC's I tested.

All 4 PC's are Windows 7 32 bit, and all 4 have failed at least once when there was no good reason to expect a failure. All 4 have been tested with fresh installations, and it doesn't seem to matter what directory it is run from (correct me if I'm wrong about that) They are all being run from their own fresh copies, from their hard drives. There are no USB devices involved. I'm using version 11.2:

* CRC32 4594ed19
* MD5 1178fe18327311aa55e9734bca42fdc5
* SHA1 3d2b99b0f834de0365beb348a7f63fc44122d583
* SHA256 2c6bc99b92ebceb6984d8e97e73b2b8af8a92295ca47f0032e59f595e8277ec8
* SHA512 2b9effb069b3a27912c3c3b4702d382e2d8a6a900522579e5339606c83a79106ecfee86051035a9ae0886114bea9292db89f83727bf5ed6ab409f34f821aec9

The MD5 matches the published hash. I see there's an 11.2.1 version in the source, but I'm not sure what to look for there. Is it for the Lazarus IDE?

John T. Haller
John T. Haller's picture
Online
Last seen: 16 min 22 sec ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
Disabled IE

IE can't be 'disabled' through standard methods on Windows, though its icon can be hidden. If you use a third party utility to disable/uninstall IE, it will break all the Windows internet components as well and everything that relies on them will fail to work including our updater. IE isn't just IE, it's also the components Windows uses to provide internet services to other programs. Some just use the download components (as NSIS' inetc does). Others use embedded browsing components (like the RSSOwl RSS reader). IE should be installed and fully patched on a given PC to ensure that that PC can run securely even if you don't use IE. And the settings in Internet Options within control panel must be correct as well. Basically, you must be able to download that URL within IE or the updater won't work on that PC.

If there were an overarching design issue, we'd be hearing about it. Loudly. Like I said, millions of people are using this to update their apps regularly. It's a simple text file on a dedicated server running Apache that's always available. The only time our server has had any downtime in the past few years is when our daily backup window occurs at 1am NYC time. During that time, Drupal (our website) will be offline for a few minutes, but serving static files remains active.

Sometimes, the impossible can become possible, if you're awesome!

badon
Offline
Last seen: 6 years 9 months ago
Joined: 2013-11-15 22:12
That isn't very specific, so

That isn't very specific, so I will just guess that you're referring to the Internet Options dialog. If that were misconfigured, nothing would work. I checked the other PC's, and none of them have IE disabled. They have all been updated recently through Windows Update too (but still no one uses IE).

The only software that has any trouble at all is the PortableApps Updater, and it fairly consistently has trouble on all 4 PC's. When it works, it is still much slower than it should be, given the 30 second timeout you mentioned. I asked you how the updater does DNS lookups, and I'm still waiting for your answer. If you don't know, just say so, and I'll work out something else. I have the source, so maybe you could point me in the right direction for figuring out whether the updater does DNS lookups, or whether it just uses the update server's IP. If it uses the IP, then we can rule out DNS issues as a class.

John T. Haller
John T. Haller's picture
Online
Last seen: 16 min 22 sec ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
It Doesn't

It lets Windows handle all of that via API calls. In short, it lets the Windows internet components handle everything including DNS lookups. You will find no networking code at all in our PortableAppsUpdater.nsi code. You're free to examine the C++ code within inetc downloadable form the component's plugin page.

Note that you will find no DNS lookup code within that either as it is handled entirely via the Windows wininet.dll APIs. You will not find it looping or counting for a timeout either as it's also handled entirely by the Windows components and set via the InternetSetOption function. It open the HTTP conneection with HttpOpenRequest and handles the initial communication and download the file via HttpAddRequestHeaders, InternetReadFile, HttpQueryInfo, HttpOpenRequest, etc.

Both the DNS lookups and the connection timeout are handled entirely within the Windows wininet.dll. inetc's code simply makes the API call to Windows and then waits for it to finish. And the PortableApps.com Updater simply makes the call to inetc and then waits for it to finish (primarily because there is no other option as NSIS is a relatively limited language). I've never seen these calls fails except when either (1) Internet Explorer is disabled/damaged or (2) a software firewall/antivirus is interfering with their proper operation.

All of the above plus you saying something about IE being "disabled" is what lead me to believe it may be a specific PC issue with IE being uninstalled by an unsupported 3rd party utility or Windows being installed by one of those 'light' 3rd party installation utilities which breaks the Windows internet components.

So you are saying that IE is properly installed and if you copy and past the link to the updater database above into IE it will properly download and display it, correct?

So, let's try this again. Let's take a specific PC where you're having an issue. Do a completely fresh install of the PA.c Platform to your desktop. Don't change any options. Make sure nothing else is running. Scan it for viruses. Then temporarily turn off your antivirus and firewall. Now run the PA.c Platform. Does it download and show a list of apps? Does the dialog box saying that it's checking for apps online display?

Sometimes, the impossible can become possible, if you're awesome!

badon
Offline
Last seen: 6 years 9 months ago
Joined: 2013-11-15 22:12
For HttpOpenRequest, do you

For HttpOpenRequest, do you give it an IP address, or a domain name? The answer to that question could allow us to rule out DNS issues without needing to look at anything else. Actually, nevermind. If it's using the domain name, it appears DNS is working because the firewalls show it connecting to 72.32.40.232, which is the IP for portableapps.com, so it looks unlikely that there is a DNS issue.

I tried doing another round of fresh install for the PortableApps Platform, and to my astonishment, 3 out of the 4 computers tested were suddenly working fine. I tried the old in-use installs, and they are working too.

One PC is still not working, so I did some more investigation. I found that the one troublesome PC is configured for automatic updates to IE, while the other 3 are not. So, following that clue, I looked at the updates that IE has been getting, and I found this one, which happens to be almost exactly a week old, since these problems began!

http://support.microsoft.com/kb/2888505

So, looking more closely I saw that the update failed silently, and it apparently is impossible to force it to retry. Right now, that's my bet for what's wrong with the remaining 1 PC that isn't able to update, and the timing of all this matches up nicely with everyone else's complaints. I'm not sure why the update failed, and why it can't roll back or retry, but I will investigate if there's a way to manually fix this.

It is interesting that the only PC that has not recovered is the one that had automatic updates configured for IE. I assume the other PC's failed for different reasons - possibly more failed automatic updates to Windows 7. In any case, they eventually resolved themselves as of a few hours ago.

I don't know why anyone would build something that is critically dependent on shoddy Microsoft crap that's totally unrelated, especially when there's no source code to investigate, and one tiny problem can take down your whole project. In light of this, I have to recommend downloading the text file without depending on IE. Really, it's just a text file. It doesn't need to crumble to pieces every time MS screws something up with IE.

Taking it a step further, I suggest breaking up the text file so it only needs to download update information about the installed apps, or the apps that are available that aren't installed yet. That may be more complicated than necessary, but really, the whole thing depends on IE, so no matter how you look at it, it's still a lot simpler than the IE house of cards.

Maybe by the time I find a way to manually fix the failed update, you can hack out IE so it's no longer a PortableApps dependency.

John T. Haller
John T. Haller's picture
Online
Last seen: 16 min 22 sec ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
Part of Windows

IE is an integral part of modern Windows. Even if you don't use IE and delete every shortcut to it, it is expected to be there and working properly. Without it, lots of apps, installers, etc will fail to work properly. Even Google Chrome, Chromium, Opera and Iron rely on specific parts of IE being present and working properly (proxy configuration, crypto, etc). Every modern OS makes networking components available as a default built-in from the TCP/IP stack to DNS lookups to handling connections. Having them working properly is having part of the OS working properly.

I'm unsure why your specific PCs had issues with a specific Windows update hotfix. None of my testbeds with that patch applied had any issues (Windows XP through Windows 8.1).

We won't be removing the functionality anytime soon as using the built-in Windows components works better with proxies than anything else (which is why Chrome/Chromium/Opera/Iron handle it that way) and it's the only way to show download progress in NSIS at present without writing our own networking component from scratch (beyond my personal abilities at present). Until the updater is rewritten in something else (on the board at some point), it will continue to use the Windows networking components. Even when we switch, DNS lookups will still be handled by the OS as they are in nearly every piece of Windows software.

Sometimes, the impossible can become possible, if you're awesome!

badon
Offline
Last seen: 6 years 9 months ago
Joined: 2013-11-15 22:12
That's not IE you're talking

That's not IE you're talking about. That's the Internet Options dialog and related configuration settings. All of that worked fine the entire time. Only IE had a problem. ONLY IE, as in, nothing else but IE. That's why it was so strange that only the PortableApps Updater was non-functional. It didn't matter that MS somehow botched automatic updates to install an unsupported version of IE, because nobody was using IE.

I have the impression that you must not be the person doing the technical work for the PortableApps Platform. I don't think you would have the time for both that and your personal attention to support matters, unless you skipped sleeping during the last 48 hours, like I did.

Either way, it would be trivial to cobble together an AutoHotKey script that could grab the version information file without needing whatever unreliable techniques the PortableApps Platform is currently using. In a couple of days or so, it could be ready for beta testing, and then you would have a whole new set of bugs to fix ... that you can actually fix! You could probably do the same thing in Python, Ruby, Lua, and a thousand other ways, and none of them have any of the same major flaws and limitations.

If you're paying someone to maintain your current way of doing things, I'd recommend reconsidering that arrangement. You claim you have no control over what happens, and we've already determined that the updater itself is not aware of when it fails. That's a lot of really bad and unnecessary compromises. To be paying somebody for all that is even worse.

Another little nitpick: If a big download is in progress, but something goes wrong and the updater is interrupted, it discards the entire download. Worse still, it doesn't even remember what it was downloading so it can try to retrieve it again later, after the current download queue is completed.

As far as unreliable mirrors go, there's no reason to accept that as the only end result you can expect. You can easily switch to use a different mirror if one of them won't provide a complete download. If you wanted to get fancy, you could use the bittorrent protocol to gain even more downloading reliability. Once again, none of those options depend on IE.

Why in the world you would deliberately make your entire PortableApps Platform dependent on a non-portable installation of a widely-disliked and totally irrelevant web browser makes no sense. It's absolutely not necessary, and I don't care what your team is telling you - IE is not required. If somehow your programmers figured out a way to make it required, then I must say, that is an impressive feat by itself, but it's not one to be proud of.

John T. Haller
John T. Haller's picture
Online
Last seen: 16 min 22 sec ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
Windows Networking Components

And we make use of the Windows networking components, wininet.dll, as already explained to you. We don't use or instantiate IE at all. However, those components are, essentially, part of IE and are affected by IE updates. So, you need to ensure IE is properly installed and patched. Microsoft's "disabled" utility merely removes the icons, associations, etc for it. IE itself is still installed and can be run by other programs as stated on Microsoft's own page. I'm unsure why you're remaining willfully ignorant of these facts.

Perhaps you'd have gotten more sleep if you'd bothered to troubleshoot the issue using the instructions I gave you 37 hours ago which would have immediately clued you in to what was broken on your machine instead of posting inane diatribes.

Sometimes, the impossible can become possible, if you're awesome!

badon
Offline
Last seen: 6 years 9 months ago
Joined: 2013-11-15 22:12
How to turn Internet Explorer

How to turn Internet Explorer off - Microsoft Windows

When Internet Explorer is turned off using this feature there is minimal impact on third-party programs (and other parts of Windows itself) that call upon Internet Explorer app services, because the Internet Explorer files that provide functionality to third parties and other parts of Windows are left intact.

John T. Haller
John T. Haller's picture
Online
Last seen: 16 min 22 sec ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
Not Disabled

That doesn't disable IE. That just turns of the icon, associations, etc. As I'd said earlier, I was referring to 3rd party utilities.

Sometimes, the impossible can become possible, if you're awesome!

badon
Offline
Last seen: 6 years 9 months ago
Joined: 2013-11-15 22:12
I wasn't able to directly fix

I wasn't able to directly fix the failed update. What seems to have happened is an update failed for IE 10, and then immediately after IE 11 was installed. All of MS's support information indicates that IE 11 is only supported for Windows 8, so there was no information about what to do next. In the end, I was able to remove it via this procedure:

1. Open Control Panel.
2. Click "Programs and Features".
3. Click "View installed updates" in the upper left corner.
4. Uninstall the most recent update for IE 10.

After I did that, Windows 7 prompted to restart, but it never finished shutting down. After an hour or so of waiting with no disk activity, I did a hard reset. Then, while Windows 7 was rebooting, it displayed the familiar message along the lines of "Windows is updating stuff, don't turn off your computer". It rebooted one more time after that, and now IE 11 is gone, but IE 10 is back again, and the failed update for IE 10 is still in the list of updates, and it won't let me uninstall it. Despite that lingering weirdness, the PortableApps Updater now works.

Life would be a lot easier if the PortableApps Updater weren't dependent on MS's pile of rotting garbage just to download a text file. This is the kind of thing that makes me want to avoid anything involving Microsoft's Internet Explorer. At the very least, it would be nice to have a way to force-feed a local copy of the update file to the PortableApps Updater so it won't have to make the world spin in the other direction just to get the file via Microsoft's tools.

Ken Herbert
Ken Herbert's picture
Offline
Last seen: 6 hours 58 min ago
DeveloperModerator
Joined: 2010-05-25 18:19
Problem solved, now just chill out, please

Please, stop misconstruing "Internet Explorer" (the component parts that are an integral part of Windows) as "Internet Explorer" (the browser). I personally believe that IE the browser is a terrible piece of bloatware, but without the background components of IE there would be no modern Windows OSes.

Telling John to stop using these components for the Updater or any other part of the PortableApps software is as pointless as telling any other Windows developer to stop using any functionality provided by the Windows API - it is the basis of programming on Windows, it is generally quite stable and trustworthy, and all such functionality is potentially just as susceptible to unforeseen issues such as you have experienced with a failed Windows update.

You continually bashing John's utilization of the Windows/IE internet components as "bad software design" is as much a bash at any other developer of any other Windows software, myself included, and it stopped being productive a while ago now.

Honath
Offline
Last seen: 10 years 10 months ago
Joined: 2014-01-22 02:02
Same problem here

I cannot install McAfee or update Torrent today. Download stopps after a second with an error: File not found (404)

It says, so it might be a problem of sourgeforge.com

Simeon
Simeon's picture
Offline
Last seen: 10 years 1 month ago
DeveloperTranslator
Joined: 2006-09-25 15:15
same here with mcafee

There seems to be something wrong with both of them as they are both online installers and fail on the file CRC check. John has to update those and he is only part time available right now due to past-surgery recovery.

"What about Love?" - "Overrated. Biochemically no different than eating large quantities of chocolate." - Al Pacino in The Devils Advocate

jwinter
Offline
Last seen: 5 months 4 weeks ago
Joined: 2008-06-02 00:40
uTorrentPortable File Not Found

For several days and from several networks the updater has failed to update uTorrentPortable. It pulls in something successfully (the first lower progress bar completes), but then comes up with the error:

"Download Error: File Not Found (404) on file http://downloads.sourceforge.net/projects/portableapps/uTorrentPortable_..."

Trying to download this file manually gives the same error. So it looks like the file is not on the server.

JohnW

Log in or register to post comments