You are here

Rewriting the PortableApps.com Platform with Web Technologies

12 posts / 0 new
Last post
gluxon
gluxon's picture
Offline
Last seen: 4 years 5 months ago
Developer
Joined: 2008-06-21 19:26
Rewriting the PortableApps.com Platform with Web Technologies

Something that I've begun to do recently is writing my applications in web technologies with HTML, CSS, and JavaScript. The reason for this is that it has many benefits, cross-platform being a big one. However, it wasn't the biggest one that I saw. Maintainability and ease of development were the biggest. The code-base for my apps would stay very simple and easy to understand to any web developer. Changes could be made very easily and quickly given HTML's dynamic capabilities.

Seeing the current state of the menu's development, I think it may be ideal to at least consider doing this. I've outlined some more reasons below.

Pros:

  1. True Cross-Platform: Very early into the platform's development, some developers here considered writing it in C++ for the advantage of being able to deploy the platform on OS X and Linux in addition to Windows. Although we have not created a launcher concept for Linux and OS X, this could be something we look into for the future. WINE is great, but a platform that runs without its inconsistencies would be better.
  2. Moving away from Delphi: The fact that the current platform is written in Delphi scares away new developers. Delphi is expensive ($999 or $199 for starter) and not worth learning due to its low usage in the industry. On the other hand, HTML, CSS, and JavaScript is used almost everywhere.
  3. Opening to the community: Web development is known to almost all programmers. Having the platform written in an ubiquitous language will have more contributors ready to make patches.
  4. Screen-reading: We've struggled with screen-reading support in the past. Screen-reading would work almost the same as a web page in a browser, so support for it is built in. Machine reading of HTML is a very established technology.
  5. Design is limitless: GUI widgets are no longer limited to the Windows toolkit. There will be support for a custom designed scrollbar and one that resizes appropriately.
  6. Theming is limitless: Practically all elements of the menu will be modifiable with CSS. Themes will have control over how many items are displayed on one page, sizes of elements (including the menu itself), and even placement of menu items. Overall, much more control than before.

Cons:

  1. Depreciation of current code: The current code has been in work for at least 5 years, so there is a lot of sentimental value to it.
  2. Size of distributables: AppJS has a 20 MB overhead, so the install size of the platform will increase from 10 MB to around 25-30 MB. This isn't a large concern for modern usb drives which are growing in storage monstrously, but a platform of this install size may scare some.

Overall, I think it is worth it. I feel like I could have the current platform written in this code within 2-4 weeks (considering my other commitments). Thoughts and opinions?

Edit: Let me clarify. AppJS is not the only framework I had in mind. I just mentioned it in the cons since it was the only one I could quickly look at the overhead for.

John T. Haller
John T. Haller's picture
Offline
Last seen: 2 hours 53 min ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
AppJS is Pre-Alpha

While I like the the route you're going down in thinking, there are some big assumptions in the above. For starters, AppJS is pre-pre-alpha stage, aka version 0.0.20, so I don't know that I'd want to base production-level software on it. As for the other points.

The pros are a bit idealistic and some turn out to be cons on further examination:

  • Cross platform capabilities would be nice, but all our portablization tech is Windows-only, so it isn't the platform holding us back. We need Wine to run the apps. And the platform works very well in Wine.
  • The full version of Delphi Professional is $999 for a full version or $49 if upgrading from last year's version. $4k is the super-ultra special corporate edition with enterprise features. The Delphi XE 4 starter edition is $199 and seems to include everything you'd need to develop the platform. A better option would be to compile under Lazarus.
  • It would open the community to AppJS developers, not web developers, since we need to learn the AppJS stuff to work with it and work with INI file read/write, etc (this doesn't exist in HTML/CSS/JS so would have to be proprietary in AppJS and it may not exist at all for needing to make Windows API calls to pull name, company and version from an EXE file). As an analogy, lots of people know Pascal (standard language to learn at US and many international universities) so it's a good base for Delphi, but doesn't mean they can hop in and do it without learning more.
  • Screen reading would be worse under AppJS than it is under the current Delphi setup. Screen readers have to be specifically keyed to work with browsers to be able to work properly. IE and Firefox have the best support. Chrome is a distant 3rd. Opera is nearly non-existent. I'd wager the current ones won't work at all with AppJS. The current Delphi-based menu uses native widgets which is why it works out of the box with screen readers. The only issue is the native listbox used in the updater/app store and that's why we have a native mode switch for it in options.
  • Design and theming takes a big back seat to all the above issues. We wouldn't sacrifice all the above to be able to improve the design a bit.

On the cons, neither of those are really a big con. Size is less of an issue. There is no sentimentality tied to the existing code. We just want it to work, have it work well, have it be fully open source under the GPL, and have all code joint copyright with us so we can link it to proprietary DLLs for drive-specific hardware features and other situations should the need arise.

Sometimes, the impossible can become possible, if you're awesome!

Gord Caswell
Gord Caswell's picture
Offline
Last seen: 2 weeks 4 days ago
DeveloperModerator
Joined: 2008-07-24 18:46
Lazarus

A better option would be to compile under Lazarus.

I've started to look into this, but haven't had time yet to do more than run Lazarus' auto-converters. Unfortunately, not everything was converted initially, so I'm having to step through more or less line by line.

I'll put up a mercurial repo if anyone's interested in helping.

gluxon
gluxon's picture
Offline
Last seen: 4 years 5 months ago
Developer
Joined: 2008-06-21 19:26
AppJS isn't the only thing we have to use.

Thank you. I like the way I think too. Smile

I wrote the original post generalizing around building an interface in web technologies, and neglected the actual framework to be used since I thought that was more trivial and could be decided later. It looks like that was a mistake since the post now suggests that AppJS was what I had in mind. I only mentioned it because it has the smallest overhead I've seen in a framework like this. node-webkit is what I've been using for my projects, and it is much more production ready than AppJS. It is being used in a lot of software projects, so I'd argue that it is very stable.

Considering that, I think a lot of the points you made are incorrect and I am probably less idealistic now. Let me clarify somethings from my original post.

  1. The platform does work in WINE, but the icons are drawn wrong (black background) and the native theme doesn't match the operating system. We could have better integration with Linux and OS X in general, or more than what we have now. You're right that this doesn't matter until we create a launcher concept for these OS's.
  2. I don't think someone has managed to compile the platform under Lazarus yet. Delphi is still payware, so I don't feel like the platform is open in the way that we can all change the code. Also, I recall the fact that the platform being written in Delphi as a huge reason for a potential C++ rewrite. I think it was actually you who mentioned that on IRC years ago. I don't remember the exact reasons you said, but I can dig through logs later. They were probably similar to the reasons of cross-platform, openness, and more familiarity with new contributors in terms of coding language. I did make the careless mistake of not checking the exact version of Delphi we used and pulled the price from the wrong version. I'll fix that now.
  3. There's actually a lot things incorrect about this next one. AppJS is open source under the very nonrestrictive MIT license, so I don't know where you got the idea of a proprietary INI reader/writer from. The code would also be in Node.js, opening up usage of the entire asset of libraries and API's from npm (Node's package manager). Node.js also allows custom C/C++ addons. Anything we can do in Node.js, C/C++, jQuery, standard JS, we can do in the potential platform rewrite. Here's an INI utility. AppJS is quick to use and learn. It's basically a few calls and manipulations to a window object. And for node-webkit you just have to point it to a default HTML file to load. There's really learning curve here if you know HTML, CSS, and JavaScript. We'll probably write a custom C/C++ module for Node.js to extract exe icons though. One may exist, but I didn't find one with a quick search.
  4. node-webkit and AppJS are both webkit based. If we want to go crazy, we also have XULRunner as an option, which will base our code off Firefox's engine rather than Chrome's. Screen-reading would be like I said in the original post, not some weird framework setup.
  5. I apologize for my tone if it is offensive (so please don't take it the wrong way), but I don't see the issues you listed as real issues now.

I really feel like this would improve the platform and allow us to add new features at a pace quicker than before. John, you're practically the only platform developer, and I think that should change so you can focus on more important matters.

The issue of overhead is the only one I really see. node-webkit is ~40 MB in application overhead. I need to experiment with reducing this size.

John T. Haller
John T. Haller's picture
Offline
Last seen: 2 hours 53 min ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
Big Issues

You may not see them, but these issues are large and remain.

  • The minor graphical issue around the icons in Wine is fixable, I just haven't gotten around to it. The theme is simply a matter of someone building one specifically for Wine (aka, just graphics design, no coding) as we can already detect Wine and make adjustments for it.
  • Others discussed a C++ rewrite. I don't personally know C++.
  • I wasn't talking about the license of AppJS. I was talking about new bits of the language outside of HTML/CSS/JS. HTML/CSS/JS have no abilities to read/write local file systems, make Windows API calls to do things like listen for USB plug/unplug and monitor file changes, etc, so all that would have to be unique/proprietary... aka NOT in the normal HTML/CSS/JS spec. So, users would have to learn that. It's got nothing to do with a license. I doubt any of these web frameworks actually have the abilities built-in to do everything the platform does. We'd have to custom-write C++ plugins for every platform.
  • They're both webkit-based. Webkit browsers + screen readers on Windows = unhappiness, at least currently. This would be a huge step backwards.
  • You misunderstood my biggest point (#3), I hope it makes more sense now.
  • As for the links, the listed example programs, most of those GUIs are pretty bad. They look like basic web pages. I've yet to see something that looks and feels like a real app based on these web engines (Brief Msg comes closest). They all look and feel like web apps shoehorned into their own window.

I'd love to get more help working on the platform. But, I don't think these alpha-quality web engines are the way to do it (node-webkit is alpha, AppJS is pre-alpha). Lazarus is probably a more likely candidate. C++ code be, too, with enough developers. We have one other interested party who has already contributed Delphi code as well which will be in the next beta in a few days (the PStart-style cascading menu from the taskbar some folks asked for).

Sometimes, the impossible can become possible, if you're awesome!

gluxon
gluxon's picture
Offline
Last seen: 4 years 5 months ago
Developer
Joined: 2008-06-21 19:26
I'm confused as to what the

I'm confused as to what the biggest issue is. I would have thought it was about screen reading, but that wasn't your third bullet point.

I feel like we're not on the same page with the HTML/CSS/JS limitations you're talking about. Node.js and its module system provides limitless capabilities that anyone with JavaScript knowledge will be able to use. Here's Node.js's native filesystem module. Here's a native event binder to detect file changes. Here's an npm module for usb plug/unplug. Anything in C/C++ can be turned into a Node.js addon by simply wrapping them.

What you're saying kind of contradicts what I've heard until now. I've always thought that web pages had the best support for screen reading. I'll look into this further. I also thought this would be an improvement since I last heard that the platform still had some issues in this area, but it looks like it was more minor.

I don't know what bad GUI of example applications has to do with this. The platform doesn't have to look like any of those apps. Here's one of my apps for something that looks a bit better (at least in my opinion).

Not that I don't believe you, but where did you find that node-webkit is alpha and that AppJS is pre-alpha? I don't see that status on either of their sites and GitHub pages.

John T. Haller
John T. Haller's picture
Offline
Last seen: 2 hours 53 min ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
Version Number

node-webkit is versioned prior to 1.0. This is, by definition, alpha quality. Or beta if stated. 1.0 is the first release you consider 'stable' and production-worthy.

My 3rd point was exactly what you think. You can't write in node-webkit with just HTML/JS/CSS, you need to know node.js as well (a much smaller subset of HTML/JS/CSS folks). And then write a bunch of plugins for all the things it can't do. Some of the things it can do, but not well. For example, fs.watch is listed as unstable, which makes sense as node.js is not considered 1.0 yet. The USB plugin you mentioned just hit 1.0 a few days ago, has been downloaded a total of 32 times, and only has a single developer. It requires two other plugins each also with a single developer, one of which is also pre-1.0. Honestly, the further you get, the more it confirms alpha quality (fun to play with, but don't build anything important with it yet). Many of these seem like proof of concepts still. And some of it seems like it is being built for the sake of the 'node.js is the hot new thing and can do everything' crowd. The same phase that every hot new thing seems to go through in the last few years (we could always write it in Ruby). That's the gist I get from it anyway.

Screen reader support in web browsers is one of the most difficult things to do and has to be done per-browser. Native control support is built into every screen reader, which is why they can do nearly all regular apps with ease (unless they use oddball custom controls). Google Chrome is only recently working properly with the standard screen readers. I really doubt that a chromium-hack to do local apps with node.js will work well with them. I could be proven wrong if you test your app with NVDA and it works well, though.

Mind you, I'm not poo-pooing what node-webkit could become in time. It's just that right now, it doesn't look like it meets our needs. It's far too young, immature and unstable (as in unfinished/unfinalized, not necessarily crashy).

Sometimes, the impossible can become possible, if you're awesome!

gluxon
gluxon's picture
Offline
Last seen: 4 years 5 months ago
Developer
Joined: 2008-06-21 19:26
If we're talking about

If we're talking about Semantic Versioning, Semantic Versioning Specification #4 and #5). That's not quite the same as unexpected behavior and crashing, and certainly doesn't mean we can't use it for something important. Node.js is at the 0.10 version number because there will be more API changes in the future, but it is used very extensively in the industry. Really, the common practice for an unstable application with unexpected behavior is to denote alpha quality on download pages, or somewhere on the site that it is not ready for usage. node-webkit and Node.js are both quite ready for production and I'm sure their respective lead developers would agree.

C/C++ Addons for Node are simply just wrappers and bindings for C++ functions. There's not a whole lot that can go wrong here if the developer understands the process.

I tested my app in NVDA. It worked beautifully. The only issue I found was reading of the "User Messages" pane and it's likely that the issue is tied to it being a textarea. I could and probably should change that to a div to fix it. I feel like some ARIA controls may also work to fix screen reading of that pane. On the other hand, I just noticed that the Platform (11.2 and 12 beta) have some issues. All I'm getting is "PortableApps.com Platform K" spit out at me. None of the items in the menu are being read to me. This contradicts the blog posts I've seen, so is there a setting I'm missing or something dumb I can't figure out?

Regardless, you've expressed your disconcert at the idea, so I'll drop it. There's no way for me to convince you that technology is ready if you don't think so. I enjoyed this discourse though. Smile

John T. Haller
John T. Haller's picture
Offline
Last seen: 2 hours 53 min ago
AdminDeveloperModeratorTranslator
Joined: 2005-11-28 22:21
Immaturity

The immaturity of the platform is one of my biggest issues. As is the extremely small number of developers. An unproven, immature, small volunteer team technology isn't something I want to base something this important on. And, regardless of the semantic versioning website, pre 1.0 means not ready for prime to most of the world. The C++ add-ons aren't merely wrappers to Windows APIs, they have to manage the communications in an ongoing way with those Windows APIs (example, listen for notifications from Windows for a USB drive insert), so we'd want those to be very solid and many of them are clearly very new and mostly untested. Couple that with the fact that the developers themselves label functionality we need as unstable (which I'm unsure why you keep glossing over), means that a platform written in it could be a fun exercise to play with, but not production-worthy enough to push out to millions of people who depend on it.

I get that it's interesting new technology and you enjoy writing in it and are a fan of it, but that's not reason enough to switch to it. Nor is it a reason to switch to Ruby or Python or C++ or Lazarus, etc. We need stability of the base product, some real proven large-scale apps as examples instead of really basic webpage-style apps, either an actual paid support policy or a large and active community around it, a larger team of developers would be nice instead of the ~3 that seem to commit, etc. I'd wager this is why you don't see any decent size apps written using it yet, just these small proof-of-concept type apps. It's still very early stage pre-1.0, so this is expected.

All the above doesn't mean it's bad or worthless, just not up to the level we'd need it at yet for PortableApps.com. And it does seem like a yet, because this has some really solid ideas and tech behind it. I'm honestly pretty curious to see how well this works and what can be accomplished with it. And it would be an excuse to learn node.js and refresh my JS skills, which I've been meaning to do. On that note, how much work do you think it'd be to get a MVP (minimum viable product) of the menu? Not the full updater or reading in non-PAC apps for now (that would need another plugin written), but just the base?

As for NVDA with the platform, I'm unsure why and will take a look. I tested it myself on Windows 7 x64 and it worked with both the local and portable versions. NVDA does have a tendency to change a lot between releases (it's not exactly stable and solid) so they could have introduced some regressions since then.

Sometimes, the impossible can become possible, if you're awesome!

gluxon
gluxon's picture
Offline
Last seen: 4 years 5 months ago
Developer
Joined: 2008-06-21 19:26
A minimum viable product

A minimum viable product would not be a lot of work. You're right that a full product would require a C/C++ plugin for icon extraction of non-PAC apps. That's probably the biggest mountain.

Creating the interface is done with standard HTML & CSS. That should take no longer than a day. The manipulation (renaming, organizing apps, creating context menus), should take around a week. I don't see a lot of work for a MVP.

I'm starting to think that the biggest potential for this is overhauling the App Store we currently have..

Edit: I'm willing to put in the effort to make a MVP for demonstration. I'll have to start after I finish the Driver Station app I screenshotted earlier though. I don't have a problem with waiting a year or even two years for these tools to mature.

Ken Herbert
Ken Herbert's picture
Online
Last seen: 54 min 12 sec ago
DeveloperModerator
Joined: 2010-05-25 18:19
Definitely an interesting concept

But AppJS is still very young, and just from a quick look at their bugtracker seems to have a lot of missing features.

I am all for it if there is a more mature alternate to AppJS, or when it works out some of its issues.

Gord Caswell
Gord Caswell's picture
Offline
Last seen: 2 weeks 4 days ago
DeveloperModerator
Joined: 2008-07-24 18:46
cross-posting from irc

I'm cross-posting this from irc so my thoughts are in the pertinent thread.

gluxonGordCaswell: I saw that you made a comment on the platform rewrite thread. Do you have an opinion to share about it?

MeI am in favour of a rewrite/port/fork/etc. of the platform to something that more people can contribute with. IE - something that doesn't cost $200USD Smile I'm personally leaning towards a Lazarus port at the moment, for the sole reason that the uinderlying architecture is similar, so a full re-write would not be neccessary. With that said, I'm open to whichever method is preferred by the community of developers as a whole ,whether that be pure C/C++, Qt C++, QML, Lazarus, Node.js, app.js, whatever.

Me, continuedI haven't had enough exposure to either app.js or node.js to form much of an opinion at this time, however in my limited browse of the appjs website, that particular tech seems to still be maturing.

Log in or register to post comments