Some weeks ago, I was pondering if it would be possible to do a native App like the Tagi iPad App (a typical newspaper app) in pure html(5)/CSS/JS (aka Niwea). So I did what a Liiper often does when he wants to explore an idea: Organize a Hackday.

Last Wednesday, Gerhard, Pierre, Fabian and I sat together and started from scratch with the project. To make the content and design part easier, we simply took the Tages-Anzeiger App as the guide. We didn't want to do something completely new, we just wanted to proof, that you can deliver a very similar experience to the end user with "just" an optimized website as with a native app.

So here's the result: http://labs.liip.ch/niweaapp/ (best seen on an iPad, but works on any modern browser)

It uses localStorage for saving the content locally (it doesn't work totally offline yet, but that would be doable in another day :)), fancy CSS stuff and some heavy magic on the swiping stuff. It also uses Pierre's excellent dispatcher library for Javascript based apps (see his talk about PhoneGap for more about that) and Hyphenator.js for more readable text, something the iPad App doesn't provide. It also has different layouts for landscape and portrait. The "app" of course could use some polishing here and there, but for only one day work with 4 persons, I'm pretty pleased with the result.

The code is published on github.

Btw, we didn't ask the Tages-Anzeiger, if it's ok to use their content for this "Proof Of Concept". I hope it is for them, if not, we will certainly find other interesting feeds (it takes RSS feeds as input and converts them to JSON on the server, so it would basically work with any feed, maybe some adjustments needed for the pictures).

As of today, we officially open source our iPhone/iPad app "Radios". We started Radios as a Liip Hackday project soon after the iPad was announced and it was clear from the beginning, that we will make the code public some day. We used a lot of Open Source code to build it, after all. We publish it under the same license as the underlying software we used. It's the very liberal MIT license and the whole code is available at our github repository. So feel free to do some great new stuff with the code.

The app is built on PhoneGap, meaning that the whole UI and most of the logic needed is written in html, JavaScript and CSS. It should be therefore very easy to get into it and enhance it. The more "core-ish" stuff is written in Objective-C and is described in another blog post.

For getting started, all you need is the iPhone SDK, an installation of PhoneGap and of course our code. Open the "radioapp.xcodeproj" in Xcode, Build&Run it in the simulator and you should be ready. If you later want to test the App on your actual iPhone, you also need a paid iPhone Dev Account (those 99$/year)

For some special features like the "3G/Edge detection" I had to patch PhoneGap and that fork is available at http://github.com/chregu/phonegap-iphone, but for just enhancing "Radios" with some new features, you don't need that fork, the standard PhoneGap should be fine.

I'm sure, you now want to go ahead, fork Radios and start developing for it. Let us know via comments or , if you did something cool and we will try to integrate it into the official version. And if you don't know, what you could add to it, here's a little wishlist:

* The possibility to add your own radiostations
* Use touch instead of click events
* A sleep function (turn off after one hour)

* More in our wiki

Apple finally approved the next iteration of our Radios App. It now not only runs on the iPad, but also on the iPhone. And if you already have iOS 4, you can listen to the radio while using other apps.

Unfortunately, something went wrong with the first iOS 4 submission. We couldn't really figure out, if it was our or "their" fault. If it would have worked out on the first time, you could have enjoyed background music from the day iOS 4 was released. Now you just had to wait another week and I'm very happy it ended up in the AppStore now.

And as promised we will open source the whole application. This week. Stay tuned :) If you want to know some technical details already, read this blog post.

Almost two week ago, we released our iPad App Radios. And today the first update, with more features, more radio stations and multilingual interface, was approved and deployed by Apple.

Radios plays mp3 streams on your iPad from some Swiss radio stations, so nothing special. But additionally, it also displays which band and song is currently playing including artist information and pictures from last.fm. All in all, it is an app which is not designed for the time you spend with the iPad on your couch, but for the many hours the device sits on the sideboard next to your couch. And it's really fun to see the pictures to the bands currently playing on the radio, especially if the pictures are a little bit older :)

The project started with the idea of trying out this new tablet format and a Liip hackday. I made some proof-of-concept work with the mp3 streaming part and then four Liipers sat together a whole day (a so called hackday, it's an institutionalized format at Liip, where you get time to "hack" on whatever you want) and built that thing. Or most of it, at least :)

The app uses PhoneGap as the underlying part. With PhoneGap one can program an iPhone app with "just" html, JavaScript and CSS. It provides some additional APIs you don't have in Mobile Safari for some device specific features, like vibration or the accelerometer.

While PhoneGap has an API for Sound, it doesn't have one for live streaming and reading metadata from that streams. We needed the first to stream the radio station feeds and the second to display the artist info. I started googling and quickly found this "Streaming and playing an MP3 stream" article by Matt Gallagher, incl. Objective-C code and all. Integrating that in the app was a breeze, even with my pretty basic Objective-C skills.

Next task then was to be able to call those newly added methods via JavaScript. Thanks to PhoneGap that wasn't too hard to implement. Basically you create some "special" methods in Objective-C, like the play method in AudioStream.m and then add some "special" javascript methods like the play method in AudioStream.js, which basically just calls the native method. It's easy as that. If you need return values from a method call, you have to do that with callbacks, see eg. getMetaData.

We also added a very simple Lang class to get the user language for displaying the info in the right language.

The one problem with the above mentioned MP3 streaming class was, that we didn't have the info about which artist is currently playing. I tried to figure it our by myself how that works. Most radio stations are using the Shoutcast Metadata Protocol. Fortunately I didn't have to implement it by myself and I found the audiostreamer-meta class by Mike Jablonski, which was based on the work abov. I put some wrapper around it and my work on the native Objective-C side was almost done.

The whole code/fork of phonegap-iphone with the mentioned plugins can be found on github at http://github.com/chregu/phonegap-iphone, the audiostream plugin code ist at http://github.com/liip/phonegap-plugins-audiostream

After having done that underlying work, we were able to start with the html/JS/CSS work for Radios. It was almost standard procedure, but with trying out some of those new CSS3 and html5 features, it took some time to figure out some things. The scrolling wasn't an easy task either. But at the end, we have a standard based setting, which could be easily ported to Android (if the streaming/metadata part was ported to Android/Java, but that's doable), or to a "Browser-only" based version. Fabian actually did a version based on this, which worked perfectly fine in a standard browser. It just does the metadata stuff on the server side.

We really think, html/JS/CSS (aka NIWEA) based development of (not only) mobile apps is the future and where the future is too far away today, we try to mix it and get the best of both worlds, like here with this Radios app. We're actually doing a rather big customer project on PhoneGap right now, which will go online pretty soon and which allows us to deliver to iPhone and Android with almost the same code base.

More about that "Apps with Web Technologies" topic was published yesterday by Hannes in the blog post "What's NIWEA?".

We will also Open Source the app very soon, we just have to clean up some things. Watch this space.

And last but not least, many thanks to Fabian, Peter and Roland for their development work. And to Hannes, Fabienne, Memi and Tobias for the input during the brainstorming phase. It was a fun project and I'm sure it will live on and get some nice new features (like adding any station :)) in the near future. Thanks to the Liip Innovation and Training Program there's certainly time and budget for that.

This is an idea I had mind in since last year, and finally was convinced to put online these days. Hoping you'll like it, Hannes

If anything about current web development can be called «glamorous»[1], it’s creating applications for en vogue platforms such as the iPad or the latest line of Android phones. After all, they're not only super sexy, but the rate of progress we see is staggering as well - a pretty tempting combination.

At the same time, web technology is constantly taking up more and more speed, progressing on many levels at an exceptional pace. Standards such as HTML5 and CSS3 as well as their support in transformative tools such as Webkit have been helping web user experience improve dramatically and at a much greater rate than the progress we have seen on any "desktop" environment - which almost stagnated for a decade.[2] And let's not forget: seeing the speed of JavaScript engines improving by orders of magnitudes clearly means: something important is happening here.

This blog entry is trying to deal with the great deal of stuff that's happening when these two ongoing developments -amazing devices and accelerating web tech- come together. That «stuff» we see out there has such as breadth, depth and scale that we shall be trying to get from stuff to patterns, and then give them names. Names that help us actually have clearer conversations about those things as they develop, like folks did with «AJAX», «RIA» or «Web 2.0». The name we're suggesting is NIWEA - Native Interoperable Web Apps. Let's see what this means:

«Native», the N in NIWEA, is a word that has been bended and stretched for years. We'll use it in just two simple ways:

• «Made for this thing» - native apps just do feel right - on your desktop, your mobile, your tablet, your TV.
• «No crutches» - web native apps use use the tech and specs of the web - bred through open standards, no plugins, no add-ons.

 «Native» also means access to the capabilities of the underlying platform: sensors, geolocation, graphics, storage, background computations, audio, video - you name it.

In the example sections there's an whole array of examples illustrating this point, runtime environments such as PhoneGap, systems like Jolicloud, native user experience enablers like jQTouch, Google's Store for Installable Web Apps are certainly amongst those that must be mentioned.

«Interoperable» can mean a lot as well, but it's a strong notion, and one that keeps getting more and more important. The «I» in NIWEA means two things:

• «Works with my sites» - NIWEA can be standalone, but mostly they'll be talking, synchronously or asynchronously, to web APIs. APIs based on web standards that have interoperability built in, enabling mashups
• «Works with my gadgets» - another kind of interop, the "write once, run anywhere" idea. Indeed, web technology is the only platform I see that has the potential to overcome the terrible fragmentation we see in mobile platforms these days. Being the only platform without an owner certainly helps.

Interoperability between devices and consistent communications between applications - browser engines, REST & Co. can do it.

NIWEAs are built using web technology, and nothing else - but «web» doesn't mean exclusively «browser» any more. And «web» doesn't mean «online only» any more either. In fact we're going more and more towards what you might want to call an «ubiquitous web», with the browser engine becoming the universal runtime.

The are many ways NIWEA bring application functionality to you, be it packaged Web OS, Chrome Store or HTML-based Adobe AIR apps, desktop «gadgets» or dashboard «widgets» - it's all native-feeling, interoperable web tech.

Why care?

So why is this relevant at all? Aren't those iPhone and iPad apps out there just so superior that there's no need to even bother about that old-school web stuff? Well, I think this is about the long term, about what we can do to foster "sustainable" development practices that are truly open and e.g sufficiently safe against lock-in. In the short term, proprietary platforms might be lucrative and spectacular, but the benefits of having web tech "everywhere" might very well be far greater: the web is where the developers are, and enabling them to do NIWEAs could very well be pretty disruptive in many areas. And: we should never ever underestimate the web.

And now?

Having a word for all that stuff happening these days could help a lot, I think. So please do let me know: what do you think about it? Do you like it? Is it stupid enough? Or way too stupid?

Would you tag your bookmarks with "NIWEA"? Could we have a "NIWEA Conference"? Would you sign an "I support NIWEA" list? Have any links that just have to be on the list below?

Already thinking about funny logos? :)

Examples & Links

• Mobile Examples
  
  • Phonegap
  • jQTouch
  • Palm webOS
  • Nokia Web Runtime
  • JIL & BONDI
• Desktop Examples
  • Jolicloud
  • Chrome OS & Chrome Web Store
  • brevity
  • Apple Widgets & Windows Gadgets
  • Adobe AIR with Webkit
  • Mozilla Prism
  • iTunes Extras
• Miscellaneous Links
  • http://www.quirksmode.org/blog/archives/2010/03/html5_apps.html
  • http://www.quirksmode.org/blog/archives/2010/04/mobile_web_or_o.html
  • http://www.zeldman.com/2010/02/01/flash-ipad-standards/
  • http://www.toolness.com/wp/?p=821
  • http://radar.oreilly.com/2009/05/google-bets-big-on-html-5.html
  • http://radar.oreilly.com/2010/02/a-prism-for-jolicloud-web-cent.html
  • http://ajaxian.com/?s=native
  • http://humanapi.org

Footnotes

[1] For a note on glamor also see the original Ajax article
[2] See also Google I/O 2009 keynote for web vs. desktop UX - (not overly scientific) slide, video

Yesterday our group participated in a rather special hack day at the local.ch offices. The topic was JavaScript, and all engineers that were already fairly knowledgeable in it picked a topic and presented it in 15 minutes. Additionally, they created a "mission", which was a short exercise that should be solved in pair programming and that required using the techniques that were presented.

We developers that are working with the local.ch team each did a short presentation:

• JavaScript Tools - Tricks and Tipps by Patrick Jezek
• JavaScript the right way by Joël Bez
• Functional programming in JavaScript by Michael Käser
• Modular JavaScript by Marco Bohler

At the end of the day I finished with a much shorter version of my talk about node.js from the last webtuesday. For the mission, I tried to create something that is easy, fun, and highlights the ideas behind node well.

The task was creating a small asynchronous web application where engineers can register themselves for Foosball, and then get immediately alerted when there are enough players. I provided a skeleton that already does nearly everything and only needs about 10 lines of code to be finished - but these lines require you to think a bit about asynchronous programming, events, closures and the concepts behind node.js.

If you are interested, you can download the mission for yourself here and give it a try. It includes the skeleton and a manual explaining everything. More information about it can be found on my personal blog, where I can also be contacted in case of open questions.

Today, Hannes and me were talking about html5 at the aptly named Brasserie Lipp for the Internet Briefing TechTalk series. There were quite a few people and it went pretty well (IMHO :)), maybe a little bit too long. Sorry about that and the hungry people.

The slides are available on slides.liip.ch, and if you want more information about html5's spread, there's more in this older blog post about the html5 feature comparison table by me.

Last Thursday I gave a techtalk [1] at Liip Zurich about which html 5 features are supported by what (modern) browser, incl. my personal recommendation what's worth following and what maybe not yet.
The result is this spreadsheet and these slides (but they are basically just the spreadsheet spread across many pages :))

The table needs some explanation. The first few columns (the browsers) should be clear, gears hopefully, too. The "Flash/JS Fallback" states, if there is a possible and feasible Fallback in Flash or Javascript.

The "Usable with latest?" column summarizes, if that feature is available in Firefox 3.5, Safari 4 and Internet Explorer 8 (sorry, no Opera here). This is under the assumption, that if this feature is supported by those three browsers, that it will actually have some market share in a few years (...). Anything else (basically meaning anything which is not supported by IE8 now) will have a hard time in the "shorter" term.

WIth "Useable with decent degration?" I wanted to state, if you could use that feature for people with modern browsers, but still support people without (maybe with less features). Without having to go through hoops in your code, eg. Native JSON is pretty easy to support.

The last column "Recommended in a 'controlled' environment (IMHO)" is just my humble opinion. It rates each feature, if it is worth to develop for/with, if you can somehow control (or know) your environment. This column assumes, that your audience is either using a modern browser or is able to install Google Gears. Still, to get a "green" by me, there has to be a decent fallback, even if that fallback is just to properly *not* support it (meaning that it doesn't hinder people using the interface without having the support for that feature).

As I said, the last 2-3 columns are my personal opinion (either the content or how I assembled it), which may differ from yours. So I'm really interested in getting feedback. I still hope it may be useful for you, I certainly was dearly missing such an overview and getting all the facts together was quite a task. I will also apply new findings to the table in the future

[1] Techtalks are a weekly institution at Liip, where a Liipie (or someone else) talks about something interesting for half an hour just before lunch.

You may remember the problems we had with Firefox 3.0.0 and the encoding in XMLHttpRequest back in June. The Firefox team now tried to fix it (Issue 431701 in their Bugzilla), but failed miserably, they made the problem worse (unintentially, of course :)). While it was technically correct before, now it's not even that, as it still sends ISO-8859-1 encoded content, while it claims it's UTF-8 under some circumstances. The bug reports claim, that it only happens on non-US versions of firefox, but that doesn't help us much. This new regression is tracked at #464958, is already fixed and should be in Firefox 3.0.5, which is due around mid-December. With the auto-update feature of Firefox, a lot of our users will have updated to 3.0.4 and can't really wait until mid-December, so we had to find a workaround...

As it is really a bug this time and not just a different implementation of the problem, a correct and beautiful solution can't be done, so we had to fall back to some ugly trickery. When the DOM Parser can't read the XML string due to malformed UTF-8, we check, if it could be an ISO-8859-1 string and convert that to UTF-8. If it then still doesn't work, we just tell the parser to disregard all invalid UTF-8 characters as a last resort (The patch).

I currently see no other decent way to tackle this problem. "If it's not UTF-8, even if it claims to be, assume that it could be ISO-8859-1 and try with that" seems to be an ok-ish idea here in the Western Europe Alphabet World. Anyone else should maybe really wait with the upgrade until Firefox 3.0.5 is ready.

In Firefox 3.0.0 there is a "strange" regression issue regarding the encoding of XMLHttpRequest requests. It's not a bug per se, it's just different behavior, which we ran into (and no other browser does it this way)

What we basically do on the client side in JavaScript:

this.data = new XMLHttpRequest();   
this.data.open('POST', dataURI);   
this.data.send(xml); 

where "xml" is a DOMDocument Object.

In Firefox 2.0 this request came with a

Content-Type: application/xml

and the xml in the POST body was encoded in UTF-8 (no encoding information in the XML declaration)

IE7 does:

text/xml; charset=UTF-8,

But Firefox 3.0.0 sends this as

Content-Type: application/xml; charset=ISO-8859-1

and the xml in the body is actually ISO-8859-1 encoded, but there is no encoding information in the XML declaration (eg. no <?xml encoding="ISO-8859-1"?>) and of course our XML loader fall flat on its nose, when it had non-ASCII characters in it...

While having the encoding information only in the HTTP header and not also in the XML declaration is (as far as I can remember, didn't look up any specs) correct from a technical point of view, it was pretty annoying to find this "bug". And now I have to check on the backend, how the request is encoded on that request on not just rely on "it's UTF-8 nowadays anyway or at least written in the XML declaration, so the XML parser can take care of it" (which was maybe naive from the beginning :))

Here's the code-snippet for the PHP server side:

function transformFromContentTypeToUTF8($str) {
    
    if (isset($_SERVER['CONTENT_TYPE']) && preg_match('#charset=([^/s^;]+)#',$_SERVER['CONTENT_TYPE'],$matches)) {
        if ($matches[1] == 'UTF-8') {
            return $str;
        }
        if ($matches[1] == "ISO-8859-1") {
            return utf8_encode($str);
        }
        return iconv($matches[1],"UTF-8",$str);
    } 
    //if no charset, then return as it came
    return $str;
}

function fixXMLEncodingFromHTTP($xml) {
    if (!preg_match("#<?xml[^>]+encoding=#",$xml)) {
        return transformFromContentTypeToUTF8($xml);   
}
return $xml;
}

$rawpost = fixXMLEncodingFromHTTP(file_get_contents('php://input'));    

// create a new DOM document out of the posted string
$xmlData = new DOMDocument();
$xmlData->loadXML($rawpost);

BTW, for non-ISO-8859-1 characters, FF 3 does transform them to numeric entities, welcome web 1.0 :)

And there's already a report of that issue on bugzilla, of course. But no idea, if they change that back soon