One of the greatest features in the Webkit-based browsers (Apple’s Safari and Google Chrome) is WebSQLdatabase, the ability for a web site to store information in a SQLite database on your browser accessible via JavaScript. This allows web developers to build database-enabled applications that run entirely in the browser, without requiring a server. This is very useful for mobile devices, which in the US enjoy flaky network connectivity at best. One very handsome example is the iPad-optimized Every Time Zone webapp.
SQLite is probably the most important open-source project you have never heard of. It is a simple, streamlined and efficient embedded database. Firefox stores its bookmarks in it. Google distributes its database of phishing sites in that format. Sun’s industrial-strength Solaris operating system stores the list of services it runs on boot in it—if it were to fail, a server would be crippled so that is a pretty strong vote of confidence. Adobe Lightroom and Apple’s Aperture use it to store their database, as do most Mac applications that use the CoreData framework, and many iPhone apps. In other words, it is robust and proven mission-critical software that is widely yet invisibly deployed.
WebSQLdatabase basically makes the power of SQLite available to web developers trying to build apps that work offline, specially on mobile devices. No good deed goes unpunished, and the Mozilla foundation teamed up with unlikely bedfellow Microsoft to torpedo formal adoption of WebSQLdatabase as a web standard, on spurious grounds, and pushed an alternate standard called IndexedDB instead. To quote the Chromium team:
Q: Why this over WebSQLDatabase?
A: Microsoft and Mozilla have made it very clear they will not implement SQL in the browser. If you want to argue this is silly, talk to them, not me.
IndexedDB is several steps backwards. Instead of using powerful, expressive and mature SQL technology, it uses a verbose JavaScript B-tree API that is a throwback to the 1960s bad old days of hierarchical databases and ISAM, requires a lot more work from the developer, for no good reason. To add injury to insult, Firefox 4′s implementation of IndexedDB is actually built on top of SQLite. The end result will be that web developers will need to build a SQL emulation library on top of IndexedDB to restore the SQLite functionality deliberately crippled by IndexedDB. If there is one constant in software engineering, it is that multiple layers add brittleness and impair performance.
Of course, both Mozilla and Microsoft are irrelevant on mobiles, where WebKit has essentially won the day, so why should this matter? Microsoft has always been a hindrance to the development of the web, since they have to protect the Windows API from competition by increasingly capable webapps, but I cannot understand Mozilla’s attitude, except possibly knee-jerk not-invented-here syndrome and petulance at being upstaged by WebKit. WebSQLdatabase is not perfect—to reach its full potential, it needs and automatic replication and sync facility between the local database and the website’s own database, but it is light years ahead of IndexedDB in terms of power and productivity.
I am so irritated by Mozilla’s attitude that after 10 years of using Mozilla-based browsers, I switched today from Firefox to Chrome as my primary browser. Migrating was surprisingly easy. Key functionality like bookmark keywords, AdBlock, FlashBlock, a developer console, and the ability to whitelist domains for cookies, all have equivalents on Chrome. The main regressions are bookmark tags, and Chrome’s sync options are not yet equivalent to Weave‘s. At some point I will need to roll my own password syncing facility (Chrome stores its passwords in the OS X keychain, which is also used by Safari and Camino).
Hi, Fazal!
I’m sorry to hear you’re so upset with us. We’ve been pretty clear about our motivations and wrote about them here:
http://hacks.mozilla.org/2010/06/beyond-html5-database-apis-and-the-road-to-indexeddb/
I know that explaining this won’t change your mind at this point, but I thought it would be worth it to go through the reasons again.
The current implementations of sql that have been exposed through WebKit have all been exposed through the same implementation: sqlite. The language for which is deeply underspecified and changes (slightly) from version to version. We know this because we use sqlite internally. In fact, the language parts of the specification for sql on the web were intentionally left blank. This does not make for a great start to a standard.
When we went out to look at something that would be doable in this space – and we agree that it’s important – we talked to a lot of people. We talked to web developers, who said “we don’t think sql is the right thing to use, but we’ll use it if it’s there.” Interestingly enough we also talked with IBM and Oracle who both said “you should do something here, but please don’t make the mistake of using SQL.”
There’s another important item at work here: distributed innovation. We know that the way that the web is built now is with libraries, not low-level APIs. (Most people seem to be using jQuery or Dojo instead of using the DOM directly.) We wanted a platform that would allow people to build what they need instead of using one very specific syntax. That’s why it’s a low-level interface. It’s not a throwback, it’s just a lower-level primitive on which people can build other systems.
The IndexedDB interfaces allow people to build a couch database in the browser so that you could do sync easily, as you say is needed. Or you could build an sql interface. Or another data interface.
We realized that developers might a little bit allergic to these interfaces because they are lower-level than they are used to. But we also know that people will build interesting libraries on top of them so they can use it in a way that makes sense.
Anyway, thanks for your comments and sorry to hear that you’ve moved browsers.
Take care!
Thank you for your thoughtful and considered reply. I am aware of Mozilla’s arguments, in fact, I linked to them. I just consider them to be spurious.
As for the argument there is only one implementation, all modern OSes’ TCP/IP stack is derived from BSD 4.2, and there are more differences between versions than with SQLite, yet I don’t see a clamor to ditch TCP/IP and go back to C-Kermit running over serial lines. The spec issue could have been addressed by documenting a meaningful subset of SQLite.
The elegance argument doesn’t wash. SQL is an imperfect but close implementation of the relational model, itself basically just first-order predicate calculus, i.e. pure math. The DBM-style IndexedDB may not involve passing strings along, but it is remarkably hideous. Not surprising from something endorsed by the parents of such remarkably successful specifications as XHTML 2.0, I suppose.
The argument about impedance mismatch between objects and the relational model is an old canard. I usually see it only from wet-behind-the-ears programmers who don’t have the maturity to realize data is more important than the application, will outlive it, and data organization is more important than minor syntactic issues. There is a reason why OODBMSes have failed to set the world on fire and set-oriented SQL databases rule the roost.
The argument that lower-level primitives allow more flexibility for libraries is just as laughable. The Btree code in SQLite is a small part of the overall code base, and expecting a JS library to implement a full-fledged query optimizer is unrealistic, specially since it would have no clue as to what the underlying implementation looks like—the two layers need to be intimately integrated for performance.
The fact this will all be implemented on top of SQLite, rather than BerkeleyDB, is just cherry on the icing. IndexedDB also conspicuously lacks set-oriented operations, and thus any such operations, such as producing a report, will require multiple ping-pongs between the JS code and the underlying engine, with crippling consequences in terms of performance.
From the comments on the Mozilla articles, it is clear an overwhelming majority of web developers disapprove IndexedDB and its reinvention of the square wheel. The ones you talked to must be know-nothing NoSQL dilettantes, or companies like IBM and Oracle with a vested interest in selling complex tools and consulting.
Mozilla’s refusal to implement the WebSQLdatabase spec, and its alignment with Microsoft to sabotage the standardization of WebSQLdatabase, lead me to conclude Mozilla is now an impediment to the web just as badly as IE or Flash, and I will have nothing more to do with it.
“We wanted a platform that would allow people to build what they need instead of using one very specific syntax. That’s why it’s a low-level interface. It’s not a throwback, it’s just a lower-level primitive on which people can build other systems.”
I have to admit, I am confused. If IndexedDB was built on SQLLite, then isn’t SQLLite a lower-level primitive on which people can build other systems?
High-level vs. low-level has nothing to do with altitude in the stack and everything to do with expressive power. x86 machine language has not suddenly become higher-level than C simply because Bochs implements an x86 emulator in C.