Oct 2011

Updated Google Reader

ReadWriteWeb: Google Reader Gets the Google Plus Treatment. “No more friending, following and sharing; it’s all circling and +1s now.”

So that part’s done. Here’s the announcement.

Two paragraphs of the announcement are about taking your data and moving elsewhere, which is laudable — data export is super-important — but this is the second Google Reader announcement in a row to make such a big point of it. I hope I’m making too much out of that. I probably am.

How MarsEdit got its name

When I was naming MarsEdit, I wanted a name that was cool, that fit with NetNewsWire (which had an earth plus satellite icon at the time), and that suggested editing-at-a-distance — since MarsEdit didn’t edit local documents but blog posts, which are normally stored on a remote server.

I was thinking about names I thought were awesome, and remembered one of my favorite place names: the Free Mars café in Belltown. It occurred to me that “Mars” was perfect, because Mars is cool, it fits with NetNewsWire’s space theme, and Mars is at a distance.

I couldn’t call the app “Free Mars,” not just because it had been used but because it would have suggested the app was free. People would get confused — so instead I added the “Edit” part at the end to make MarsEdit. (Adding “Edit” at the end was an homage to the awesome BBEdit.)

Free Mars didn’t exist as Free Mars by the time MarsEdit came along, though it lived on, in a location around the corner, as the Cyclops. And it still lives on, and that’s where Seattle Xcoders go to eat and drink and talk twice a month.

I found an image gallery for Free Mars. Very late ’80s, very awesome. I loved that place. (I ate a lot of hummus there, and they served me beer when I was underage, thereby ensuring I would be a regular customer.)

And of course here’s the website for Cyclops.

P.S. I was also considering breaking out NetNewsWire 1.0’s outliner into a separate app called MoonLiner. Icon would have had the moon plus a passenger rocket — something like a 747 mixed with a spaceship. But OmniOutliner was so good that I didn’t bother doing MoonLiner. (Though I loved the name.)

UI Master Crit panel at Voices That Matter Boston

Some call them contestants, though they don’t win anything. I call them victims — but don’t tell anybody. That’s just between you and me.

They are the brave developers who will volunteer to have their app critiqued by a panel of blood-sucking vampire geeks UI experts in a session at Voices That Matter in Boston this November.

You should volunteer for the slaughter constructive criticism. There’s that one UI thing in your app that you struggle with — you know what it is. Bring it.

P.S. Use priority code BSTSPR7 if you haven’t already registered — it saves you $150. (The code is case-sensitive.)

RSS and desktop apps

Scripting News: “RSS has always been a great basis for a web app. I don’t think it should be a desktop app.”

In one sense, it doesn’t matter if Dave is right or wrong about whether or not RSS readers should be desktop apps, because there will always be people who love RSS and who love writing desktop (or iOS) apps, and they will want to work with both loves at the same time.

I like web apps and I use a bunch of them. I also like native apps. I’m not ideological about one or the other, but I happen to prefer writing native apps for a bunch of reasons. It’s just more fun for me.

As long as that’s true of a lot of people, you’ll continue to see native apps for RSS and for everything else. (Never mind the profitability or not — people will write native apps out of love.)

Google Reader + Iran

Alex Kessinger makes some good points about Google and Google Reader — including one that surprised me: Google Reader is big in Iran because it’s hard for the authorities to block.

How are you using Glassboard?

As NickB wrote, we made Glassboard so private we don’t know how people are using it. If you’re a Glassboard user, and you have a story to share about how you use it, please do send us email at stories@glassboard.com. Thanks!

Kippt seems cool

I’ve not yet found a bookmarking service that I love, but this new Kippt could be it. It’s attractive and simple — and, best of all, has no social aspect. It uses Twitter and Facebook. Why create another social network when an existing one will do?

“No, we don't do that”

Dave Winer on my explanation of RSS syncing: “It’s a million times more complex than RSS itself.” Yup.

Why “Just Store the App Data on Dropbox” won’t work for RSS readers

When I was developing an RSS reader, users often asked me if I would make it sync via Dropbox (or WebDAV or iDisk or similar).

For a while, years ago, before Dropbox existed, my reader did sync by writing files to iDisk. It didn’t work very well at all, and, looking back, I shouldn’t have shipped this feature. I made a mistake.

Syncing for RSS readers seems like it should be easy. In the simple case, you have three things to sync:

  1. The list of feeds (probably as an OPML file).

  2. The list of articles that have been read.

  3. The list of articles that have been flagged.

It seems like all you have to do is make each copy of your RSS reader read and write three files to Dropbox, one file for each item in the list above.

It seems like each reader could just read the files at startup and write out new files when it quits. Problem solved!

But not even close.

Apps run for a long time and run at the same time

If you launch a reader on your Mac and on your iPhone, it’s entirely likely that you might leave it running for days or weeks. That means that the apps would have to read and write periodically.

That leads to a couple questions:

  • How often should they check those files for changes? The more they check, the more performance is hurt. But the less they check, the less in-sync the apps are.

  • What happens if two or more apps want to access the same file at the same time?

  • What if you add a feed on one reader and delete a feed on another? If each app writes the entire feeds list, then one reader will see that the added feed isn’t in the list. Was it deleted? There are other easily-imaginable conflicts. You don’t want to constantly harass the user by asking them questions.

Lists can get very big

If you could solve the problems above, a large feeds list wouldn’t be too big to read periodically.

On the other hand, the list of read articles could be massive. It would get bigger every day. Reading and writing this entire list, were it a single file, would become a giant performance problem.

It would also be very prone to conflicts: if something isn’t on the list, but one reader thinks that item was read, does that mean the user marked it as unread, or does it mean that the other reader just hadn’t seen that item yet, or...?

If you could solve the conflicts problem, you could mitigate the size and performance issue by saying that you’ll only sync read states for recent items, for the last two weeks or a month.

Note that Google Reader syncs read states for just recent items, though the rules are a bit more complex than just n days. Doing it in a similar way would be a support issue forever, since people would still expect everything to be synced without any performance penalty — and rightly so.

Flagged articles may no longer appear in subscribed-to feeds

What happens if you flag an article, and then that item drops off the feed it’s in? Or what if you flag an article but then delete that feed later?

I think you expect to keep that article forever, or until it’s unflagged.

That means that you’d have to store the entire contents of the flagged articles on Dropbox too. For some people that could grow to a sizeable amount of data. Maybe not for most people, but the system has to be built to accommodate all users.

And, again, you have the issue of conflicts — and the consequences of losing a flagged article could be very bad.

Lists will not work

The above almost makes it sound like lists could work: all you’d need are policies to keep the sizes down, methods to deal with multiple apps wanting to read and write at the same time, and systems to deal with conflicts.

You could do the first two, but there are certain types of conflicts you can’t handle with this approach (particularly regarding deleting things).

In other words, the “just store the app data on Dropbox” approach won’t work.

A system that would work

I’ve worked with four different RSS syncing systems, and every one of them had holes. The ideal syncing system would work something like this:

An RSS reader would report actions and times to the system. For instance, it would tell the system things like this: "[Feed] was added to [folder] at [datetime]" and "[Feed] was deleted at [datetime]" and "[ArticleID] was marked unread at [datetime]."

By making deletes explicit and by including dates and times, the system would be able to deal with the conflicts that the list-of-data approach couldn’t handle. (A last-action-wins policy is sensible here.)

However, when pulling data, the clients would not want a replay of the actions: an RSS reader would want the current state of the feeds list, and lists of articles marked read/unread and flagged/unflagged since the last pull request. In other words, the system would have to have some smarts — it would have to know how to put that data together. Plain old files don’t have that kind of smarts.

(Why not just replay the actions? Say you just added a reader on your iPhone and you didn’t have one before, but you’ve been syncing for a year. Should the app on your iPhone download a year’s worth of actions? No.)

This system needs a web app with a database. The web app would not have to have an HTML UI: it could be built solely to support syncing, and be otherwise invisible.

But a web app with a database isn’t free — someone has to write and maintain the code and someone has to run it. Developer hours cost money. Bandwidth, storage, and CPU time cost money.

(And no, you can’t just put a SQLite file on Dropbox and use that for this case. Remember that each client would see a copy of the database — Dropbox just treats it like any other file that should re-sync on change.)

Mind the gap

The system described above would keep feeds and article states consistent between multiple readers, and that would be all that I personally would require. But it wouldn’t be enough for everybody.

The system above doesn’t handle this scenario:

Say you like to keep the last two weeks of articles in your readers. You normally read on your Mac, and you haven’t launched the reader on your iPad in a week. When you do, you expect your iPad to have the exact same articles that your Mac has.

But it doesn’t — it’s missing some, because some articles have fallen off the feed. So you write to support and explain that syncing isn’t working for you, because there’s a gap in the articles on your iPad. You prove it easily with screen shots comparing the same feed on your Mac and on your iPad.

To handle this case, the syncing system needs a feed retrieval system that would read and store the contents of all the feeds known by the system once an hour or so.

What happens when there are a million feeds in the system? This gets expensive. I wouldn’t be surprised if the feed retrieval part of this becomes the most expensive part by far.

And in the real world

Nobody wants to build and run this because there’s no money in it.

Google Reader and Mac/iOS RSS readers that sync

The recent announcement of changes coming for Google Reader worries me some. Though I don’t use Google Reader directly, I — like many iPhone, iPad, and Mac users — use RSS readers that sync via Google Reader.

This announcement isn’t just a reminder of the fragility of the system: it removes some features that people use. Google Reader’s social and sharing features are going away in favor of integration with Google+.

This makes sense from Google’s point of view — a single, Google-wide social network should be better than a separate network for each Google app.

Nevertheless, there are users who will miss the current, about-to-disappear features. (See What the Upcoming Google Reader Changes Mean for FeedDemon for an example of how one RSS reader will have to change.)

When I say that the system is fragile, I don’t mean that Google Reader itself is fragile. I mean that using it as a syncing system for other apps is fragile. There are a couple things that RSS reader developers know, that the average RSS reader user probably doesn’t know, which will make this more apparent:

  1. The Google Reader API is undocumented, unsupported, and unofficial. This is unlike Google Maps and others which do have documented and official APIs.

  2. Because the API is not official, there is no commitment to keeping it working.

Though RSS reader developers have received much-appreciated unofficial support from Google, that’s not the same as building a feature on solid ground. Smart developers are rightly uncomfortable relying on undocumented, unofficial, unsupported APIs.

But, because syncing is so important for so many RSS reader users, developers have had to gamble. With syncing part of the definition of a minimum viable RSS reader, and with Google Reader the only option for syncing, the choice to use Google Reader is more necessity than choice.

Questions about the future

The gamble has paid off so far, with the exception of the features about to be replaced. It’s been an especially good thing for RSS reader users, who can mix-and-match clients on different platforms, who can move with relative ease between one client and another. (The situation is similar to Twitter clients: users are not stuck with one client from one particular developer.)

While I remain optimistic about the future of Google Reader and apps that sync via Google Reader, there are a few reasons for concern:

  1. Though Google Reader is a mature app, its lack of an official, documented, supported API sends a message, intended or not, about its importance to Google. And Google has cut apps before, even high-profile apps like Google Wave.

  2. FeedDemon author Nick Bradbury asked on the unofficial API support list if Google Reader is being retired. At this writing, a couple days after the recent announcement, the question remains unanswered.

  3. Despite all the #googlereader and @googlereader traffic on Twitter since the announcement, the @googlereader Twitter account has been silent. (At this writing.)

  4. The wording of the recent Google Reader announcement emphasized how users could export their data. While that’s a good thing — being able to move one’s data is important — the emphasis made it seem like a request or a warning. That characterization is mine, yes, but it’s shared by other developers (not just RSS reader developers) that have read that announcement. This may not have been the author’s intention: it’s possible the author was being careful and doing the right thing by mentioning data exporting. But it didn’t read that way, at least to some developers.

  5. We don’t know what the value of Google Reader is to Google. It doesn’t appear to have ads a la search or Gmail, though ads do appear in feeds via Google’s AdSense for feeds, and that presumably makes money. Does Google Reader feed the search engine ranking system? Does it exist to help ensure the goodwill of influential writers who use Google Reader? I can speculate, but I don’t know, which is my point.

At any rate, I remain optimistic, because I don’t have any solid reason to expect things to change.

But Google Reader has just changed, and some syncing RSS readers will lose some features, and I take that as a reminder that it could change in a way that breaks syncing, and Google would not have broken any official APIs.

I’m not an RSS reader developer any more. But if I were, I’d start looking for an alternative syncing system right now.

Archive