inessential by Brent Simmons
17 Nov 2011

How about IMAP for RSS syncing?

On Twitter, Maize Hughes asked about using IMAP. Just store the RSS articles as email on an IMAP server.

He elaborated: “Imagine a desktop Thunderbird install as a mailserver I could get to from my iPhone.”

It’s a creative idea — I like the idea of re-using existing technology — but there are some problems with it. The first obvious one is that you won’t be able to get to your desktop from your iPhone when you’re not at home. (This is true for most people. I’m aware of DynDNS, yes.)

So the IMAP server would have to be on a publicly-accessible server somewhere. And then there are additional problems:

  • RSS articles can change. Emails don’t change, once sent.

  • You get a lot more RSS articles than email. (Obviously not true for everybody, but generally true.)

  • How will the IMAP server get the RSS feeds? Will it fetch and parse them on behalf of all its users? If so, that’s very expensive. If, instead, the clients will send the parsed articles to the IMAP server for storage, that means a lot more work on the part of the clients and a lot more bandwidth used.

  • An efficient RSS syncing system would take advantage of the fact that you don’t have to store a given article separately for each different user. If 100 or 100,000 users subscribe to Daring Fireball, it should store Daring Fireball just once, not 100 or 100,000 times. I’m not sure IMAP servers are designed that way.

  • RSS articles don’t have reliable unique identifiers. Good feeds do. Bad feeds have unreliable identifiers, and some feeds don’t have identifiers at all.

You could, I think, with a bunch of coding and fitting-things-together, cobble up a syncing system that uses IMAP. But it wouldn’t be very efficient, and it’s not something that would work for general use.

In early 2010 I wrote up an idea for a fairly simple RSS syncing server. I think it’s a solvable problem as long as you recognize the limits. But I don’t have time to work on it (and I no longer write a client app that would use it). Nevertheless, I still think the best bet is an open source system that is simple and easy-to-install.