January 2018

Check out episode #7 (I wish we had called it 007) where Ken Case talks about all the good stuff coming up in 2018.

There will be new releases of OmniOutliner, OmniPlan, and OmniGraffle — plus a whole bunch of great stuff for OmniFocus 3.0 for Mac and iOS, including tags, JavaScript scripting, and OmniFocus for the web.

You can subscribe to the podcast or just listen to this episode — there’s a player on the page. Or you can read the transcript.

* * *

I’ve been wondering if other companies in our world are doing similar podcasts.

There is the Supertop Podcast from our friends who make Castro. And there are much larger companies (such as Microsoft) who create videos and podcasts.

It may be that Omni’s size is somewhat unique in the Mac and iOS world: it’s big enough that we won’t run out of people to interview and topics to talk about, which could be tricky for a 5- or 10-person company. And it’s small enough to still be intimate — my goal is to eventually interview every single person who works here. (Though, of course, people can decline: it’s not required!)

Anyway: there was a time before companies had blogs, and now they have blogs. I wonder if more and more companies will find value in having a podcast too.

(And a Slack group. And a Micro.blog account.)

When IndieWeb started, some years back, the first thing I noticed was that they appeared to be against the idea of feeds — or, at least, what they called side feeds: RSS and similar.

Their idea was that the data a reader might collect should be encoded in the page itself, using microformats. The argument (I think; I hope I’m not misrepresenting anyone) was that microformats are simpler than mantaining a separate file.

I don’t know which thought occurred to me first:

I took it personally, since I’d spent much of my career working with feeds, and I took this as a suggestion that my career was all wrong and they wouldn’t be interested in someone like me, someone who would be an ally in their committment to the open web.

I objected on practical grounds, too. Feeds have been and remain tremendously successful — see podcasting, for one thing — and the way forward is to build on that success. And, while I understand the elegance of microformats, side feeds have critical advantages that microformat-feeds don’t.

So I decided to ignore IndieWeb.

Here’s the thing, though: that was me being a total jerk.

(I’m telling you about this so you can learn from my mistake.)

Fast-forward

In the last couple years I’ve been chatting with Manton Reece, Micro.blog creator, a bit — and Manton agrees with me about feeds.

But Manton also works with IndieWeb, and has mentioned any number of good ideas they have that I should look into.

It took me a while, but I realized that I was acting like a Clinton or Sanders supporter still arguing with the other side about the 2016 primaries — after losing the general election to a monster, after the point where those small differences matter at all.

And then I started to get excited about IndieWeb. I may have a difference of opinion when it comes to feeds, but who the hell cares when there’s so much good stuff to do? (They’re not holding my opinion against me; why should I hold theirs against them?)

And so I decided to support the h-feed format in Evergreen (hopefully in 1.0). That’s just for starters: I also decided to look into everything else and see what makes sense to support.

My default position now is: if a format or API or syncing service makes sense for a feed reader, then I’ll (at least try to) support it.

I published the first draft of Evergreen’s Coding Guidelines yesterday.

I posted this because I’d like to have people help me with the app — but I’m not ready yet. Or, rather, a few people who I know are starting to help, and I want to keep it small for now since I’ve never done this before.

If you’re interested in helping, I hope you’ll forgive me as I take it slow and learn how to manage an open source app with many contributors.

* * *

This isn’t the only app I want to do: there are several others. Ideally I’ll get to the point where I’m managing several open source apps — assuming people are interested in helping, which is a big assumption — and it’s all going great. But it will take time to get there.

(All my ideas are Mac and iOS apps that have something to do with the open web. I’ve always been happiest when working at that intersection.)

Anyway… it seemed like a good idea to write up coding guidelines early, so I did.

PS When I was faced with that blank document window, I started by typing the words “No subclasses,” because sheesh I really, really hate subclasses. The rest tumbled out.

There are, as I mentioned previously, two types of syncable actions: immediate and deferrable.

An immediate action is something like adding a feed: it requires that the server is reachable right now, so that the feed actually gets added.

A deferrable action is something like marking articles as read. The sooner the server knows, the better, sure — but it can wait if necessary. It can even wait for weeks or months.

I’ve decided to make it simple and categorize these actions like this: any structural action that affects the list of feeds and folders (or tags) is an immediate action, while any action that affects the status (read, starred) of articles is deferrable.

Or, in UI terms: if it’s something you do in the sidebar, it’s immediate; it it’s something you do in the timeline, it’s deferrable.

Most importantly: you can read articles while you’re offline.

Discarded alternate approach

You could argue that all actions should be deferrable. For instance, Evergreen could remember that you want to add a given feed, and add that add-feed action to the sync queue and wait for the server to become reachable again.

But doing so would add more chance for conflicts. Consider the case where the add-feed action got queued (on your day Mac) and then just sat there. Then, on your night Mac, you add the feed and it succeeds — then you decide you don’t like it after all, then delete it.

The next day, back on your day Mac, the add-feed call finally goes through, and then you have to delete it again. (And then you report an Evergreen syncing bug.)

There are other issues as well: do you add the feed locally? And read it? No, because this gets ambiguous quickly, as Evergreen’s article IDs won’t match the article IDs the server has assigned. Evergreen might not even have found the same feed URL the server found (in the case where you just gave it a website URL).

So we’ll stick with these two categories, immediate and deferrable.

To refine the definitions: a deferrable action is one where the state change can be reflected locally without risking serious conflicts or ambiguity.

Immediate actions

These will result in an immediate API call, and the result will be reflected in the UI. This is straightforward.

Deferrable actions

These actions will be stored in a database.

The simplest way to do this is probably a set of tables: articlesToMarkRead, articlesToMarkUnread, articlesToStar, articlesToUnstar. Each table would have one single column: articleID.

This layout is probably the most efficient way to add/delete articles.

This is not actually an ordered queue, but I think that’s okay. Most of the time all four tables will be empty.

When articles are marked read (for instance), then those articles will be added to articlesToMarkRead and deleted from articlesToMarkUnread. The system will then know that it has pending status changes to the send to the server.

Then, periodically, the app will attempt to contact the server. Most of the time this should happen within a minute or so.

Part of the idea is using this system to coalesce calls: the app shouldn’t call the server every time you select an article (which marks it read). Better to update the database and call the server very-soon-ish, but not necessarily this exact second, to allow for multiple articles to queue up.

The downside to this particular layout, which may make me change it, is that supporting additional status types means adding more tables. But the alternative is to add more columns to a single table, which is better but not necessarily that much better.

Another downside is that there’s no automatic guarantee that an article ID won’t exist in, say, both articlesToMarkRead and articlesToMarkUnread at the same time. But the answer here is simple, careful coding: a single bottleneck function that never adds to one without deleting from the other.

Most of the time, a user will do a thing — mark some articles as read, for instance — and then Evergreen will tell the server (Feedbin, Feedly, etc.) right away. That’s good and easy.

The hard part is this: what happens when the server is not reachable for some reason?

Continuing the example: Evergreen could, in the case of the unreachable server, refuse to mark those articles as read.

It could insist that syncing works only when the server is reachable — which is not that weird, especially given that it can’t pull new data from the server unless it’s reachable.

But it’s weird enough. Consider that selecting an article marks it as read. This policy would make it impossible to even just read what you’ve already downloaded.

Offline Syncing

It’s clear that syncing requires some sort of offline ability.

Rather than set a policy for each possible user action (marking articles read, deleting a feed, etc.), I want to set a single policy and single structure for handling these actions.

(Note: there will be some things that do actually require a reachable server: downloading new articles and adding a feed come to mind. This design refers to actions where it would be okay to defer notifying the server.)

Proposed Policy

When the user performs a syncable and deferrable action, that action and the relevant parameters will be remembered.

Evergreen will attempt to notify the server periodically. Once it succeeds, it will forget that action and its parameters.

(Update 18 Jan: 2018: the original version of this design had a time limit, which has been removed. The concern was that this offline queue would grow unbounded — but it can’t, since if you can’t reach the server you can’t download new articles, which means the queue is limited to your current local data set.)

Proposed Implementation

When the user performs a deferrable action, the app tries immediately to notify the server.

If this first attempt fails, then the action is recorded in a SQLite database with a timestamp and with the parameters it needs to be able to make that call again later. Those parameters will be minimal — article IDs instead of entire Article objects, for instance.

The app will periodically attempt to empty this database: starting with the oldest action, it will notify the server of each. On success, a given action is removed from the database.

Undo

Evergreen supports undo as much as possible. For instance, if you mark a number of articles as read, you can undo it.

In this case, Evergreen will notify the server of the mark-read action, and then, upon undo, notify the server of a mark-unread action, thereby undoing the effect of the first action.

In the case where the mark-read action is in the offline action queue, that will be found and deleted, and there will be no queued action to reverse the effect.

Non-Deferrable Actions

Actions that require an immediate response from a reachable server include actions such as adding a feed. In the case where a user attempts one of these and the server can’t be reached, an error sheet will explain the problem. Such actions will not be queued.

There will not, however, be an error sheet for performing a refresh when the server is unreachable. Instead, in that case, a warning icon is placed next to it in the sidebar, a la Mail and similar apps. Clicking the icon will attempt a connection to the server, and an error sheet explaining the problem will appear if the server still can’t be reached.

Other Notes

Evergreen works like Mail and similar apps in that it has a set of accounts for services such as Feedbin, Feedly, and others, along with an On My Mac account that is not synced.

Each account has its own section in the main window’s source list; each account has its own set of folders and feeds.

Accounts will be added to Evergreen in Preferences, in an Accounts pane.

The policies of these accounts will be respected. For example, if the On My Mac account automatically marks articles as read after n days, it’s possible that some service does this after some other number of days, or never does that at all, and Evergreen won’t override those policies for that service.

Another example: the On My Mac account doesn’t support folders-within-folders — but if a feed service does support that, then the app will support it too, for that service.

(Current plan: Evergreen 1.0 will ship with Feedbin support, and Evergreen 2.0 will add support for more services.)

Let me know if any of the above sounds weird or if I missed anything. (There is contact info at the bottom of Evergreen’s main page.)

This episode features Liz Marley, OmniGraffle engineer, formerly a tester and PM.

She talks about switching from testing to writing code — and about how she came to do technical talks using Swift playgrounds. Playgrounds with ducks, that is, because every good playground has ducks.

The latest build of Evergreen, fresh from the lab, now adds MarsEdit to the sharing menu in the toolbar.

When you choose the command, it sends the current article to MarsEdit — which opens it in a new window, and then you can edit and add your own commentary before posting to your blog.

* * *

See SendToBlogEditorApp.m for the code that packages up the article and sends an Apple event.

The code is not MarsEdit-specific — other apps in the past have supported this same Apple event, though I don’t know if any other current apps do. If I find some, I’ll add them to the sharing menu alongside MarsEdit.

* * *

The Apple event code is written in Objective-C, even though I almost always write new code in Swift. But, with Apple events code, Objective-C is easier.

It’s easy to call from Swift: see SendToMarsEditCommand.swift.

(Reminder: this is all MIT-licensed. You can use this code.)

The latest build of Evergreen adds Micro.blog to the Share menu in the toolbar, if you have the Micro.blog Mac app.

It sends the title and link of whatever you’re reading over to the Micro.blog Mac app, and you can edit it before actually posting.

This is hugely important. RSS readers exist not to just make reading easy but to make the web a conversation.

The next release will (probably) include MarsEdit, for the exact same reason. If you have ideas for other apps to include, let me know (see the bottom of the Evergreen site for contact info).

Ideally, every app like Micro.blog and MarsEdit would have a sharing extension — but, where they don’t, I can add them as long as they support some kind of way to do so.

Technical Details

Micro.blog supports a URL scheme for sending it a post: see See SendToMicroBlogCommand.​sendObject for the implementation in Evergreen. (And of course feel free to use any Evergreen code in your app.)

History

It’s not that we realized just today that posting from an RSS reader is important. Since posting was always an absolute requirement, way back in 2003 I released NetNewsWire 1.0 with an integrated blog editor.

I got that idea from Radio UserLand, which was an RSS reader and blog editor by the company where I worked before I wrote NetNewsWire. It’s Dave’s idea, not mine.

By the time I was working on NetNewsWire 2.0, it became apparent that smooshing a blog editor into a reader app meant that the blog editor suffers. The blog editor in NetNewsWire 1.x was pretty darn bad. So I had the idea of splitting out the blog editor to a separate app — inducing mitosis — which you can read about in my MarsEdit report of late 2004.

Mitosis was a success.

One of the keys of that success was that the API for contacting MarsEdit had to be open, and NetNewsWire had to support any blog editor, or any other app, that supported that API — because otherwise I would have been unfairly leveraging success with NetNewsWire on behalf of my new blog editor, and that would have been wrong.

MarsEdit still supports that interface, and my next step is to write the client side in Evergreen, so it can send to MarsEdit and any other app that supports that interface.

I am, by the way, delighted to be writing code for a 14-year-old API that still works.

Ben Curtis posted feed_searcher to GitHub</a> — it creates custom search feeds, and even has a handy Deploy to Heroku button. Cool.

(This is in reference to App Idea: Mentions).

I had this idea around the time Apple came out with Bonjour (née Rendezvous), and all these years later I realize I’m never going to get around to it.

Here’s the scoop:

The idea is microblog posts (tweet-like) but that live inside a specific network only.

You run a Mac app that:

• Lets you post short messages.
• Shows short messages from people inside your network.

The app runs a small webserver that’s discoverable via Bonjour, that has a specific service name. That webserver publishes a feed of your posts.

The app also downloads a feed from every other app it finds (and caches those feeds, for when other apps are offline).

In other words, it’s instant office microblogging with no centralized server. Completely peer-to-peer.

The design is what you expect — a reverse-chronological list of posts with usernames and avatars. You can reply, mention people, mute people, etc. (I think you’d follow everybody on the network automatically — so muting becomes the thing, not following.)

* * *

Important thing: your posts are network-specific. So if you go to a coffee shop or the airport with your Mac, you won’t be publishing your at-the-office posts.

* * *

I haven’t done research into whether this is do-able on iOS. Is it possible to run a little webserver, in a backgrounded app, that other people on the local network could connect to? Seems unlikely — but I don’t know. Cool if you could.

* * *

I have no idea how you’d make money with this one. If you charge for it, not enough people would use it to make it worthwhile. (The standard dilemma of social networking apps.)

Maybe, though, you could add some IAP features? One might be to let people add RSS feeds, so it could also function as a lightweight RSS reader. A person might want posts from Daring Fireball and wherever else added to their timeline.

Or just add that feature anyway, and do the whole thing for fun.

PS “Inside Story” is a long name. Should be one word. Scoops? Bulletins? Officecast? Gossip? (“Gossip” was, if memory serves, the name of an app Chuck Shotton was working on a long time ago. Probably reusable at this point.)

(I mean, c’mon, office Gossip. It’s so perfect.)

“Hold on — I need to check my Mentions.”

Ten years ago or more we had several blog-specific search engines and services: Technorati, BlogBridge, and others.

One of the great things about these services was not just being able to search for something but being able to set up persistent searches: that is, you’d get a search as an RSS feed, and in your feed reader you’d get results from all over the place on the thing you’re searching for.

In the obvious and common cases, you’d set up searches for people linking to your blog, writing about the apps you work on, mentioning the place where you work, and mentioning you.

I’d like to see something like this for today, but where the scope is just the Apple/Mac/iOS community. It would crawl the obvious sites (such as Daring Fireball and Loop Insight) and it would crawl the many blogs and microblogs that make up the community.

I think this is best as a web service, though it could have Mac and/or iOS client apps. It needs to provide feeds for people using feed readers.

* * *

With that reduced scope, and with the better tools and cheaper cloud computing these days, I don’t think it would be terribly expensive. Not like it was 15 years ago.

You might want to make money with this, though, and there are a few ways to do it: charge extra for things like notifications and email alerts. Charge money for client apps. Make the first three search feeds free, and charge money for a ten-pack of searches. Etc.

* * *

An alternate version of this idea isn’t a web service — it’s a Mac and/or iOS app that does the crawling. A kind of specialized Mac/iOS feed reader.

The list of feeds-to-crawl would probably just be an OPML file on the web, and the app would periodically grab that list.

(If you did this, you could use RSParser as starter code, since it parses OPML, RSS, Atom, JSON Feed, and RSS-in-JSON.)

This would make a great open source project, but it could just as well be free or for-pay.

* * *

One challenge is handling spam and abusive or hateful content. You’d most likely want to have a suggest-a-site form, so you don’t have to go out and find every single site there is.

But you have to be able to say no, and you have to be able to update the list of sites-to-crawl quickly if a site turns bad or inappropriate somehow.

You’d also need a way for users to report an issue.

(Again, the reduced scope — Apple-related blogs — makes this somewhat easier than if you tried to do the entire blogosphere.)

* * *

I want this! I’d use it every day. I know other people who would too.

But I’ve got enough on my plate that there’s no way I can do it myself — though I’d be happy to answer questions and provide feedback to anyone who does want to do it.

I have a vision for 1.0, and then, as time goes on, I have to cut it back beyond where it hurts. I have to keep punting. I hate this part of shipping software.

But I also love it because it reminds me that I have the stomach for it. Shipping software is an emotional skill.

After shipping — no matter what — there will be people who absolutely cannot believe that feature X wasn’t included. In fact, it’s the one thing they totally need.

And they’re right. Not wrong. And I would have loved to have included feature X, loved I not quality (the app maker’s honor) more.

* * *

I wondered if Evergreen as open source instead of as a for-pay app would affect what and how much I punt. I hope the answer is: not at all; the decisions would have been the same.

I think that’s right because I’m trying to make as good an app as I can, which would be true regardless.

But it is nice not to have to consider money as I make decisions. I do think about how I want it to have as many users as possible, and I want people to love the app. And I have to think about the economy of my time. But I don’t have to consider money.

Another difference between open source and commercial is that I can be utterly transparent about what’s been punted. Check out Evergreen’s 2.0 milestone. It will keep growing. Every single one of those things was originally supposed to go in 1.0.

* * *

Some things I keep reminding myself as I make progress toward 1.0…

This isn’t the last release, it’s the first. There will be many more. (I hope to work on this app for 20 years. It’s been only three years so far.)

Any feature I ship, I probably have to support forever. So it’s wise to be cautious.

Quality — design, stability, performance, lack of bugs — is way more important than any collection of features.

And, perhaps most importantly: shipping 1.0 means learning about your users, how they use your app, and what they need — and those lessons will change future plans, as they should. It’s best to learn those lessons early, before doing too much.

In this episode I finally get around to talking to Mark Boszko, the show’s intrepid producer and The Omni Group’s Video Producer.

I’ve known Mark for more than ten years — we met at a SXSW conference many years ago, long before either of us came to Omni. Mark didn’t even live in Seattle in those days. Now he does, and now we have the pleasure of working together on a podcast.

PS If you like movies, definitely check out the podcast Mark hosts: The Optical.

Mike Monteiro writes of Twitter CEO Jack Dorsey:

Jack let it happen. He watched as a once-entertaining, once-illuminating, once-vital network to global communication became a garbage fire of hate. He did nothing to stop it. Or curb it. He didn’t see a problem.

Our current crises of democracy and good faith did not just blow in with the wind and transform the air without our knowledge or consent.

These crises were made by people, and we knew what they were doing, and we agreed to this.

Jack Dorsey is one of those many people. Just one. But one with a kind of power that nobody in the world should have: the power to directly control a vast amount of the world’s communication.

It’s not that Dorsey failed to consider the good of the world. Or, really, it’s not just that. It’s that this kind of power should not exist at all.

But we agreed to it. We’re still agreeing to it.

Twitter — and Facebook, and the power of tech companies — is not our only problem.

But I have no doubt that had Twitter not become a loving home for hate, Trump would not be President now. In that universe we’d still have big problems, yes, but not like this.

How we can stop agreeing to this

The great social network is, or ought to be, the web itself.

The unruly web — unregulated and uncontrolled — is, perhaps paradoxically, the easiest place to limit hate. Not because we can stop people from publishing, but because we don’t have to live by Dorsey’s and Zuckerberg’s rules and designs.

I don’t know all the details of how we get there, or what it will be like once we do. That’s fine: that’s part of what makes the journey fun.

Software

Consider a few apps.

Overcast and Castro and others help ensure that podcasting is not just a vital and exciting medium of independent publishing but is also open and built on standards. Anybody can write any kind of podcasting software they want to — but nobody can control podcasting.

And nobody can force you to listen to hate. You pick the shows you want to hear.

MarsEdit lets you write whatever you want to write and publish it on the web. You’re limited only by the law and whatever terms of service your hosting provider may have.

All the words you read in MarsEdit are your own. And nobody can make you read what other MarsEdit users write.

Evergreen (which I’m working on), NetNewsWire, Reeder, Unread and other RSS readers work like Overcast and Castro but for written words. You choose what to read, and if a blogger you like suddenly turns hateful, you hit the Delete key.

Then there’s Manton’s new service, which needs its own section…

Micro.blog

It’s a publishing platform and a social network, based on standards.

You don’t even have to use Manton’s Mac or iOS apps: you can write posts in MarsEdit or other blog editor, or read your timeline using an RSS reader.

People could, though, sign up for it and flood your mentions with hate. In theory. So I asked Manton about that, and he wrote:

Micro.blog is similar to MarsEdit in a way in that it can be used to write hateful posts, etc. What we have to do as a social network is limit the damage. So, by default, if someone writes something terrible… No one sees it. It doesn’t automatically show up in trends (because we don’t have them, for this reason) and it doesn’t show up in Discover (because that’s curated by a human). Replies are where it’s an issue, and that’s where good tools and automatic flagging and reporting are needed.

In other words: the rules are different, and the easy exploits on Twitter are not so easy on Micro.blog. And there’s an actual committment to fighting this.

Manton also writes on his blog:

Imagine instead a service based on blogs, where the internal posts on the platform were the same format as the external posts. The curators of the platform would have more freedom to block harassing posts and ban nazis because those problematic users could always retreat to their own web site and leave everyone else in the community alone.

That’s how the web is supposed to work. It’s a core principle of Micro.blog.

(I’m @brentsimmons on Micro.blog, by the way. Here’s my microblog. I plan to post there more often than on Twitter in 2018.)

I should also mention…

Slack

I’m not sure where to put Slack in all this — except to say that admins control who’s in their groups, and I’ve never seen hate and harrassment there. I run a few groups, and I would have zero tolerance for this, and so would everybody I know who runs groups.

Well. One more thing about Slack: it meets some of the needs that Twitter used to meet. The talking-with-friends-and-family needs are very well covered there.

And the more we find ways outside of Twitter and Facebook to meet those needs, the less we’ll use Twitter and Facebook.

Maybe, back in 2012, Twitter did five important things for you that only Twitter did. I bet, in 2018, that that’s down to one or two.

Fun

The period from 1995-2008 (roughly speaking) was fun. It seemed like everybody was coming up with new things, and people were experimenting, and we were finding new joys in new connections, both human and technological.

Then, as Facebook and Twitter (and Google Reader; can’t forget that thing) grew, it’s as if we froze.

And those things were fun for a while, but they’re not now, and it’s obvious we made a mistake in allowing that much power to concentrate.

It’s time for the thaw: it’s time to get back to having fun. You’re free to make whatever you want.

What I’m Not Saying

Rebuilding the social open web is not the one cure that we need for all our ills. I’m fully skeptical of technological solutions to problems of culture and politics.

But it is an important thing we can and should do.

My small hope for 2018 is the knowledge that I’m not the only person thinking that way.

Related Reading

Me in 2011: What we talk about when we talk about RSS

Me in 2013: Why I love RSS and You Do Too

Anil Dash in 2012: The Web We Lost and Rebuilding the Web We Lost

IndieWeb is a thing I need to learn more about.

Tantek Çelik (video) - The once and future IndieWeb