inessential by Brent Simmons

December 2017

Evergreen’s Parser as Separate Open Source Framework

I copied Evergreen’s parsing framework, RSParser, to a separate repository on GitHub.

It has no dependencies other than system-supplied libraries. It’s offered via the MIT License.

It builds a Mac framework only at the moment, but adding an iOS target should be easy. It’s the one issue in the bug tracker (at least so far).

Otherwise it’s fast and stable and does the jobs it’s designed to do.

Things it parses

  • RSS
  • Atom
  • JSON Feed
  • RSS-in-JSON
  • OPML
  • Internet dates
  • HTML metadata
  • HTML links

In addition, you can build your own XML or HTML parser by creating an RSSAXParserDelegate or RSSAXHTMLParserDelegate.

It’s a mix of Objective-C and Swift. More recent code is in Swift.

Parsing a feed that’s RSS, Atom, JSON Feed, or RSS-in-JSON is a matter of calling FeedParser.parse and getting back a ParsedFeed object. Pretty simple.

WKWebView Workarounds

In Evergreen I’m using WKWebView instead of WebView, because it’s the new and improved WebKit view. NSHipster writes:

Boasting responsive 60fps scrolling, built-in gestures, streamlined communication between app and webpage, and the same JavaScript engine as Safari, WKWebView is one of the most significant announcements to come out of WWDC 2014.

Sounds great! I’m on board.

One of the best parts about it — easily enough to sell me — is that a crash in WKWebView won’t crash my app.

(Back when I was doing NetNewsWire, crashes in the old WebView — the predecessor to WKWebView — were the most common crash. But, to be fair, many if not most of those were actually Flash crashes.)

I’ve also heard, though I’m not sure how to verify, that WKWebView is better for accessibility, too. So it’s a win all around.

But it’s not a win all around

WebView — good ’ol trusty friend — has a bunch of things that WKWebView is missing.

The new web view has no built-in support for finding text, for instance. I’m not sure what I’m going to do about this, since the ability to hit cmd-F and look for some text is a pretty fundamental thing, and I can’t skip it.

It also has no delegate method for when you mouse over a link. Seems like another fundamental thing, right? Any browser offers you a status bar or some way to see the URL of the link your mouse is over.

What I ended up doing there: my article renderer adds some JavaScript that adds event listeners. (See ArticleRenderer in the renderedHTML() method.) Then I added handlers in DetailViewController: see viewDidLoad and the WKScriptMessageHandler extension.

This worked fine: now my status bar shows the URL of the link your mouse is over. Cool.

Then the issue of scrolling came up

Evergreen’s design includes a key feature: you can go through all of your news just by hitting the space bar repeatedly.

If there’s more to scroll in the web view (the article), it scrolls. If not, then it goes to the next unread article.

This is a coffee-in-hand feature, which is critical for a news reader.

But WKWebView provides no access to its scroll view. (It does on iOS, yes, but not on Macs.)

The logic goes like this:

if canScrollDown()
	scrollDown()
else
	goToNextUnread()

There are two parts to this, and both (I thought) required access to the scroll view.

canScrollDown()

Well… JavaScript comes to the rescue again, just as with the mouseover-link handling. I can run a script in the app and get the results — and JavaScript can tell me the content height and the current amount of scrolling. (I already know the frame size, but I could have gotten that via JavaScript also.)

I did this an DetailViewController, in the fetchScrollInfo method.

(Unfortunately, this is asynchronous. I expect it to be super-fast, but I won’t know if this is a problem without a bunch of testing. Let’s set that issue aside.)

The fetchScrollInfo method creates a ScrollInfo struct (at the bottom of DetailViewController) that has canScrollDown and canScrollUp properties.

So that’s how I know if the web view can scroll. Part one of two solved.

scrollDown()

I tried making it scroll via JavaScript. It’s super-easy. But it’s not animated, and it needs to be. (In Safari, with a scrollable page, hit the space bar — notice how it animates.)

So I did some research and found some JavaScript utilities, some needing JQuery, for animated scrolling.

But do I really want to add that much JavaScript on every single article load? No way. And will their animation match the animation users expect? I strongly suspect it would be noticeably different.

(I write Mac apps. “Noticeably different” is the kiss of death.)

So I did some nerd-sniping, on Twitter and Slack, to see if anybody had a suggestion that would work.

People had ideas. Nothing worked.

Twist ending

So I’m frustrated, and I’m just about to switch back to my old friendly friend WebView, because I know it will do what I want.

The voice of another old friend — the guy who mentored me in my early days as a programmer — came into my head, as it often does in these situations.

“What is the real problem you’re trying to solve?” asks Dave Winer, in my head.

The real problem is not arbitrary animated scrolling. The real problem is that I want to do a page-down exactly the same way it would work if the WKWebView had focus and you hit the space bar. Exactly the same animation and scroll distance.

But not arbitrary scrolling: just a page-down.

So I think maybe I can spy on the event stream; maybe I can figure it out by seeing what happens when I click in the web view and hit the space bar. Maybe I can post an event that will do the job.

To do so I make an NSApplication subclass, which I do (in Swift, of course), and break on sendEvent.

But of course this doesn’t work. The app crashes immediately saying it can’t find the EvergreenApplication class. (Which is right there! Dude!)

Whatever.

Okay, maybe it’s back to WebView after all. Forget WKWebView.

Then I do a search in Xcode on pagedown, just out of a last gasp of trying — and, forgotten long ago by me, but look: there is a scrollPageDown method in NSResponder — and WKWebView is an NSView subclass which is an NSResponder subclass — so, could it be? maybe? it might work?

I tried it: sure enough! It works! With animation and everything. It’s perfect.

* * *

So, in the end, the action method is in MainWindowController — see scrollOrGoToNextUnread.

(It’s a little weird due to the async thing I mentioned earlier. But it does the job.)

And now — after three years of working on this app — I was finally able to go through all my news just by hitting the space bar. Big day for me.

Evergreen Diary #2: Random Notes

I’ve been working on Evergreen for about three years, and so I considered writing about playing such a long game — but then Daniel released MarsEdit 4.0 after seven years.

I tip my fedora to the master.

* * *

But there is at least one difference: it’s taking so many years just to get to one-point-oh. I’ve never been on any kind of project that took so long just to get to the initial release.

The app’s been built from the bottom up. Since I knew in advance what I needed, I could write code that wouldn’t get actually used for years.

Here’s a small example, written almost two years ago: RSHTMLMetadata.h.

The scoop: it always bugged me that in NetNewsWire, when it was mine, the favicon downloading system never checked the metadata in a feed’s home page to find the favicon.

It always just appended /favicon.ico — which wasn’t always correct. So it would get the wrong one or just not find it.

(Note: it’s entirely likely that the current version of NetNewsWire has fixed this bug.)

So, a year-and-a-half ago, I wrote code in RSHTMLMetadata that pulls the favicon URL from web page data.

And then, just last month, when I finally got around to writing the favicon downloading system, it took very little code to actually pull the favicon URL from a web page (in FaviconURLFinder.swift):

let htmlMetadata = RSHTMLMetadataParser.​htmlMetadata​(with: parserData)
return htmlMetadata.​faviconLink

This pattern — ground-up development, where you know what you need in advance — isn’t really different from apps developed much more quickly.

The difference is the amount of satisfaction I feel when I finally connect old and new pieces. It feels great.

And that’s the position I’m in now, now that I’m at the top layer: code that I wrote a long time ago is finally getting used. The puzzle is coming together.

* * *

Of course, I didn’t, and couldn’t, know everything in advance. I didn’t know I’d want Open Graph and Twitter images from web page data — but I was glad when I found that the code I’d already written was easy to extend.

* * *

Since I have no commercial interest in Evergreen — since it’s free and open source — I can take all the time I need. One of the many advantages of this is spending more time writing unit tests than I used to.

There aren’t enough, yet — not even close; and there never are enough — but they do exist. Here, for example, are the tests for HTML metadata parsing.

And when I find a bug I add a new test: for example, the feed type detector was not detecting Natasha the Robot’s feed as RSS. So I fixed the bug and added a test.

* * *

From the now-it-can-be-told department… a couple years ago, in November 2015, I collected a list of blogs written by women that would be “of interest to Mac/iOS developers, designers, and power users.”

This was for Evergreen. When I was working on NetNewsWire, the default feeds were all, or almost all, written by men. For Evergreen I wanted to fix that, and I also wanted to create a larger feed directory that was inclusive. (Here’s the plist for the directory. It still needs a bunch of work.)

If you have feeds to suggest, please do a pull request. Or send me a note on Twitter via @evergreen_mac.

* * *

I just started working on syncing via Feedbin. It’s what I use, and syncing is necessary. It’s likely that 1.0 will ship with just Feedbin syncing, though, and I’ll add other systems in follow-up releases.

Also: I just got a new iPad Mini which travels to and from work with me. I didn’t expect to love this device so much! So now I’m thinking of doing Evergreen for iOS after all. (Also open source, as part of the same project.)

* * *

It gratifies me to see my friends working on open web stuff. Dave Winer, of course, is always on the ball. Daniel just released MarsEdit; Manton is working on Micro.blog.

This is a political act, and, I think, fundamentally conservative: it wants to preserve what’s great about the web, and it recognizes that concentrations of power are bad things.

Power should be distributed to the individuals, not hoarded by large companies and governments that have their own interests in mind, which match ours purely by coincidence from time to time.

Next year may be a horrible year for a whole bunch of reasons, but it may also be a year where the open web fights back. Evergreen wants to help.

The Omni Show episode #4 with Andrea McVittie, Omni Slack Group

Yesterday we published an episode with Andrea McVittie, User Experience Designer at The Omni Group.

Andrea will brook nae interference with the puppies. Be nice!

She also talks a bit about how design works at Omni and about design and ethics. After we posted the episode, she followed up with a tweet where she codifies her ethical guidelines.

* * *

Recently Omni created a Slack group — you can join up.

It’s not intended as a replacement for support. But you can talk with Omni people and with other people who use Omni apps. It’s not super-busy; it’s, well, nice. I like having the people who use the things we work on so nearby.