inessential by Brent Simmons

Apple Privacy Changes

I was actually surprised at the changes Apple is making to stop tracking. It’s not enough to stop tracking on the web, and it’s not enough to stop tracking in iOS apps — which is happening probably way more than you think it is — so Apple did both.

While I know that Apple takes privacy seriously in a way other large tech companies don’t, I still didn’t expect them to go this far. I’m glad they did.

My pet theory is that this set of changes is the most important thing to come from WWDC this year. These privacy changes will, I think, have far more impact on the tech industry, on society, and on our lives, than SwiftUI or a new processor for Macs or anything like that. (As fun as those things are.)

It’s always going to be an arms race, I suppose — see this press release from Kochava:

Options exist to perform identity resolution using hashed-email-to-device linkages, device connections by household, and other first-party identifiers key in solving for identity resolution and attribution.

[…]

Further scarcity of the IDFA forces greater reliance on attribution by fingerprinting. Fingerprinting is a probabilistic method of attribution based on device IP & user agent that’s less precise than deterministic attribution based on the globally unique IDFA. Nonetheless, a high degree of accuracy is still maintainable with fingerprinting…

IDFA stands for “identifier for advertisers.” One of the changes Apple is making is that when an iOS app asks for the IDFA, the system will ask the user to consent to being tracked. When consent is not given, the IDFA will just be a string of zeros.

It’s self-evident that pretty much everyone will say no, which makes the IDFA useless. App makers will want to avoid even the shame of asking for consent.

Kochava is saying — and I’m betting they’re not the only company saying this — that they’ll find a way around the IDFApocalypse to identify users. They will probably succeed, too, at least to a certain degree.

However, Apple has shown that it has a mandate to fight, and the will, and it doesn’t mind dropping down some very large technical hammers to protect our privacy.

* * *

It’s important to note — before people get stigmatized unfairly — that most of the tracking and metrics collected by various websites and apps is done so with innocent motives. Marketers want to know which campaigns are more effective; they want to get the most bang for their buck. Product designers want to know which features are more popular; they want to know what’s working for people and what isn’t. Publishers want to know which pages people visit and how they got there. Engineers — like me — want to be warned of potential problems.

There’s nothing wrong with wanting those things! The people who want those things aren’t trying to snoop on people or anything — they’re using data to do their jobs better.

The problem is that the tech industry, in order to serve these needs, did what it always does: code up the thing, take the biggest bite it can, and hope to make enough money, and amass enough power, to be able to repel any future ethical distractions. So now we have mass surveillance.

But Apple recognizes that there’s still a need to know, for instance, which of your ad campaigns is doing best — and so there’s SKAdNetwork, which is a thing I don’t totally understand yet, but I get that it answers marketing questions in the aggregate (which is all marketers should want) and doesn’t violate privacy.

I like that Apple knows that it’s not enough to just shut down the bad actors — people who have questions to answer, but who have no interest in violating privacy, need solutions.

PS See the WWDC 2020 video Build trust through better privacy for way more about all this.

I Had Been Worried About the Mac

I spent the month or so before WWDC like you — suffering through a pandemic, outraged by violent racism, worried about democracy. Heartsick and appalled, mad and sad.

Nothing has changed since WWDC, either. Except for one thing. A small thing in comparison, but important to me — I had been very worried that Apple would, as part of the ARM transition, lock down macOS so that only Mac App Store apps would be permitted.

That didn’t happen. And Apple employees explained that it’s not going to happen — and, given that it didn’t happen this time, given that they had this chance, I believe them.

I understand adding security features to the Mac. But to take away our freedom to create whatever Mac apps we want, and distribute them without Apple’s, or anyone’s, seal of approval, would be to take the heart out of my career.

But that’s not what happened! I feel great about this. I’m going to stop worrying about the Mac.

The New Way of Making Apps

I’m excited about the new features in SwiftUI this year. This reminds me of the early 2000s when I switched from writing Mac Toolbox apps to Cocoa apps. It was a whole new way of writing apps, and it was so much better.

I jumped right on it, back then — and I feel no less enthusiastic for this new new thing than I did for Cocoa almost 20 years ago.

Apple has essentially said, I believe, that the way we’ve been making apps is all legacy. AppKit and UIKit both. SwiftUI is the future.

NetNewsWire and SwiftUI

We re-jiggered the NetNewsWire roadmap somewhat.

  • Mac/iOS 5.0.x: bug fix releases
  • Mac 5.1: Feedly, feature parity with iOS
  • Mac/iOS 5.5: iCloud sync, other integrations and features
  • Mac/iOS 6.0: SwiftUI, features tbd

The thing to call out here is NetNewsWire 6.0. We’re already at work building a SwiftUI app where Mac and iOS share as much UI code as possible.

The work is going very quickly: I’m amazed. If you want to follow along, or even help, take a look at the swiftui branch.

I’m super-psyched for this. If it means Mac and iOS can share most of their code, and we can add features more quickly (because SwiftUI makes for so much faster development), then we can ship more and better versions of NetNewsWire more often. I want that!

Reporting in After More Than a Month at Audible

I’ve been reluctant to write about how my new job is going — I don’t want to look like the guy who drank the kool-aid, and I certainly don’t want to be the guy who couldn’t read the room during our new multi-crisis normal.

But, maybe, some good news, even if for just one fortunate person, is okay to write about? I’m not even sure. But some of my friends have suggested I write it up, so I am.

* * *

Anyway. It’s going well! I love the job and the people and what we do.

Telling stories by way of human voice is among the most elemental and powerful of arts, and I believe that stories transform lives. My work at Audible is motivated by the same thing in me that makes me make NetNewsWire (an RSS reader), that made me create MarsEdit (a blog editor), that makes me write this blog.

Audible acts like a company with a mission. It seems like every company claims solidarity and support these days, and most of these claims are shallow and opportunistic. But Audible is committed to revitalizing Newark, NJ — from hiring locally, to Newark Working Kitchens, to Newark Venture Partners, and plenty more — and it’s helping, for real. This is not some new face for the current moment: it’s part of the company’s DNA and history.

And if you read the Audible blog, you’ll find that the company is dedicated to bringing us the stories that need telling and that urgently need to be heard.

It’s a good place that’s doing good, and I am proud to work there.

* * *

I’m on iOS. I’m senior enough not to be embedded in a scrum team, but I’m an individual contributor, not a manager. My job, broadly speaking, is to help the team increase velocity and quality. (My job isn’t strictly limited to iOS, but that’s where my focus is.)

My first month was spent meeting people (over video; via Amazon Chime) and learning things. The largest company I’ve ever worked at had about 100 people: Audible is much larger — 20 times larger? I’m totally just guessing — and that means I’ve had to learn about the ways of large companies. (Also remember that Amazon is part of this, usually in the background.)

I’m starting to be able to contribute a little — just recently I committed my first code. In any given month I might be writing a ton of code, or hardly any, or somewhere in between. While writing code is important, my job is more about things like architecture and best practices — it’s about finding ways to make the team better.

My background leading the NetNewsWire open source project is very relevant here. I learned, while running the NetNewsWire project, that people will rally to a higher standard if you can show them that it’s possible to reach it and then lead them there.

During my first month I felt like a detective from an Agatha Christie book, interviewing people and taking notes — What happened? How did we get here? What the heck is an ASIN? Those were the easy things to learn, and the hard lessons, where I learn how to take my experience and help lead us to that higher standard, are to come.

But that’s also the challenge! And the fun. It’s why I signed up.

* * *

I love this job every day except when I have to get up early due to time zone issues. Sheesh! (This happens just once or twice a month, seems like, so it’s not at all bad. It’s fine. But I Am Not a Morning Person.)

* * *

I haven’t noticed that the people I work with have a lot of public social media presence. (Maybe I just haven’t gotten clued-in yet?) But here’s Jeff Merola, the engineer I work most closely with. He’s smarter than I am, which is wonderful.

Accessibility and the Dynamic Nature of Objective-C

Doug Russell, who used to work on accessibility at Apple, writes:

some of the code that powers accessibility on apple platforms is just disgusting to look at and to work on.

most of the code that makes apple software accessible lives in what’s called an accessibility bundle. without diving into the minutia of the thing, bundles are a way to load something akin to a plugin into a cocoa app at runtime if an assistive technology is activated. it involves manipulating the app or framework class hierarchy and using objective-c dynamism to read app state and build up a usable accessibility hierarchy. insert a super class here, read an instance variable there, swizzle in a method and store the state for it in associated objects.

In other words — Objective-C and its runtime play a big role in making Apple’s great accessibility possible.

What happens when that’s not really a thing anymore?

Work at Universe

When I was looking for a job, I talked with the folks at Universe a few times. I love what they’re doing — an iOS app that helps people make websites — and I really enjoyed talking with the team. Such a great bunch.

The good news is: they’re still hiring. They have a bunch of jobs, even — iOS, Swift backend, database, product design, marketing, and support. Check ’em out!

PS Here are their key values.

WWDC 2020 and NetNewsWire

I love seeing so much attention paid to the Mac this year!

I’ve applied for a Developer Transition Kit for NetNewsWire. My thinking: since NetNewsWire is open source, other developers can, and do, look at the code to help them write Mac apps. The sooner we have NetNewsWire updated, the sooner it’s available as an example for other developers.

Other thoughts…

The new Mac operating system, Big Sur, big number 11, Onze-y-baby, has some appearance and behavior changes which of course we’ll adopt. One of NetNewsWire’s values has always been to stick pretty close to Apple’s design for the platform. We do that because, well, we figure users of a given platform actually like the platform design, and that’s why they picked it. (It also tends to mean less work, which is a good thing.)

We’ll not be switching to Catalyst. It appears to be much-improved, but standards for a good Mac app are high, and I’m skeptical that Catalyst is all the way there yet.

Instead, our plan is to converge our UI code over time by using SwiftUI. This way we can go view-by-view. (It’s worth noting that we already do share some UI code: the article view is mostly shared, for instance, even without using SwiftUI or Catalyst.)

I’m looking forward to the rest of the week. I especially want to hear more about the new outline view in SwiftUI. 🐣🐥

The App Store Doesn’t Make Apps Safe

Another misconception about the App Store is that it makes apps secure and safe. It doesn’t.

There are things that do make apps safe. No matter how an iOS app is distributed, it runs in a sandbox. An app requires permission from the user to do things like access the address book or microphone. This is just how iOS works: it has nothing to do with the App Store.

The App Store review process probably does run some kind of automated check on the app to make sure it’s not using private APIs and doesn’t contain some kind of malware. However, this could be run as part of a notarization process — this doesn’t have to be tied to the App Store. (Mac apps outside of the Mac App Store go through a notarization process.)

Otherwise, App Store review is looking for basic functionality and making sure the app follows the guidelines.

As far as checking that an app doesn’t crash on launch — thanks? I guess? As for following the guidelines: the guidelines are about protecting Apple’s interests and not about consumers.

I would like to say that the App Store filters out bad behavior, but I don’t think it does. We’ve all seen various scam apps, and we’ve seen otherwise well-behaved apps do things like abuse the push notifications system.

It probably catches some egregious scams that we never hear about. I’ll apply the benefit of the doubt. But it didn’t catch that, for instance, Path was uploading the user’s address book. The community outside Apple catches these things, and Apple changes how iOS works so that these things can’t happen without user permission.

And, at the same time, the App Store is a magnet for scam apps. Even in a world where side-loading is possible, scam apps would stick to the App Store because that’s their best shot at getting users to stumble across them.

My grandmother

People have asked if I’d want my grandmother to download iOS apps outside the App Store. The answer is yes. That was how she downloaded her Mac apps, after all. (She was an avid Mac user.)

I’d feel secure knowing that the apps, just by virtue of being iOS apps, are sandboxed and have to ask for permissions. (I’m also imagining a Mac-like notarization step, for additional security. I think this is reasonable.)

In other words: Apple has done a very good job with iOS app security and safety. The fact that we think this has something to do with the App Store is a trick, though.

(I’m not arguing for getting rid of the App Store, by the way. I’m arguing for allowing an alternative.)

The iOS App Store Brings Users Only Because It’s the Only Choice

One might argue that developers should love the App Store because it brings the users.

AppleInsider writes about the App Store, Hey app, and David Heinemeier Hansson:

Like any other product or service, Hey has to persuade people that they have a problem it can solve, and that it’s worth paying for. You can’t persuade people of anything, though, if they don’t know about it. And then if you do persuade them, you can’t profit without a way to get your product into their hands.

His first argument against the App Store on Apple’s cut got Hansson and Hey a lot more notice than it might have. But it’s the App Store that gets his product to people. It’s the App Store that means if he persuades people it’s worth it, they can instantly have it on their iOS device.

This is a misconception that many people have — they think the App Store brings some kind of exceptional distribution and marketing that developers wouldn’t have on their own.

It’s just not true. It lacks even a grain of truth.

Setting up distribution of an app is easy and cheap. I do it for NetNewsWire for Mac with no additional costs beyond what I already pay to host this blog. This was true in 2005 as much as now — distribution is not some exceptional value the App Store provides.

And then there’s marketing. Sure, being featured used to mean something to revenue, but it hasn’t meant that much beyond just ego points in years. To be on the App Store is to be lost within an enormous sea of floating junk. No matter how well you do at your app description and screenshots — even if you get some kind of feature — your app will not be found by many people.

Build it (and upload it to the App Store) and they will not come.

Instead, you have to do marketing on your own, on the web and on social media, outside of the App Store. Just like always. The App Store brings nothing to the table.

So while it’s true to say that all of an iOS app’s users come via the App Store, it’s only true because there’s no other option.

If I could distribute my iOS app outside of the App Store, I would. I’d switch in a heartbeat. Even though it’s free and money isn’t my issue. It would make my work as an app maker easier.

I can’t reconcile in my mind the tension between Apple as the think different company, the pirates, the rebels, the company at the intersection of tech and liberal arts — and Apple the company that runs this legalistic, nitpicky, greedy, inhuman, happy-face Kafka App Store.

One Advantage of the App Store That’s Gone

The best part of the App Store, years ago, from this developer’s point of view, was that it was easy to charge money for an app. No need to set up a system — just choose the price, and Apple takes care of everything. So easy!

But these days, in almost all cases, you’d be ill-advised to charge up front for your app. You need a trial version and in-app purchasing (IAP) and maybe a subscription.

Here’s the thing: this is a massive pain in the ass to implement, test, and support — Apple does not make it easy. It could, I think, make certain common patterns basically turn-key (like trial versions + IAP), but it hasn’t.

This means that, for many developers, the very best thing about the App Store — the thing that actually helped their business — is gone.

And it’s not just gone — it’s probably actually more difficult doing this stuff via the App Store than doing the same things (trial, IAP, subscription) using non-Apple systems such as Stripe.

(And, as a bonus, Stripe isn’t going to review your app’s business model and tell you no.)

Why NetNewsWire Is Fast

NetNewsWire is fast because performance is one of our core values. Being fast is part of the very definition of the app.

I suspect that it’s hard to do this any other way. If you take a month or two to speed things up, from time to time, your app will always be — at best — just kind of heading toward satisfactory, but never to arrive.

The best general advice I can give is just this: make sure performance is part of the foundation of your app. Make sure it‘s part of every decision every day.

Make sure, in other words, that performance isn’t just a topping — it’s the pizza.

Below are some of the specific reasons NetNewsWire is fast. Because NetNewsWire is — like many apps these days — basically a fancy database browser where data comes from the web, some of these will apply to other apps.

The below items are in no particular order.

Fast RSS and Atom Parsing

The most painful way to parse XML is with a SAX parser — but it’s also how you’ll get the best performance and use the least memory. So we use SAX in our RSParser framework.

On my 2012 iMac, parsing a local copy of some past instance of the Daring Fireball Atom feed — relatively large at 112K in size — happens in 0.009 seconds.

That’s fast, but we do another thing as well: run the parser in the background on a serial queue. Since parsing is a self-contained operation — we input some data and get back objects — there are no threading issues.

Conditional GET and Content Hashes

The parsers are fast — but we also do our best to skip parsing entirely when we can. There are two ways we do that.

We use conditional GET, which gives the server the chance to respond with a 304 Not Modified, and no content, when a feed hasn’t changed since the last time we asked for it. We skip parsing in this case, obviously.

We also create a hash of the raw feed content whenever we download a feed. If the hash matches the hash from the last time, then we know the content hasn’t been modified, and we skip parsing.

Serial Queues

The parser isn’t the only code we run on a serial queue. When an operation can be made self-contained — when it can just do a thing and then call back to the main thread, without threading issues — we use a serial queue if there’s any chance it could noticeably block the main thread.

The key is, of course, making sure your operations are in fact self-contained. They shouldn’t trigger KVO or other kinds of notifications as they do their work.

(A simple example of a background thing, besides feed parsing, is creating thumbnails of feed icons.)

We Avoid the Single-Change-Plus-Notifications Trap

Here’s an example of a trap that’s easy to fall into. Say a user is marking an article as read. Calling article.read = true triggers, via KVO or notifications or something, things like database updates, user interface updates, unread count updating, undo stack maintenance, etc.

Now say you’re marking all articles in the current timeline as read. You could call article.read = true for each article — and, for each article, trigger a whole bunch of work. This can be very, very slow.

We have specific APIs for actions like this, and those APIs expect a collection of objects. The same API that marks a single article as read is used to mark 10,000 articles as read. This way the database is updated once, the unread counts are updated once, and we push just one action on the undo stack.

Coalescing

We also try to coalesce other kinds of work. For instance, during a refresh, the app could recalculate the unread count on every single change — but this could mean a ton of work.

So, instead, we coalesce these — we make it so that recalculating unread counts happens not more often than once every 0.25 seconds (for instance). This can make a huge difference.

Custom Database

For an app that is, again, just a fancy database browser, this is where the whole thing can be won or lost.

While Core Data is great, we use SQLite more directly, via FMDB, because this gives us the ability to treat our database as a database. We can optimize our schema, indexes, and queries in ways that are outside the scope of Core Data. (Remember that Core Data manages a graph of objects: it’s not a database.)

We use various tools — such as EXPLAIN QUERY PLAN — to make sure we’ve made fetching, counting, and updating fast and efficient.

We do our own caching. We run the database on a serial queue so we don’t block the main thread. We use structs instead of classes, as much as possible, for model objects. (Not sure that matters to performance: we just happen to like structs.)

To make searching fast, we use SQLite’s Full Text Search extension.

I could, and probably should, write more articles going into details here. The database work, more than anything else, is why NetNewsWire is fast.

Sets and Dictionaries

We often need to look up things — a feed, given its feedID, for instance — and so we use dictionaries frequently. This is quite common in Mac and iOS programming.

What I suspect is less common is use of sets. The set is our default collection type — we never want to check to see if an array contains something, and we never want to deal with duplicate objects. These can be performance-killers.

We use arrays when some API requires an array or when we need an ordered collection (usually for the UI).

Profiler

Instead of guessing at what’s slow, we use the profiler in Instruments to find out exactly what’s slow.

The profiler is often surprising! Here’s one thing we found that we didn’t expect: hashing some of our objects was, at one point, pretty slow.

Because we use sets quite a lot, there’s a whole lot of hashing going on. We were using synthesized equality and hashability on some objects with lots of string properties — and, it turns out, hashing strings is pretty darn slow.

So, instead, we wrote our own hash function for these objects. In many cases we could hash just one string property — an article ID, for instance — instead of five or ten or more.

No Stack Views

My experience with stack views tells me that they’re excruciatingly slow. They’re just not allowed.

No Auto Layout in Table Cell Views

When people praise a timeline-based app like NetNewsWire, they often say something like “It scrolls like butter!” (I imagine butter as not actually scrolling well at all, but, yes, I get that butter is smooth.)

While we use Auto Layout plenty — it’s cool, and we like it — we don’t allow it inside table cell views. Instead, we write our own layout code.

This is not actually difficult. Maybe a little tedious, but laying out a table cell view is pretty easy, really.

I figure that optimized manual layout code is always going to be faster than a constraint solver, and that gives us an edge in smooth scrolling — and this is one of the places where an otherwise good app can fall on its face.

And: because that layout code doesn’t need a view (just an article object and a width), we can run it at any time. We use that same code to determine the height of rows without having to run an Auto Layout pass.

Caching String Sizes

Text measurement is slow — slow enough to make even manual layout too slow. In NetNewsWire we do some smart things with caching text measurement.

For example: if we know that a given string is 20pts tall when the available width is 100 and when the available width is 200, we can tell, without measuring, that it will be 20pts tall when the available width is 150.

Summary

There’s no silver bullet. Making an app fast means doing a bunch of different things — and it means paying attention to performance continuously. 🍕

Focusing

Tomorrow’s the first day at my new job. Exciting!

Starting a new job has led me to look at my entire list of responsibilities — which is too long — and figure out what I need to drop so that I can pay enough attention to the projects that need it most.

My most important projects (outside of my job) are NetNewsWire and this blog. This blog, because, well, blogging is part of how I breathe. And NetNewsWire because I love the app — and it’s a real thing in the world now, with users, a team of developers, and great features coming up.

I wanted to do another half-dozen or so apps alongside NetNewsWire, starting with Rainier, but I’m dropping development on those so I can concentrate entirely on NetNewsWire. This is personally disappointing, but it’s honest: I just don’t have time for Rainier and these other apps. Work on these would take away from NetNewsWire, and that would be wrong.

Another move I’m making: Manton Reece has agreed to take over the repo and website for JSON Feed. I’ve been the bottleneck here with a 1.1 version, and I shouldn’t be. Manton will take care of this way better than I’ve been able to. (I hope to get everything transferred over to Manton in the next few weeks.)

My New Job

As of this morning the ink is all dry, and I can happily report that my new job is at Audible. I’ll be an architect on the mobile team.

I’m very excited for this job! It’s perfect for me in so many ways — not least that it’s about books.

My plan is for this to be my last job — I plan to work at Audible until I retire. I start Monday. 🐣🐥🕶

Archive