December 2016

I first learned protocol-oriented-programming with Objective-C, and I was very pleased to see the Swift team emphasize this style.

But, at least at this writing at the end of 2016, I still run into problems when I use this style of programming in Swift.

Here’s the problem I’m trying to solve:

Variant Value

I’m working on a schema-less hierarchical database. Tables can contain tables, and they can contain values such as strings, numbers, booleans, dates, arrays, and so on.

To represent these values, I’d rather use a Value protocol rather than a Value class. If it’s a protocol, then I can have completely separate implementations: BoolValue and DateValue and ArrayValue and so on would conform to Value, but otherwise would have different implementations.

But here’s the thing: ArrayValue needs to have an actual Swift array of type [Value].

So let’s say you want to add something to that Swift array, where arrayValue is of [Value] type. Let’s say you’ve created trueValue, which is a BoolValue, and BoolValue conforms to the Value protocol.

arrayValue += [trueValue]

You can’t. Because even though BoolValue conforms to the Value protocol, this isn’t handled automatically. The compiler knows all the types involved, but it still won’t let you do this unless you explicitly cast. The following works:

arrayValue += [trueValue as Value]

In Objective-C, no casting is required:

[arrayValue addObject:trueValue];

This may seem like nitpicking on my part, but that added bit of housekeeping makes protocol-oriented-programming in Swift feel a little bit unnatural.

A natural form of protocol-oriented-programming might start with this premise: any time the protocol type is called-for, you can use an object that conforms to that protocol.

More complexity, to push the point

Let’s say you have two arrays:

let array1 = [trueValue as Value]
let array2 = [trueValue]

You and I both know that those arrays contain just one object, and it’s an identical object. Those two arrays are absolutely equal.

But you can’t compare them:

array1 == array2 will not compile, because array1 is of type [Value] and array2 is of type [BoolValue]. Even though BoolValue conforms to Value, this won’t work.

In Objective-C, you’d just use isEqual:, and it would work as expected.

But it gets worse:

let array1 = [trueValue as Value]
let array2 = [trueValue as Value]

Okay: now you know those two arrays couldn’t possibly be more equal.

But array1 == array2 still won’t compile, and you’ll get the following error:

binary operator '==' cannot be applied to two '[Value]' operands

In Objective-C, you’d write:

NSArray \*array1 = @[trueValue];
NSArray \*array2 = @[trueValue];

And [array1 isEqual:array2] will return YES.

(You could type those arrays using lightweight generics, and the outcome is the same: it works.)

So: a second premise for natural protocol-oriented-programming might be: a protocol-conforming object should have the same features as other objects — for instance, you should be able to make it Equatable (the equivalent of, in Objective-C, responding to isEqual:).

Orange leader. Dominance and narcissism and corruption and bullshit and lies. Families breaking up. The claim of making the world great masking a business empire.

See Leah Remini: Scientology and the Aftermath. It’s riveting.

And I can’t help but feel like, in a way, all of America just joined Scientology. (Not literally, of course.)

* * *

The Man in the High Castle has long been my favorite Philip K. Dick novel, and one of my favorite novels ever.

(My Mom pushed it on me when I was a teenager. I’ve been a Dickhead ever since.)

In it, the Germans and Japanese won the second World War (or did they?) and America is split up and occupied — by the Japanese empire on the west and the Reich to the east.

I’m five episodes into the TV show. It doesn’t faithfully track the novel — it’s much more of a thriller than I remember from the book. Which is fine: the novel is a novel, and the TV show is a TV show.

And I can hardly pull myself away, not least because it also speaks to the current moment.

Damn I’m slow.

It’s been a thing for years where Republicans talk about the “elites,” and I laugh — because the Republican party, the party for the super-rich and mega-rich (and the generals, it seems) is clearly the party of the elites.

I honestly didn’t know what they meant. I know who they meant — they meant people who make TV shows and movies, write books, sing songs, improve public schools, or who just lead nice lives in the cities and vote for Democrats. (That last one includes me.)

But here’s what they mean, specifically, I think: the “elites” are the people who will, directly or implicitly, tell you you’re a racist. Or sexist, or homophobe, or Islamophobe, or some form of bigot.

And since calling you a bigot has become, somehow, the very worst thing you can do in America, the “elites” are the very worst people.

The “elites,” just by making Modern Family and calling for health insurance even for people who aren’t white, are somehow telling these Republicans that they’re bad people. I get it now. (Maybe I’m the last person to understand this!)

Tough shit.

If 2016 was a tough year for a number of reasons, it was tough especially because we agreed to hand a machine gun to a monkey.

So we can all agree that we’d like to put 2016 behind us. Good riddance to a bad year.

The problem is that 2017 is the year that the monkey pulls the trigger.

Vesper for iOS is up on GitHub.

It’s presented as a historical artifact rather than as a living project. It’s definitely not an example of how to write apps these days — and it’s not even an example of how to write apps in 2013.

There are good parts and bad parts, embarrassing parts, parts that clearly need refactoring, etc. etc.

The only changes from the code in our current repository were to remove the incomplete Mac app code, remove Hockey, remove Ideal Sans (the embedded font), make a few code changes related to switching to the system font, and get it to compile with Xcode 8.2.1.

Surprisingly Big

It’s been a while since I lived in this code and, coming back to it, I was struck by how very much code there is. It’s a little nuts. This is just a note-taking app, after all.

If you remember the context, though, it’s perhaps not that surprising.

It was written while iOS 6 was current, and it still looks like an iOS 6 app under the hood. But, at the same time, we were anticipating iOS 7, and so Vesper was an art project — we wanted Vesper to join Letterpress and Twitterrific and a few others as one of the first Modernist apps.

But we hadn’t actually seen iOS 7, and so we invented Vesper’s look and feel from scratch, though with some idea of where the puck was heading. That — combined with wanting to use Ideal Sans everywhere, even in standard things like alerts — meant we had to do a ton of custom UI and animations.

It’s interesting to me that 2013 was about the last time you could plausibly think that that’s the right thing to do. It’s clearly too expensive now — and was too expensive then, too, but we hadn’t realized it yet.

The irony is that we thought Vesper was one of the first apps of a new era — the era that officially kicked-off with iOS 7 — but, in the end, it was one of the last apps of the era where it was not uncommon for developers to spend massive amounts of time in UI invention.

It’s a different era now. I don’t mean that critically — iOS 7 and the follow-ups have brought iOS closer to where the Mac has been all these decades, where you have standard system UI that’s good and that doesn’t require much customization. That’s a good thing.

Were we to write Vesper now, based on that same initial idea (drag to reorder plus tags), we’d do it in far less code. (I’d guess it would take about a third of the code.) But would we still use Ideal Sans? I bet we would. What a lovely font. I regret that we couldn’t provide it in the open source version.

Forking

You’re welcome to do whatever you want with Vesper for iOS as long as you respect the license (MIT). Also: the name Vesper and the app icon remain the property of me, Dave, and John.

However: everything I said above is true. Under the hood it’s an iOS 6 app. You really should treat it as history. There may be lessons to learn — what not to do as well as what to do — but I don’t recommend using it as the basis for a new app.

More blog posts

There are specific things in Vesper worth writing about. There will be more posts. (Though probably not today. It’s sunny in Seattle! I’m going outside.)

I just posted Vesper’s API server up on GitHub.

Again: this is provided as historical artifact, not as living software. It no longer runs anywhere. And I don’t make claims about quality — it’s just that it may be interesting. (And may not be.) It gives me something to write about, at least.

For possibly-helpful background, see the Vesper Sync Diary, which was written while I was working on this code.

Azure Mobile Services

It’s a Node.js server — but it ran on top of Mobile Services, which means you couldn’t just plop it down on a machine and run it unless you added the exact features that Mobile Services provides.

Nevertheless, I think the code is somewhat readable, even if it would be difficult to run it. (You’d also have to set up a database with the exact same schema, which you could figure out from the code. I don’t recommend actually trying to get this running.)

Where the code is

See the api and shared directories. You can ignore the extensions and table directories. The scheduler directory has just one simple script in it.

Things I Liked

The Azure folks provided a ton of help. They were great, and I enjoyed using the service.

Once the code was written and tested, I made almost no changes. It just worked. And keeping it running wasn’t a problem until near the end (I think I had to upgrade the database plan, and that fixed it).

Things I Didn’t Like

It’s JavaScript. Because I’ve spent the last 15 years writing Objective-C, I would have been far more comfortable writing in Ruby. It would have been more readable and better-organized. Hopefully I could have better avoided callback-hell (where callbacks are nested inside callbacks which are nested inside callbacks, etc.).

But, hey, JavaScript gets the job done.

The other thing I didn’t like was that there wasn’t a way to run a Mobile Services Node site locally, since the online version takes care of a bunch of things. In practice this wasn’t as bad as it sounds, but being able to run it locally would have been nice.

The Heart of Vesper Syncing

See api/notes.js. Syncing was done per-property: each property of a note had an associated modification date. When notes come into the server, and there are existing versions, those properties are merged. See the mergeOneNote and mergeOneProperty functions.

It should be no surprise that almost the exact same code — only in Objective-C — runs on the client side. It also merges property-by-property as notes come from the server, since there may be local changes that are newer.

Also see, in api/tags.js, mergeTags and mergeTag.

Encryption

The text of notes was stored in the database encrypted, with a key that was stored in the config for the site but not in the source code or in the repository. In shared/vespernotes.js, see encryptedTextForNote and decryptedNoteText.

One of the features of this was that I could change the encryption key without re-encrypting the entire database. The keys were stored with names of the form VESPER_TEXT_KEY_0, VESPER_TEXT_KEY_1, etc. And there was another config item that specified the current key. When the current key failed to decrypt note text, it would try the previous key, and so on back to the very first (zeroth) key. See the loop in decryptedNoteText.

This is different from providing end-to-end encryption, of course. These days that’s probably the way to go. But at the time we wrote this it was reasonable not to do that. Times change.

(Note that nobody ever asked for any data from our system, and we would have to create a mechanism for that, which we never did.)

I’m making Vesper open source as a historical artifact, because parts of it may be interesting, and not because of any claim that the code is exceptional or that it’s an example of how to write apps these days.

There are good and bad parts of the code. The iOS app, in particular, is definitely old-school — it was written for iOS 6 originally, and doesn’t use Swift, size classes, Auto Layout, and so on. It’s not modern.

Perhaps the main reason for making these open source is that it gives me something to write about.

There are three components: the accounts management site, the API server, and the iOS app. I’ve released the accounts management site first.

It’s Boring

It’s a Node.js site that ran at https://accounts.vesperapp.co/, which no longer exists. It ran on Azure.

Vesper’s syncing system — see the Vesper Sync Diary — was home-grown. The biggest reason: the existing systems provided by Apple at the time wouldn’t work for a browser-based app, and a browser-based app was on our to-do list.

We also couldn’t use Twitter or Facebook membership for our accounts system. Partly because not everybody has an account (truly), and partly because we found that people worried that Twitter or Facebook would be able to read their notes. It wasn’t true — Twitter or Facebook would have had no access — but we still didn’t want to make people worry.

(Aside: nobody from law enforcement — nobody from anywhere — ever asked us to turn over any data of any kind.)

One of the problems with writing your own sync system with its own accounts system is that you have to provide a website where people can verify their email address, change their password, and so on. That’s what this site was for.

We also used it for internal purposes: so that support could look up individual accounts (though not their notes) and so that we could get stats on current usage.

Things I Liked

The best part of this site is that it’s separate from the API server. I like the idea of using multiple sites with a single API server that is the only piece that can talk to the database.

In this case it was just this one additional site, but if we had done Vesper as a web app, it would have been similarly arranged: it would have talked to the API server and not had direct access to the database.

Another thing I liked: it’s pretty easy to get things done in Node.js. And Node is a pretty fast server.

Things I Didn’t Like

I don’t actually like JavaScript as much as I like Ruby. In retrospect, I’d rather have done this as a Sinatra site. However, the API server was also written in JavaScript, and it was simpler to use just one language for everything server-side.

I had trouble with Jade. Check out the templates in the views directory. They look nice and clean, but I found that actually writing HTML this way was a pain. I’d much rather have just written straight HTML.

Questionable Decision

The reset-password token was encrypted data rather than a random string. Encrypted in the token was an expiration date (good for two hours), and the app did remember which tokens were used — but only in memory, so it was possible that a token could be re-used if the app was restarted before the two hours was up for a given token.

A better choice would have been to create random tokens and store them in the database — along with username and expiration date — and then delete them from the database when used or expired.

Anybody who got the source code could not have created or decrypted tokens without some additional keys, and those keys were stored outside the repository as environment variables. This is an important concept: source code leaking should be a survivable event. Every component of Vesper was written with that in mind.

The One Hard-Coded Secret

Yes, the username for support’s lookup-user page really was cano-ichiro, and the username (though not the password) was hard-coded in the source code.

I leave it as an exercise to the reader to figure out how we ended up with that username. (Hint: to Dave this username probably made no sense whatsoever.)

Thing Not Done

We never had a way for users or even support to delete an account. This was a high priority, but it never got done.

To delete an account I actually went to an Azure-provided database management app and entered in raw SQL to delete an account. This is a terrible thing to do. Luckily it was also quite rare. But I absolutely should have made it possible for users to delete their own accounts. (It’s a moot issue now: they’re all gone now.)

(Just to be clear: this has nothing with outliners or the app I work on all day.)

In Welcome to the Outline:

I could have linked to all of these stories, but instead they’re bundled into this handy thing below. We call it a stack. Enjoy.

No.

It’s been said that there are several stages of grief, starting with disbelief, hitting depression somewhere in the middle, and ending with acceptance.

And many people have assumed that our reaction to the latest Presidential election would, or should, follow that model.

I don’t think that’s right. At least not for me. My stages-of-grief model looks like this:

1. Rage.

That’s the complete list. I don’t claim that it’s productive or moral or that I’m a good person — just that it’s true.

Here’s the thing: this isn’t really grief. You probably know all too well what it’s like to lose a loved one. A very sad thing happens, and you recover slowly, and not in a straight line. That’s grief.

But the Presidential election isn’t one sad thing that we’ll recover from. It’s a promise that terrible things will happen later.

This is not grief.

To everyone who continues to call for unity, who says that we should get over it and come together as a country, I’d ask why should I.

This March will be my third trip in three years to Yosemite — I can’t miss it. I’m hopelessly addicted to one of the most beautiful places in the world.

You can go too! It’s 20% off if you register this week.

It’s not a code conference — it’s about people and art and love. Sessions are in the morning and evening, so you can go on hikes during the day.

It’s so much more beautiful and awe-inspiring in real life than on your desktop.