March 2014

If your app makes unread counts an option — and off by default — is that enough?

Here’s the thing: compulsive people will turn on unread counts, because it’s more information, and because it will help them make sure not to miss anything.

The better choice is to decide what kind of app you’re making. If it absolutely has to have unread machinery — counts, badges, bold headlines, little blue circles, notifications, whatever — then fine. Maybe email apps have to have these things.

But if you can make an app that doesn’t need these things, then don’t even make them an option. Because the very people who shouldn’t turn on that option are the people who will turn on that option.

One of my favorite podcasts is Edge Cases by Andrew Pontious and Wolf Rentzsch. (My superpower is that I can spell Wolf’s last name without looking it up.) You should listen.

In the recent episode they mentioned that at one point it seemed I was concerned about NetNewsWire’s effect on its users — whether or not it was healthy.

It’s true. I was concerned. Memory, which can be tricky, tells me that there was a tweet by Jacqui Cheng (the exact tweet was years ago, probably hard to find) where she was complaining about returning from a trip to thousands of unread items.

I suggested she mark all as read.

It felt good to say that — so good that I repeated that advice many times over the years.

There Was Nothing Like This

When I first designed NetNewsWire in 2002 there were no native Mac RSS readers. I don’t think there were any that had an unread count, and certainly no three-paned-aggregators.

(Though NetNewsWire was — I think — the first one, or at least the first one to ship, it’s likely that this was invented independently by a few different people.)

There was no Twitter or Facebook then. There was email, the web, chat, and Usenet. NetNewsWire was modeled after Usenet — NewsWatcher, specifically. (The name was an homage to NewsWatcher and to nn.)

I liked the idea of seeing articles from different sources, and knowing which I’d read and which I’d hadn’t, and I especially liked going quickly through unread items.

Adding an unread count seemed like a natural thing to do — Mail had one, after all.

It didn’t even occur to me that it might not be safe.

I’d never heard anyone complain about having too many Usenet articles to go through. Maybe I just hadn’t heard — but it really seems like it wasn’t a thing.

Yes, I’d heard people complain about email. But I figured that email was different: you have to read your email, and often you have to reply. RSS is different: you don’t have to reply, and it’s just stuff to read if you feel like. There are no have-tos about it.

So what I thought I was making was a lightweight little app that saved you a bunch of time. What we all used to do was just have a bunch of bookmarks, and we’d manually check our favorite news sites a bunch of times a day.

The point of NetNewsWire was to save you that time.

Works as Designed

So it did what it was supposed to do. But it also did things I didn’t expect. Instead of a dozen bookmarks, people had a hundred feeds. Or two hundred. Or two thousand.

And there was a tyranny behind keeping track of unread items and showing an unread count. People reacted in different ways, but many people felt like they always had to go through everything.

Including me. To this day.

I did not know this was going to happen. That was not the idea: it was a side effect of reasonable (at the time) choices.

My Advice

If you use an RSS reader, don’t feel like you have to read everything.

You get stuff via other means too — email, chat, Twitter, etc. — and it’s likely that anything super-important will find you.

Mark all as read when you need to and don’t stress about it. It’s going to be fine.

If you’re an app developer, please consider the effects of your work on other people. It may be hard to guess — but just try imagining what spiral someone who’s compulsive might fall into.

Does your design lead people to feel like they’re a slave to the app, rather than the other way around?

This is especially true if you’re badging and doing unread counts and notifications. I know it “drives engagement” and is something people ask for. After all, it’s nicer to be notified rather than having to check for new stuff.

But remember that this can be hard on real people.

Nick Bradbury, FeedDemon author, and I have talked many times over the years about how we’d design an RSS reader were we starting over. The first thing we always say: No unread counts! (And then we get distracted, because we’re probably in a bar after a long day.)

But it should be pointed out — contra Andrew and Wolf — that I loved my app every single day. Madly.

I’ve started running a weekly sponsorship on my blog (a post every Monday) — I’m following the model of Daring Fireball, Marco.org, Shawn Blanc, and others.

I’m also running one display ad, an ad via The Deck. I’ve worked with The Deck before and I’m a fan. (The free version of NetNewsWire 3 ran ads from The Deck.)

Recently I realized that I have three projects: Vesper, The Record, and this blog. To take this blog as seriously as I take the other projects means, among other things, justifying the time I spend on it.

In case you’re interested, here’s more information about sponsoring inessential.com.

Heyday gives people the superpower of perfect memory, with a beautiful lifelog that automatically records the places you go, things you see, and people you meet. Apple liked us so much they selected us for Editor’s Choice.

Heyday’s small team of 9 based in South Park San Francisco, includes Zac West, who was the lead developer of Adium for over 3 years. Great technical ability is a given, but since Heyday relies on engineers to own product decisions, product and design sense is highly valued. Visit http://hey.co or send an email to siqi@hey.co and you’ll hear back in a few hours.

Though Q Branch Standard Kit has a bunch of tests, Vesper doesn’t. I’ve never been very good about automated tests for my apps.

I’m changing that right now. The next version of Vesper will be very well-tested — mainly because I need to make sure every aspect of syncing has been tested thoroughly.

I’ve been bad about testing in the past because so many things are hard to test. Or they’re so obvious that a quick human test is the best thing. Or they require a great deal of state. Does that animation look right? Does the browser open when you tap a link? Does that thing popup after you type two characters? Etc.

(That’s not to say that automated UI testing isn’t do-able. It is. But it’s more that I haven’t even gotten that far. I should do regular testing first.)

Initial Victory

I had a nice class with an ugly name — VSV1DataMigrater — that did what you think: it took a Vesper 1.x database and migrated the data to a Vesper 2.x database. Then it moved the old database to a new location so the migration wouldn’t happen twice.

I couldn’t figure out how to test this. It did a bunch of async stuff. It wanted some databases to be in the right place in the file system. The migration would run automatically at app startup anyway, since the database would have to be in the right location — which means the test wouldn’t run because the database would get moved before the test got a chance to run.

After some futzing, I finally asked myself what part of this I really needed to test. Answer: data extraction. I want to make sure that the objects generated from the old database were what I expected. That is: notes have the right text, tags, attachments, and so on.

So I took a hammer to this class and broke out a VSV1DataExtracter class. (Another ugly name, but accurate.) It takes an FMDatabase and returns the extracted model objects, and doesn’t do anything else.

Since it takes that parameter, I can use a test database stored anywhere (rather than where the app expects it). I added a test database to the Vesper Tests target.

Since it has no notion of a queue — but can be called from within a queue, and will be when running normally — I could call it from the main thread, so I don’t have to deal with async XCTests. (Which looks like a pain. I expect I’ll have to go there soon.)

I consider this a case where the needs of testing made for better code. It’s good that the data extracter is its own class that takes a parameter and runs without side effects of any kind.

Next Up: VSTagSuggestionView

Here’s a case I’m not sure how to handle.

The tag suggestion view (which appears as you’re typing a tag) searches through the list of tags using some NSPredicates. It’s a single method in VSTagSuggestionView — but it’s obviously the kind of thing that can and should be tested.

The method looks like this:

- (NSArray \*)tagsMatchingSearchString:(NSString \*)searchString;

It asks the data layer for all tags that have at least one note, then runs the predicates on those tags and returns an array of matching tags.

The first thing to do is obvious: it should take an array of tags as a parameter, so tests don’t rely on the state of the database. I’d change the method to look like this:

- (NSArray \*)tags:(NSArray \*)tags matchingSearchString:(NSString \*)searchString;

But here’s where I’m not sure of the best thing to do.

I could leave the method in VSTagSuggestionView, where it belongs (nowhere else is this method needed). But that means that to test it I have to add the method to VSTagSuggestionView.h so a test method can see it. It also means that a test method needs to instantiate a VSTagSuggestionView.

I could fix the second problem by making it a class method. But that still leaves this method in the .h file, where it would exist solely for the purpose of testing. I don’t like that.

Or I could make it a separate class — but it seems crazy to do that for one pretty simple method that’s used in just one place. (I’d be on the way to creating testing bunnies.)

What do experienced testers do in this case?

Update 2:15 pm: Lots of good advice via Twitter. (Email too.)

Here’s what I decided for this case: I created a VSTagSuggester class. It has one method which can be directly tested:

+ (NSArray \*)tags:(NSArray \*)tags matchingSearchString:(NSString \*)searchString;

I could worry about creating a zillion of these and ending up with test bunnies. But so far it’s just one of these objects, so I’ll keep calm.

I didn’t like how searching was implemented in Vesper 1.x. I’ll describe it so you can freak out at me. I’ll note that it does have the advantage of working, but that’s it.

Vesper 1.x Searching

Given a search string, it queries the database with one or more like %word% expressions. (One for each word in the search string.)

The database has a searchText column that is a copy of the text column but with everything made lower-case, with whitespace and punctuation removed, and with diacritics stripped.

In other words, if note.text is “Pick up some milk and éclairs” then note.searchText is “pickupsomemilkandeclairs.”

While this works, it’s not pretty. It means keeping that searchText column updated, and it means a larger database than I’d want. It means repeating data which is very not cool — that’s where it feels the ickiest.

What I Should Do

SQLite has full-text search (FTS) extensions. (I believe these are included with the on-the-system SQLite. If not, I’d have to compile my own SQLite and use that. Do-able.)

While I’d like to use FTS, it’s more of an effort than is warranted at the moment. (My priority is syncing.) In particular I’d have to determine if it supports character sets such as Chinese and Japanese. A little research leads me to think that my only option is writing some code and testing, which is more than I’m going to do right now.

(I will get to this, but not now.)

So that leaves me with my old-fashioned approach. One day a few months ago I nuked that searchText column in the database, which means that searching has been broken in the dev version of Vesper for a while. Any solution needs to not use that column.

What I Did

Conceptually I’m still doing the same thing: checking each note to see if it contains all the words in the search string. I’m still lower-casing and still stripping whitespace and diacriticals. (But leaving punctuation in place, at least at the moment, which is a difference.)

The difference is that I’m doing this on-the-fly, by adding a custom function to SQLite, which is something I’d not done before, which turned out to by easy. Here’s a gist from Gus which shows how it’s done.

This is cool because I can write a query like select blah where textMatchesSearchString(text, ?); — where textMatchesSearchString is my own code.

And textMatchesSearchString is simple. It uses enumerateSubstringsInRange to loop over the search words, then uses rangeOfString with NSCaseInsensitiveSearch and NSDiacriticInsensitiveSearch to check for matches. If a row’s text matches all search words, then it returns YES via sqlite3_result_int. (It’s a custom SQLite function, sure — but it’s just a block, and Objective-C and Cocoa are perfectly legal in that context. Nothing weird.)

It’s still plenty fast enough. I don’t even notice the fetch time. And that evil searchText column remains gone.

Future

The longer Vesper exists, the more likely are big databases. This solution won’t be good enough forever — I’ll have to start using some kind of full-text search eventually. (Probably SQLite FTS, though I can’t say for sure.)

I hear the siren call, loud and lovely, enticing me to go all the way to FTS right now. But I resist, knowing that delays to shipping syncing just aren’t worth it. Somehow I resist.

(You’re still welcome to freak out at me.)

My old co-worker Nick Bradbury, who now works on WordPress for Android, writes Android’s Overblown Fragmentation Problem Revisited:

I haven’t found the number of devices to be as big an issue as the number of OS versions.

That’s an interesting point — different devices is one thing and different versions of the OS is another thing. If I had the ability to pick just one of the two problems, I’d pick different devices over different OSes. (Nick’s piece includes a breakdown of OS versions that their customers are running.)

Every episode of The Record in season one has been recorded live and in person — which means three mics and Chris building a fort of foam and cardboard so that each mic can be isolated. (It’s tricky.) Chris later does the sound editing too.

My job is writing the show notes and publishing. I listen to an initial rough edit and type everything that gets mentioned (on Wednesday or Thursday), and after that I go and find links for everything.

The latest episode with John Chaffee has 106 links and is typical. (Some show notes are longer.)

I do this partly because The Record is a serious act of documentation and partly because I want the website to be interesting itself and not just a way of delivering a feed to your podcast client.

Or at least that’s what I say. I think I really do this because I ask myself what I would want from a podcast like this — and I would want all the links. All of them. In a big pile. So that’s what I do. (Plus: it’s fun.)

PS I just checked The Record’s numbers: the average number of downloads is over 10,000 now. I’m so pleased.

In The Record #7, John Chaffee — of BusyCal, which is good, which I use — talks to me and Chris about being a Mac developer in the ’90s, what it was like at Now Software, and how he got tired of mobile and came back to the Mac.

The Day Microsoft Gave Up World Domination and Settled For Relevance:

It used to be that Mac support was a vestige of history and a mistake that would not be allowed again. Now, they’re prioritizing other platforms over their own, and supporting more other platforms than their own.

We’re meeting at the Cyclops tonight at 7:30 pm for an unofficial westside Xcoders.

I felt like making a list of all the Seattle-area apps, because I’m proud of my area. Here are apps that were or are made here (to a substantial degree) or were born here or moved here.

Vesper
VoodooPad
Acorn
Napkin
NetNewsWire
Versions
Kaleidoscope
MarsEdit
Glassboard
OmniFocus
OmniGraffle
OmniWeb
OmniOutliner
OmniPlan
Microsoft Office
The Magazine
BusyCal
BusyContacts (coming soon)
Delicious Library
Vellum
The Loop Magazine (TypeEngine)

I’m sure I’m not thinking of some apps (or just don’t know about them).

Most of these apps are represented at every Xcoders meeting. (Except for TypeEngine — I’m not sure I’ve met anyone from there. They’re encouraged to attend, obviously.)

Correction, later: The TypeEngine folks totally did hang out when Jim came to visit. Cool. Of course they should come out more often. :)

I posted about why I’m not running my own server — and then my site went down. (Almost down. Super-slow.)

If it had been my own server I could have rebooted Apache (assuming that was the problem) and had it back up quickly.

Instead I emailed support. It was back up in 15 minutes.

The quick and obvious answer is that this site should be on my own server. But it’s more complicated than that.

This site is hosted on an inexpensive shared-hosting plan on DreamHost. It’s been there — along with some other sites and my email — for about 10 years. I’ve been quite happy with the performance and customer support the entire time.

This site is a statically-rendered site, and it stands up to Daring Fireball and Hacker News and everything else without even blinking. (Example: Hacker News linked to that crazy $1,850 opt-out thing from Network Solutions — and I saw the following comment on Hacker News: “I was really pleased with the instantaneous load of this blog.”)

Had This Been My Own Server

Were this blog on my own VM or machine I would have noted the outage and immediately worried if the server had been hacked or was suffering from a DOS attack. If hacked, how would I know?

But — most likely — I would have fixed the problem very quickly by restarting the server software.

Unless I was asleep. Or on an airplane. Or out with friends. Then I might not even have known about the issue.

That’s what I like about shared hosting for static sites: other people notice when there are problems, and other people have responsibility for fixing them. Since it’s a static site, if I don’t like the service for any reason I can easily move.

The drawback is that I can’t necessarily fix an issue quickly. It may take 15 minutes. (Or more or less.)

What Worries Me

So I’ve established that my site can handle serious traffic. But there’s something I hadn’t thought about before — what about extremely concentrated bursts of traffic?

A link from Daring Fireball or Hacker News certainly brings a lot of visitors, and seemingly all-at-once. But there are degrees of all-at-once.

I tweeted my post. That brings a bunch of traffic very quickly, and that’s never been a problem. Then Marco retweeted it — and traffic immediately spiked more than normal for a new post. (Google Analytics real-time view made that clear.)

So my theory is that Twitter links bring in way more concentrated bursts of traffic. Daring Fireball hits are spread out just enough that it has no impact on performance, but Twitter links can swamp a server. (Is my theory.)

If That’s True

I don’t know if that’s true — I have to spend more time observing. (If Marco retweets this, and the server gets slow again, then that’ll be a good indication.)

If it is true, then I need to find another solution, because I can’t have this blog non-responsive at the exact moment people want to read something here.

Possible Solutions

I could pick another shared-hosting provider that can handle crazy traffic bursts. Problem: I don’t know who that would be. They’re all going to claim that they can.

Or I could write a little Node.js blog rendering/caching server and set it to autoscale instances as needed. In theory (in theory) this would work splendidly, but it could get costly. (And it would mean writing some code.)

Or I could could get a VM — say the $5/month plan from Digital Ocean. For a static site I’d just need Apache or Nginx and a hard drive for my files. Sounds simple, and a gentle way to dip my toes into running my own server.

But would that actually be fast enough to handle a Marco-spike? And what happens if — again — I’m asleep, on a plane, or otherwise away from my computer?

The cheapest play, and best for me in terms of learning new things (which is not nothing), is the VM.

But could I sleep?

I’m looking forward to BusyContacts. I’m a BusyCal user and fan.

In Web Hosting for App Developers, Marco makes some important points.

One is that high-level cloud hosting (think of Node.js hosts, for instance) may come with “unexpected changes, limitations, and costs.”

Another is that you can save yourself from these problems by running your own servers — and “running your own servers really isn’t hard” and you can set up reliable servers that require a minimum of administration.

I don’t disagree.

It’s entirely possible that Q Branch will run its own servers some day. But we’re not right now, and I’ll explain why.

(Actually, we do have an internal-only server at MacMiniColo. Great service. But we’re not running servers for external services.)

Why

I ran servers — as the sole responsible guy (with no got-hit-by-a-bus-policy) — for seven years, from 1995 to 2002.

Many of these were Mac servers — pre-OS-X — which meant they were un-hackable. (Unless you did something remarkably stupid.) But they could go down. And did.

Very early in my career I ran one of these in my bedroom behind an ISDN connection. It would crash once or twice a week. I ran a system extension (I forget the name) that automatically rebooted in case of a crash. Sometimes this happened at 4 am and we’d wake up in panic at the sound of the Macintosh startup chime.

One of my Mac servers crashed and never came back — the hard drive was fried. (Code name “Roma.” An old 8500 that I cherished. Great computer. I’m still using the Apple Extended Keyboard II that came with that machine.) My second blog was on that machine and it was gone. (This blog is my third blog.)

Later I also ran Windows and Linux boxes (late ’90s, early ’00s). One of the Linux boxes was hacked and apparently used in a DDOS attack. I hadn’t updated BIND quickly enough.

These should have been in a data center, but instead we ran a T1 to my house and called it a data center. Primitive days.

Times have changed, obviously. I’d run VMs in a real data center, and have good backups, and way better admin tools.

But the thing that I hated, that I couldn’t stand after seven years, was being on call all the time, day and night, for the servers.

But. But.

But I’m still on call for our web services. True.

And I do have to do some administration. Our hosted services (two Node.js servers, S3, and a database) don’t come without some housekeeping.

And it’s likely that this will all cost more than running our own VMs. And I might run into things I wish were more flexible (hasn’t happened yet, but I grant that it could).

So it’s a matter of degree: running web stuff means running web stuff, no matter how you do it.

But running our own servers has a cost. A month or so for me to feel comfortable enough with it to trust it and myself. (I know myself: that would take a month. A month where I don’t get much, or any, coding done.) And then probably a year of day and night anxiety.

I’m willing to make this particular trade-off: more productivity and less anxiety for higher costs and possible inflexibility and limitations down the road.

I’m willing to make that trade-off precisely because I know I could run servers, and I could switch later (though not without some pain) if I choose to. And I can see that happening — if not for Vesper, maybe for a hypothetical other app.

Marco writes:

Most developers reject the idea outright without even trying because it’s unfamiliar and intimidating. It’s considered an extreme, horrible, unfathomable situation that must be avoided at all costs, usually by people who have never tried it.

That may be true of most developers, though not of me. With me I’d just rather wait and see if at some point in the future I need to, for whatever reason. (Cost, flexibility, whatever.)

If I don’t, then great. If I do, then I do, and I’ll apply teeth to ammo. But not till then.

(Experienced developers know all this stuff. If that describes you, you can skip this post.)

Say you’re crazy — which is not recommended, but it could happen — and you want to write your own data layer. One way to do that is to write very specific code that you could never reuse.

A better way is, of course, to write generic, reuseable code. But how do you do that with FMDB and SQLite? How do you make it so the data layer can work with any object?

It’s pretty easy, actually.

Simple, Non-generic Example

Let’s say you have a BSPerson object with the following properties:

@property (nonatomic, assign) int64_t uniqueID;
@property (nonatomic) NSString *name;

You’ve created the table in your SQLite database with something like this:

CREATE TABLE if not exists people (uniqueID INTEGER UNIQUE NOT NULL PRIMARY KEY, name TEXT NOT NULL);

To fetch that object by its uniqueID, you’d use an FMDatabaseQueue and code like this:

FMResultSet *rs = [database executeQuery:@"select * from people where uniqueID=?;" @(uniqueID)];

(Note that uniqueID is boxed, since FMDB wants objects.)

Then loop through the result set — there’s just one result in this case, but you might have multiple results and you’d create an array of results. But we’ll stick to one to keep this simple.

BSPerson \*person = [BSPerson new];
while([rs next]) {
  person.oneUniqueID = [rs longLongIntForColumn:@"uniqueID"];
  person.name = [rs stringForColumn:@"name"];
}

That’s super-specific, right? The code knows about BSPerson and its properties.

Reusable Example

First let’s make BSPerson more complex. We’ll add two properties:

@property (nonatomic) NSDate \*birthDate
@property (nonatomic) NSData \*imageData;

The object creator needs some information: it needs the class of the object and some properties information. You could get this information a few different ways (in code or convention, or you could write a JSON or plist file that gets turned into a dictionary).

Let’s say the properties dictionary looks like this:

@{@"uniqueID" : @"int64", @"name" : @"string", @"birthDate" : @"date", @"imageData" : @"data"}

And of course the class is [BSPerson class]. Pass the class and dictionary to the object creator as objectClass and properties and you can do this (after the executeQuery):

id oneObject = nil;
while([rs next]) {
  oneObject = [self objectOfClass:objectClass withRow:rs properties:properties];
}

(Yes, again I’m assuming a single-row result set, just for the sake of simplicity. Making an array is hardly any additional code.)

objectOfClass looks like this:

id oneObject = [objectClass new];
 
for (NSString \*oneProperty in properties) {
 
  NSString \*oneType = properties[oneProperty];
  id oneValue = nil;
 
  if ([oneType isEqualToString:@"date"]) {
    oneValue = [rs dateForColumn:oneProperty];
  else {
    oneValue = [rs objectForColumnName:oneProperty];
  }
 
  [oneObject setValue:oneValue forKey:oneProperty];
  }
 
return oneObject;

(Note: you wouldn’t really use @"date" and so on as literals — you’d define constants. Of course. And you might think twice about storing image data in a database. This is all just for illustration.)

A few things to note:

• Even though BSPerson’s uniqueID property is not an object, setValue:forKey: does the right thing. The function is getting an object from the FMResult, but setValue:forKey: knows to set it as an int64_t, because that’s how the property is defined. This is cool. Very, very cool. (Similarly, you could have an integer in the database that goes to a BOOL property.)

• FMResultSet’s objectForColumnName: handles integers, floats, blobs, and strings. The one thing to watch out for is dates. If you never have any dates, you could probably get away without having a properties dictionary — you could just use an array of property names and rely on objectForColumnName:. Or you could define a convention where a property name ending in “date” is always an NSDate property.

• You could extend the type system — you might have plist and archived types where your code automatically serializes and deserializes the values. It’s best to avoid this, but it can be useful, sparingly.

It’s tempting to want to use property introspection — via class_getProperty and property_getAttributes — instead of specifying types explicitly or via convention. I haven’t done this, and I tend to shy away from runtime functions. However, this could be a case where it’s useful and warranted. You couldn’t assume that every property has a corresponding column, but you could check the column names (which you can get) against the corresponding properties to see when you have an NSDate.

Justin Williams explains what he and I and other people have been thinking:

The ‘full stack’ paradigm is starting to make its way to app development going forward as well. It’s no longer enough to just know how to write code for a single platform. To be truly relevant and valuable you need to have an understanding of API design and implementation and cloud computing as well.

I’m using it. I’ve been a Focus user for I-don’t-know-how-long, and I love the update. The forecast view is especially cool.

I have no idea what’s going to happen the day we ship Vesper with syncing. I don’t know what traffic will be like.

My job is to make sure our servers can handle it, no matter what — which means I’ve thought a lot about scaling.

The Big Thing

The heart of the app is notes and tags, and it’s tempting to start by thinking about the notes and tags database and the API server.

But that’s not really the big thing. The big thing is pictures.

One single picture may be bigger than a given user’s notes and tags. A user with several pictures will almost always have more in picture data than in notes and tags.

We’re solving that problem like this:

• Pictures are stored in Amazon S3. They’re not stored in a folder on our server; they’re most definitely not stored in a SQL database.

• Client apps talk directly to S3. Pictures are not uploaded to our API server and then sent to S3 — the picture data never hits our API server. (The API server does, however, build connection strings for clients to talk to S3. But that’s super-fast and not a burden on the API server.)

I have no doubts about S3’s ability to scale. It handles much larger systems than ours.

And because the API server doesn’t have to handle binary uploads, it’s free to handle notes and tags syncing.

Notes and Tags Syncing - Deltas

The worst thing I could is upload the entire database and download the entire database. The trick is to do deltas — upload only the things that have changed, and download only the things that changed on the server.

Each note has a serverModificationDate field on the server. When a note changes on the server, that field is updated to the current time. It’s not a client-modified date — it has to be the date modified on the server.

When a client asks for notes with recent changes, it sends an opaque sync token that the server decodes to a date. That date ends up in a SQL query on the serverModificationDate field.

Note, though, that I return an entire note when it’s been changed. I could return only the changed fields, but that would mean keeping a server-modified-date for each field, which means more code and a bigger database. I think it’s worth sticking with one field and returning an entire note.

Data simplification

In Vesper 1.0 notes were identified by a UUID string — which is a string 36 characters long. For a client-only database that’s not the worse thing ever.

But when you have a server with everyone’s data (everyone who syncs) then it’s important to try to reduce database size. So I switched to using a 64-bit integer as the primary key for notes.

It’s a small bandwidth saving, but it’s a big saving in database size.

(Last time I wrote about this I was considering using CityHash. But just using SecRandomCopyBytes and checking for collisions is simpler. Since, on the server, the primary key is note ID + user ID, I don’t have to worry about collisions across users.)

Another simplification: I’m not syncing properties that could be calculated.

Some of the obvious ones are things like the links property. Though it’s stored in the database on the client, there’s no need for the server to know about this. The client can regenerate the links array for a note whenever its text changes.

(Other client-only database fields: links, thumbnailID, truncatedText.)

Handling Deleted Notes

The easy way to handle a deleted note is a deleted column in the notes table. But this isn’t the best use of space.

Instead there’s a deletedNotes table that stores only note IDs. (On both client and server.)

This means that when a note is deleted it’s gone from the server — its row in the notes table is deleted, and its ID is added to the deletedNotes table.

Cheating

The client database in Vesper 2.0 has four main tables: notes, tags, attachments, and deletedNotes. It has two lookup tables that relate ordered tags and ordered attachments.

The server database has just three main tables: notes, tags, and deletedNotes. And it has zero lookup tables.

Though a note can have many attachments, a given attachment relates to just one note. Were attachments a separate table, retrieving notes would also mean fetching from the attachments table. But instead I’m storing the attachments as JSON in an attachments column in the notes table.

I’m doing something similar with tags: the tag names for a note are stored as JSON in a tags column in the notes table.

You might argue that this is the wrong way to do it, and I’d understand that argument and generally agree. (And, in fact, the client database is structured the way you’d recommend.)

But by doing things on the server this way I get a couple benefits.

• Much simpler code on the server.

• Easy and fast fetches. The server can retrieve a note with one call to one table and no joins — the fastest-possible fetch. It doesn’t also have to fetch from two lookup tables and from tags and attachments tables just to retrieve a note and its relationships.

Node

This is all running in Node.js on a system where I can easily increase the number of instances as needed. Node is reputed to be fast at things like this.

Summary

Pictures are handled by S3, which scales.

The API server itself shouldn’t be a bottleneck.

The only bottleneck is the database, since there’s just the one. I’ve done everything I can think of to keep it small and simple. The server is fetching just changed objects, and the queries are all kindergarten SQL with no joins. The appropriate indexes have been created.

So — in theory — scaling on day one will just be a matter of making sure there are enough instances of the API server running. How hard could that be?

PS I think I wrote this mainly to reassure myself that it’ll be okay.

Facebook published an open source library for doing tweaks to an app’s configuration while it’s running. I’m definitely intrigued.

Mary Jo Foley reported yesterday that Windows Azure will be renamed as Microsoft Azure.

Google Cloud just dropped prices dramatically. (Via @justin.)

We think cloud pricing should track Moore’s Law, so we’re simplifying and reducing prices for our various on-demand, pay-as-you-go services by 30-85%.

The Amazon Web Services summit is Wednesday in San Francisco. It’s reasonable to expect some announcements then.

Microsoft’s Build conference (also in San Francisco) is April 2-4. Might be more announcements then, too.

In general I expect cloud providers to keep lowering prices and adding features. There are a bunch of smaller providers too. You can get an SSD server from DigitalOcean for $5/month. Heroku has free tiers. Etc.

I keep saying to Cocoa developers: if your app needs web services (including syncing), and you’ve been reluctant to learn this stuff and to pay for web services, it’s a good time to get over your reluctance and get to work.

If it’s true that Windows Azure will be renamed as Microsoft Azure, then I’m pleased. Smart move.

(Via John Moltz.)

Mary Jo Foley, who I never thought I’d quote on this blog, writes:

The rebranding makes sense, given Windows Azure isn’t all about Windows. Azure customers can run Linux in virtual machines on the operating system. Azure users also can run Oracle databases and middleware, and use non-Windows-specific development tools, including Java, Ruby, PHP and Python.

I didn’t think about calculated properties in my recent post on immutable objects.

I suspect I’m not alone in never feeling all that great about calculated properties. You either calculate them on-demand every time — which is fine if performance is not an issue and you don’t need to make them observable — or you calculate them when the data they depend on changes.

The latter is a pain and requires some special fiddling with KVO, notifications, custom accessors, and so on. (Do Core Data users use awakeFromInsert and awakeFromFetch? I always avoided this particular topic with Core Data.)

Not that it’s all that hard. I just don’t like that particular code.

But with immutable objects, it’s so simple: calculate on-demand just once. Since the data doesn’t change, there’s no need to set up recalculation machinery. So clean.

Justin Williams is a cranky old programmer who wants you to get off his lawn. And park your car on the other side of the street. And muzzle your children.

How I Evaluate Third-Party Dependencies.

Nick Harris, my co-worker at NewsGator and Sepia Labs, wrote a book: Beginning iOS Programming: Building and Deploying iOS Applications. Published by Wrox. Technical editing by Kyle Richter.

For me it was a nice surprise when I looked at the table of contents and found that it covers not just table views, the camera, and maps but also web views, web services, and making a universal app.

I’d like to make Vesper’s model objects immutable — value objects — but I’m not sure it’s do-able in a reasonable way. I’m writing this up to figure it out.

Why do this?

Mutability is a source of bugs. I’m doing syncing, which is difficult, and I should take every opportunity to reduce the possibility of bugs.

However, it’s important to remember that there are always trade-offs. A system with immutable model objects may have other bugs — after all, it must deal, somehow, with changing data.

Current Model

Objects are uniqued and cached in an NSMapTable with weak references. The idea is simple: if you make a change to an object in one place, any other place with a reference will get the change, since it’s the same object.

Concrete Example

On your day phone, you change the name of the “ice cream” tag to “Ice Cream.” It tells the sync server about the change.

Later, on your night phone, you edit a note with an “ice cream” tag. Syncing updates that one tag object so its name is now “Ice Cream,” and this is reflected in the UI.

The note and tag objects are unchanged except for tag.name.

How I might deal with this

Let’s agree to this: in the detail view, when you’re editing a note, the VSNote object is mutable: it’s a VSMutableNote.

Let’s also agree that this is the only time a VSMutableNote appears in the app. (It’s the only mutable data model object of any kind in the entire app.)

So VSTag is immutable, along with everything else. The above scenario happens like this:

Sync server reports name change from “ice cream” to “Ice Cream.” A new VSTag object is created.

The VSMutableNote object (referenced by detail view) has a tags array — which includes the old “ice cream” tag rather than the new “Ice Cream” tag. It needs to replace the old tag with the new tag.

I think the simple answer is an NSNotification. (Name TBD.) It would get sent when a new VSTag is created. The VSMutableNote object would then refresh its tags array — it would make sure it has the latest VSTag objects.

With the current system the detail view controller already needs a notification of some kind — it needs to know when any tag.name changes in its tags array, so it can refresh the UI. This is not a big difference.

Pause and figure out where we are

We have four model objects: VSNote, VSTag, VSAttachment, and VSTimelineNote.

I think I’ve found that VSTag can be immutable. VSAttachment already is. VSNote can be immutable except for that one mutable note referenced by the detail view controller.

(The sync system can use immutable VSNotes, and post a notification when there’s a change. The VSMutableNote held by the detail view can watch for that notification and do the right thing. Similar to how it handles a tag change.)

This leaves VSTimelineNote.

VSTimelineNote

The timeline holds an array of these, and no other part of the app (outside of the data layer) references VSTimelineNote.

The way they work now: when a VSNote changes, a notification is sent, and the corresponding cached VSTimelineNote (if there is one) is updated.

This means that the timeline view controller gets those changes automatically, since a VSTimelineNote in the cache is the same object as in the timeline’s notes array.

However, the timeline view controller does need to know about any changes to notes, since it may have to refresh the UI. Which means — just as in the tag/note situation for the detail view — there already needs to be a notification about changes.

When that notification is received, I could replace oudated VSTimelineNotes in the notes array with their updated versions instead of updating values.

I think that completes it: immutable model objects are not only possible (in Vesper’s case) but barely any different from the code that exists now.

Fallout

This has the effect of taking out one part of the data layer code: the uniqued-objects cache.

The whole point of that cache is to make it so that all references to a given tag (for instance) are to the same VSTag object, so that when it changes in one place, it’s changed in the other places.

(You may note that this is Core-Data-like.)

But if I’m updating objects by replacing them, then the uniqued-objects cache makes no sense. Which means I can delete some code — and probably gain some performance by not having to maintain and access that cache.

It also means that my detached-copy system — which is to create an immutable object that doesn’t appear in the uniqued-objects cache — isn’t needed. This was used in API calls and the sync system. All objects are detached copies, and I never have to think about whether an object is detached or not, and I never have to make a copy if not.

It also means that I could handle sync merging on the database queue rather than on the main thread queue. This helps ensure that the main thread isn’t blocked for any reason.

Finally, it removes a step in the data layer where I make an immutable dictionary version of an object before passing it to the database queue and saving it.

I think it’s a win, and not much work. I’m going to do it.

Simon Harris, Functional programming in object oriented languages:

I’ve starting to think of constructor arguments as the mechanism for partially applying all the methods on an object. Considering an object as a partial application of a set of methods is really quite interesting to me. It almost dictates that methods MUST operate, in some way, on the state of the object — just as we always read good OO code should — only there’s a nice explanation as to why: If they didn’t operate on the object’s state, they wouldn’t be partially applied.

Mark Bernstein, Refactoring: Taming The Bunnies:

If bunny methods repeatedly need to get stuff from the mother object, consider passing the information they need as arguments. If you have too many arguments, make an argument object. If a method on bunny A needs to talk to some other bunny, pass the bunny as an argument. The point here is to unravel the tangled strands of yarn and to knit up your raveled sleeve of care.

Curt Clifton, Three Threads:

We can encapsulate data and operations in objects while maintaining a purely functional style. In some sense this is just a higher-order form of functional programming. Instead of passing a single function as a first class value, we pass an object which is a set of functions.

John Carmack, Functional Programming in C++:

Telling an object to change itself is lesson one in a basic object oriented programming book, and it is deeply ingrained in most programmers, but it is anti-functional behavior. Clearly there is some value in the basic OOP idea of grouping functions with the data structures they operate on, but if you want to reap the benefits of functional programming in parts of your code, you have to back away from some object oriented behaviors in those areas.

(Via Buzz Andersen.)

I don’t hate writing code in C. Quite the opposite — I love writing in C. It’s low-level enough to be fun. (Languages are fun when they’re either very low level or very high level.)

From time to time I wonder how much of the typical app could be written in C. Often I’m not thinking of C-and-only-C: I’m thinking of C functions but allowing Cocoa types.

That is, the app would be mostly C functions, but you’d still use NSArrays and everything else you’re used to when using Foundation and UIKit/AppKit objects.

There’s nothing you could do about some situations: you have to subclass NSView, UIViewController, and similar. The frameworks just don’t work otherwise.

But there are so many things that don’t actually have to be Objective-C objects.

Looking at Vesper: database access could all be in C (making heavy use of blocks and GCD). Thumbnail renderer and thumbnail cache — C. Attachments storage. VSTheme. Much of the sync system. Parsers. Etc. These could all be implemented with C functions (but allowed to use Cocoa objects).

Not that I’m doing that.

But here’s what tugs at me about it — my parents taught me a basic rule: a function should take some arguments and then return something based on those arguments. It should always return the same thing, given the same input values, and it shouldn’t look at anything else but those arguments.

I was raised on pure functions, in other words.

Pure functions are not only conceptually beautiful, they’re also easy to think about and they’re especially valuable when dealing with concurrency.

Not that everything listed above would be entirely pure — database and file system access doesn’t make for pure functions. But rendering a thumbnail does. Transforming JSON into a model object does. Merging two model objects does.

So there’s another thing that tugs at me — a certain type of honesty. There are all these singular resources, things that will only ever exist once and once only: the database of notes, the folder of attachments on disk, and so on. Pretending that these are object-oriented doesn’t feel quite right.

C APIs

I’m a fan of well-done C APIs. But when designing these you run into a dilemma pretty quickly, and that dilemma illustrates the tension between pure functions and singular resources.

Do you:

1. Create static variables inside the .c file? Or…

2. Pass around a pointer to a struct that the functions take as an argument?

If you do the first, your functions won’t all be pure because they refer to internal state.

But if you do the second you’re making coding more difficult — you have this pointer to pass around. Every caller needs to have a reference. And maybe you’re pretending that a thing could have multiple instances when really there’s just the one, which is not strictly honest. (But also isn’t the worst thing.)

Inventing Object-Oriented Programming

Consider Objective-C. It’s exactly as if you’re implicitly passing around a pointer to a struct to every method, and the pointer is always called self.

This is nice. It sort-of solves the dilemma with C APIs described above.

But it leads me to wonder if the following (contrived example) is a pure function or not:

- (NSUInteger)numberOfNotes {
  return [self.notes count];
}

That’s a pure function if you remember that self is the implicit first argument. It’s not mutating anything or doing I/O. Given the same inputs it will return the same value every time.

But I don’t think of it as a pure function. It is arguably referring to state outside of itself (even though, arguably, it’s not).

Given that I make my Mom happy when I use pure functions, I’m tempted to write it like this:

- (NSUInteger)numberOfNotes:(NSArray \*)notes {
  return [notes count];
}

But that’s absurd — it’s more work, and numberOfNotes is only ever about the number of notes in self.notes. It begs the question of when the parameter might be something other than self.notes. (And all I’ve done is push work to the caller, which has to reference self.notes.)

It’s absurd, but I still understand wanting to do it. I tend not to, but I often kind of wonder if I should, since I really really (as is obvious by now) like pure functions.

This is where I get a little frustrated with object-oriented programming. If you agree with me that the first version of numberOfNotes is not pure, then I think you have to say that object-oriented programming works against pure functions.

It also works against the honesty of presenting single things as single things, since the system is designed for multiple instances.

(C isn’t necessarily better: instead, those two things, purity and singleness, can work against each other.)

Sympathy

I end up thinking more and more about functional programming. I haven’t done any — but it’s so close to my earliest coding lessons that it talks to the lowest, most basic layers of my programmer’s brain.

I listen to a small voice that is strict about no side effects and no mutations and no looking outside the parameters list, and that voice is constantly complaining about object-oriented programming.

Which makes me highly sympathetic to the goals of ReactiveCocoa, while at the same time I wonder if functional programming is too much standing-on-your-head for user interface code, which is all about state. (I draw no conclusion: I’m just wondering, and I’m keen to find out the answer.)

It also makes me interested in languages such as Go and Rust. And so on.

Anyway. Things are happening.

Dave Wiskus on Vector #34:

There’s no Apple Design Award for being the guy on the team who’s right about the most stuff.

People sometimes ask me what it’s like to work with Dave and John. Answer: it’s great.

I have a theory: people who have a track record of doing good work may be egotistical bastards — me, for instance — but their egos are served by the shipping product and not by being right about the decisions along the way.

People who insist on “winning” the process will create products that are less good.

The shipping product is everything. Everything else is nothing.

I’m not sure I have the best pattern for serializing and de-serializing model objects as JSON.

I’ll describe what I have.

QSAPIObject Protocol

Some (but not all) of my model objects can be represented as JSON. Those model objects are not subclasses of anything (by design) — but they do implement a couple protocols, including the QSAPIObject protocol.

This protocol requires two methods:

- (NSDictionary \*)JSONRepresentation;
+ (instancetype)objectWith​JSONRepresentation:​(NSDictionary \*)JSONRepresentation;

This part makes sense, I think. There’s no shared code in JSON serialization/deserialization, so it makes sense as a protocol as opposed to a subclass.

But here’s where it gets tricky.

Collections

I want some common code to handle arrays — I want JSONArrayWithObjects and objectsWithJSONArray.

These methods are exactly the same for all model objects: they create an array, and populate it by calling the QSAPIObject protocol methods.

What I did: I created a QSAPIObject class (in the same file as the protocol with the same name) and gave it two class methods:

+ (NSArray \*)JSONArray​WithObjects:(NSArray \*)objects;
+ (NSArray \*)objectsWith​JSONArray:​(NSArray \*)JSONArray class:(Class<QSAPIObject>)class;

So, to turn VSNote objects into JSON dictionaries, I do this:

NSArray \*JSONNotes = [QSAPIObject JSONArrayWithObjects:​notes];

And to turn JSON dictionaries into VSNote objects:

NSArray \*notes = [QSAPIObject objectsWithJSONArray:​JSONArray class:[VSNote class]];

This isn’t the worst thing. It works perfectly well. But I don’t particularly love calling class methods — why not just make them C methods?

NSArray \*JSONNotes = QSJSONArrayWithObjects(notes);

As C methods, there’s one entity (a function name) rather than two (class and method), and I like that. But, on the other hand, using C methods for things like this can be considered a code smell — it’s a sign that we’ve stepped outside of our object-oriented world. (Same with class methods, but with C it’s more obvious.)

Another thing I don’t like is that class parameter to objectsWithJSONArray. That’s a dead give-away that something’s gone wrong here in object-land.

I could add these collection methods to the QSAPIObject protocol, and make the model objects implement these. Then I’d have something nicer like:

NSArray \*notes = [VSNote objectsWithJSONArray:​JSONArray];
NSArray \*JSONNotes = [VSNote JSONArray​WithNotes:​notes];

That’s a more ideal API, for sure. But then the model objects are duplicating code — unless those methods are just one-liners that call the common implementations in the QSAPIObject class methods.

And if I do that, I’m basically adding boilerplate code to my model objects that isn’t strictly necessary, that would be there just so I could feel a little better about the API. Calling the QSAPIObject class methods is perfectly work-able, if un-lovely.

Hmmm.

Paul Goracke’s Xcoders talk Core Data Potpourri is up on Vimeo.

Thanks as always to Omni for hosting these talks and for recording and posting the videos. (Mark Boszko handles the camera and Kyle Sluder does the editing.)

Also thanks as always to Omni for showing that you can make high-quality apps, charge a fair price, and build a great business.

Peter Lewis’s Finger was not just scriptable via AppleScript, it was recordable.

If you’re relatively new to the Mac, you may have no idea what that means. The idea is simple: open the AppleScript Editor app (in /Applications/Utilities) and click the Record button. Then go do some things in another app interactively. What you do will be recorded as a script — if the app is recordable.

It’s rare that you’d want to keep the script as-is — usually you’d want to generalize it some. But it’s a great way to get started on a script.

I remember this being an important thing for app implementors to do in the ’90s, but I rarely see it mentioned now. (My old apps NetNewsWire and MarsEdit were both scriptable, but neither were recordable.)

A quick tour tells me that some apps are still recordable. Well, the Finder and BBEdit, at least.

How it works under the hood:

A recordable application is one that sends Apple events to itself when a user performs actions with the application. If the user has turned on recording in the Script Editor application (with Script > Record), actions that generate Apple events are recorded into an AppleScript script.

I feel certain you could do this with AppKit — but it would be somewhat of a pain. (It was a pain with the Mac Toolbox, too, but since you were on your own to define how your app handled user events, you might as well have picked Apple events.)

At any rate: recordable apps are cool, and I’d love to see the feature get more support from AppKit. But it’s one of those things that’s cool but is perpetually just under the priority horizon. Which I understand.

PS Finger may be old, but Stairways Software is still making cool apps.

On The Talk Show #75 John and I talk about scripting languages and the future of Cocoa development. And about coffee and popcorn and things.

After writing about how I was breaking up Vesper’s timeline view controller class into smaller objects, I got a bunch of advice suggesting that I go all the way: break out the table view delegates and data source too.

So I decided to try it. I created a basketful of bunny objects.

Here’s the thing: the timeline view controller is complex and the pieces have to work together.

So next I tossed in a ball of yarn to connect the bunnies that need connecting.

The separation into separate objects wasn’t nearly clean, in other words. I had to expose a bunch of internal methods and properties of the view controller to its bunnies. Way more than I felt comfortable with. (And there’s no @bunnies keyword to protect those.)

And some of the bunnies had to know about each other: the drag controller delegate needed to know about the table view delegate and datasource. The timeline cell delegate needed to know about the datasource. Etc.

If I can generalize from this — and I’m not sure I can; it may apply to this case only — I’d say that the more complex a view controller is, the more I want to break it into smaller objects, and the more likely those smaller objects will constitute a mess.

Happy position

Mess is in the eye of the beholder. I have a low tolerance for a situation where a bunch of objects need to know about each other. (Put another way: I’m happy typing x.y, but too much x.y.z I take as a code smell.)

The key is to find the right balance. Breaking out the animation code (one-third of the timeline view controller) into separate animator objects was absolutely the right thing to do.

Animation code is not like the rest of the code — it’s not like the actions, delegates, and datasource implementations. It’s ugly and specific and it ends up as big blocks of distraction when left in the view controller.

This leaves the timeline view controller at around 2,000 lines, which is way more than I’d like. But I can also say: hey, that’s what it takes to implement the timeline. It’s not extra code. It’s what’s needed, and not more and not less.

Warning

Again: I’m not sure it’s right to generalize from my experience with this one view controller. I prefer small, focused objects, and I’m highly sympathetic to the idea that large view controllers should be broken up.

My point isn’t that it’s never a good idea to break up a large view controller. My point is that it’s worth noticing when it’s not actually helping, when it’s adding complexity — and that it’s worth finding the right balance, which takes some thinking. (As for me, for this case, the right balance is to break out the animations and leave the remaining code in place.)

Mark Bernstein on Subclassing:

But both kinds of controller have something in common: their views fit into the right panel of the GetInfo popover. So there’s lots of stuff that they share in order to manage the view and in order to talk to the rest of the system. The common stuff – the stuff about being a swappable Get Info pane – belongs to the superclass. The specialized stuff – books, maps, texts, tf-idf similarity measuring – belongs in the individual pane controller.

He’s right. While in general I tend to avoid subclassing my own classes, there are times when it’s the right design.

This is particularly true when you have a design where the parent class doesn’t actually work (or nearly work) on its own, when it’s there to provide some shared code that all the subclasses need. In that case it would be dumb to duplicate that code, and just as dumb to stand on your head to avoid duplication.

It’s just that it’s not the thing we should always turn to first when doing design.

And, as Mark points out, subclassing framework classes (except those designed for subclassing) is a terrible idea.

There’s an @inessential account on Twitter.

The avatar and bio are out-of-date. It used to link to articles from this blog, but it stopped working 13 Feb. 2013.

I’d like to update the avatar and bio and get it working again — but I don’t know who owns this account.

Clearly it should be me, and it might be me. I just don’t remember. I got as far with Twitter support as finding out it was owned by a Gmail address. I used to have a Gmail account, and it could have been me, but I really don’t know.

If you’re reading this, and you own that account (not a big stretch, I figure), please send email (brent at ranchero) or tweet me @brentsimmons. Thanks!

Vesper’s timeline view controller is a monster class. Xcode tells me it’s 2,950 lines long (including blank lines), which is at least five times larger than I’d like it to be. Ten times larger than ideal.

A third of that code is transition animations.

Since Vesper was written for iOS 6, it was written before the new UIViewController transitions API. It’s time to switch to the new API.

(It’s amazing how an app less than a year old can incur technical debt. One thing I’ve learned is to deal with debt sooner rather than later, before I spend all my time paying interest.)

Here’s the plan:

1. Break the animation code out into separate classes that implement UIViewController​AnimatedTransitioning.

2. Get that code to work with the existing system. (Without actually using the new transitions API.)

3. Decide whether or not to stop there for the next release.

4. If not stopping there, then modify the current view controller hierarchy so that it’s using a UINavigationController. (Right now it’s not, because a UINavigationController wouldn’t have worked for us in iOS 6.)

I like the plan because I have the option of not going all the way, and I’d still get benefits: partial tech debt reduction and better-factored code.

But that will get the code down to about 2,000 lines. I need to break this up some more.

Going Lighter

Chris Eidhof’s Lighter View Controllers (objc.io #1) has some good advice: separate out data source and other protocols, move domain and web service logic into the model layer, and move view code into the view layer.

In my case, domain and web service logic is already where it belongs, outside the view controller.

What I do have is a bunch of delegate implementations — VSTimelineCellDelegate, UIScrollViewDelegate, UISearchBarDelegate, UITableViewDataSource, UITableViewDelegate, VSTableViewDragControllerDelegate, and UIGestureRecognizerDelegate — some view layout code, and some action methods.

I could move the view layout code to the view easily enough. (About 80 lines to move.) I think I’ll leave the action methods in place, since I consider actions part of the heart of a view controller.

Which leaves all those delegate implementations. While wisdom these days suggests moving UITableViewDataSource and UITableViewDelegate out of the view controller, I’m not sure I agree. I want the view controller to contain its very most core code, and I consider that core code to be UIViewController overrides, actions, touch handling, and table view datasource/delegate. (Reasonable people may disagree.)

Which still leaves a bunch of delegates that could be moved into separate classes: VSTimelineCellDelegate, UIScrollViewDelegate, UISearchBarDelegate, and VSTableViewDragControllerDelegate.

Moving those out probably won’t get the class down to 500 lines, but it’ll be close.

The Danger

I’ve gone on ensmallen-all-the-things kicks before, and what sometimes happens is that small objects proliferate like bunnies, and one day I end up looking at it and saying, what is all this shit? How can I possibly need 10 different objects to implement one goddamn view controller? How do they all work together?

So, while I do have to cut down this big view controller, I have to be mindful that, in a way, I’m just shuffling complexity around rather than reducing it. Smaller, focused objects are better — but there’s still a complexity cost in having more things that work together. And it’s easy for me to go too far in that direction.

The posts (though not the code) for both inessential.com and ranchero.com have been living in public repositories for a few years.

I forget why I did this. But if you ever need a bunch of text to use for testing, feel free to use these.

Inessential repository
Ranchero repository

In a previous post I outlined some options I had for a timeline view controller that has three different configurations.

In the end I settled not on subclassing but on using a delegate. It still felt weird, though, as if this couldn’t possibly be the best solution.

That weird feeling was the right feeling.

Qnoid

Markos Charatzas, with the delightfully odd @qnoid handle on Twitter (which makes me think of pizza), diagnosed the problem correctly:

the problem is a creational one i.e. https://gist.github.com/​qnoid/​57c50a629a984209c6b5 … if you are comfortable with Java, related: http://qnoid.com/​2010/10/11/Don%27t​-underestimate-the-factories​—Don…

He’s right — the problem is a creational one.

But factories?

We Cocoa developers are a pastoral lot — Shire-dwellers — and factories are part of the hellscape of modern industrial languages. Not Cocoa, not Objective-C.

Except that that’s not entirely true. See the docs on Class Factory Methods.

While we’d never create a factory object, we’d add factory methods to a class. Do it all the time.

Here’s what I’d forgotten: class factory methods can be more than a single convenience wrapper for init. Which means I can do this:

+ (VSListViewController \*)listViewController​ForAllNotes;
+ (VSListViewController \*)listViewController​ForTag:(VSTag \*)tag;
+ (VSListViewController \*)listViewController​ForArchivedNotes;

This has nice advantages:

• Simple API.
• No need to create another object as delegate or configuration object.
• Self-contained — I need only look in VSListViewController.m to find relevant code.

So that’s what I’m going to do.

Now I just wish I knew why I didn’t think of this right at first. Well, next time I’ll remember it.

Dave Winer:

If this article about feeds on the IndieWeb wiki represents their philosophy, then their name is wrong — they’re not about independence, because feeds give us independence.

The article criticizes RSS and Atom for a few reasons, and those reasons are easy enough to swat away that I leave that as an exercise for the reader.

What gets me is where it notes some shutdowns: Google Reader, Apple removing RSS from Mail and Safari, Twitter removing feeds, etc.

For every shutdown of RSS feeds by a big company, an indie product such as Feedly, NewsBlur, Feed Wrangler, FeedPress, and Unread shows up.

RSS is at its most interesting and useful when big companies ignore it. It’s the domain of indie developers.

Marcel Weiher: The Siren Call of KVO and (Cocoa) Bindings.

You probably already figured it out: bindings are one-way dataflow constraints, specifically with the equation limited to y = x1. More complex equations can be obtained by using NSValueTransformers. KVO is more of an implicit invocation mechanism that is used primarily to build ad-hoc dataflow constraints.

The other day I mentioned that I’m not a fan of subclassing my own classes. But I may have a case where it’s the least-bad option.

Vesper’s timeline view controller has three configurations: all notes, notes for a specific tag, and archived notes.

There are a bunch of properties that are different based on the configuration:

• title
• tag
• notes fetcher (like an NSFetchedResultsController)
• canReorderNotes
• canCreateNewNotes
• searchesArchivedNotesOnly
• no-notes image name

To avoid subclassing

I could do a few different things:

• Pass all these to the init method.

• Create a VSTimelineConfiguration object and pass that to the init method.

• Tell the timeline view controller what type it is — VSTimelineTypeAllNotes, VSTimelineTypeTag, VSTimelineTypeArchivedNotes — and add logic to the view controller so it does the right thing based on type.

• Create a delegate for the timeline view controller.

In the normal run of things, I’d just pass all these to the init method. But there are other things the init method needs too, and it’s getting to be the biggest init method I’ve ever written. So that’s out.

Creating a VSTimelineConfiguration object feels a bit like cheating. I’d be creating a whole new class just to make my init method less long. I’m not sure that’s a great trade-off.

Telling the timeline view controller what type it is is distinctly not object-oriented. It means adding some switch statements. That’s out for sure.

A delegate’s not bad, but it makes the whole thing wordy. Methods to implement look like the below, and there are a bunch of them.

- (NSString \*)titleFor​ListView​Controller:​(VSListViewController \*)controller;

The cleanest solution for a delegate is a new class that implements just those methods. (Because the timeline-creating object already has plenty going on already, and I don’t want to add any more to it.) Which means the delegate is just a more-verbose version of a VSTimelineConfiguration object.

The argument for subclasses

I’d have three subclasses of the timeline view controller: one for all notes, one for tags, one for archived notes.

I could create a new timeline view controller by specifying the correct class and a small and sane number of parameters to init.

What I like about that: a timeline view controller itself knows everything it needs to know, which means I don’t have to look in multiple places.

The argument against subclasses

But of course I really would have to look in multiple places, the parent class and the subclasses, as I’m working on the timeline view controller.

And if I didn’t like creating a new delegate or VSTimelineConfiguration object, why would I like creating three new objects?

Okay, okay

I’ve worked this out by writing about it. (Which is how I work these days.)

I’ve talked myself out of subclassing. I’m going with the delegate. Even though it’s wordier, even though it’s a new object, it’s the best solution.

For a long time (at least a year) I didn’t have a visible RSS link on my blog, on the theory that people’s RSS reader would find the feed. But I learned that some people just assumed the blog doesn’t have a feed. It does: you can subscribe here.

(Yes, I do tweet some of the posts, but not all.)

Also: on feedshare.net there’s a list of feeds I subscribe to. Good stuff there. (It’s slightly out-of-date: I’ve added a few since. But it’s a good list anyway.)

Tim Wood — yes, that Tim Wood, Omni Group founder and CTO — talks with me and Chris about what it was like being a NeXT developer back in the ’90s, before the acquisition by (of?) Apple.

It’s all in Seattle Before the iPhone #6.

I figured we were all on the same page about Singletons, but some people have asked me why I used the pattern I used in my previous post.

I should be careful with my terms.

A true singleton is a class where, no matter how many times you alloc/init it, you always get the same instance.

A shared-instance singleton is typically referenced via something like [MyClass sharedInstance]. (Think of NSUserDefaults and NSFileManager.) These aren’t necessarily true singletons, but they might as well be.

A functional singleton is typically created once and then used in multiple places. It’s often passed from object to object — but it’s also often just made a property of the app delegate, and other objects get it from the app delegate.

I don’t like any of these. (Except for the functional singleton that actually does get passed around.)

In our object-oriented happy place, the following is true: an object knows about 1) the objects passed in to it, and 2) the objects it creates and thus owns.

True and shared-instance singletons are created, owned, and retained by themselves. Not happy.

When functional singletons are passed around, that’s good — but when they’re owned by the app delegate, that’s less good, because it means that other objects are reaching across the object graph to get a reference. (And reaching through a UIApplication shared-instance singleton.)

The problem of single resources

There is only one application object. An RSS reader has one database of articles. If your app stores pictures as files on disk — for instance — then that folder is a single resource. There’s a single group of settings.

Maybe you’ve got a bunch of these: database, pictures, caches, etc. Let’s call it six things. Are you going to pass all those things around? Rules of good coding say yes.

You might not have to pass all six all the time, but you’ve got to take the time to figure out which ones to pass around, and sometimes you have to be prepared to pass an object through a fairly long chain, through a number of objects that don’t care about it. This makes for more code and more room for error.

So you’ve decided to break, or at least bend, the rules a little bit, because you value simpler code.

The only question, then, is: how are you going to break the rules?

How I break the rules

I don’t ever use true and shared-instance singletons. (I did in the past. I stopped.) I don’t like objects that own themselves. Gives me the screaming meemies. I’m ideological on this point, and I realize you may disagree.

I will allow myself a few functional singletons that are owned by the app delegate, because at least they’re owned by something — and I’ll even let my objects reach across and grab those. I try to keep this number super-low, definitely in single-digits. (An example is Vesper’s VSTypographySettings object, which is used in a bunch of places, and it would be a major pain to pass it through the chain of objects from the app delegate to where it’s needed.)

I don’t like this at all — it feels dirty, for sure — so I keep looking for ways to get this down to zero, but I haven’t made it yet.

In the case I wrote about, VSRowHeightCache, I saw no reason to create this in the app delegate, which doesn’t care or know about row heights. It’s used only by the timeline view controller, but its data needs to be shared across timeline instances. (Since it’s common that the same notes appear in multiple timelines.)

So I considered that the actual cached data is another single resource — like a database, the file system, or a group of settings — and that I could allow the view controllers to stay pure by instantiating (and owning and retaining) their own VSRowHeightCache instances.

When I realized I could limit the area of the singleness down to an NSMutableDictionary and two C methods, I decided that that was better than a true or shared-instance singleton — because I prefer that to an object that owns itself.

Which, again, you can disagree about — but it allows me to stay in my object-oriented happy place.

The counter argument goes something like this: it really is a singleton, no matter how you hide the fact, and the honest thing is to treat it as such. An object is data and behavior, and if the data is singular, then there should be a single object.

Which almost convinces me. Not to create it as an actual singleton, but to create it once in the thing that creates timeline view controllers, and then pass that one instance to each view controller. Which I might do.

PS I should, but I won’t, adopt the tag line for this site “Doing it wrong so you don’t have to.”

I don’t particularly like global objects with a single instance when they get attached to the app delegate or are made available via a sharedInstance class method.

I do it, sure — sometimes it’s the least-bad option. But I try to keep it to a minimum.

In my code it’s a frequent issue with caches. Real-life example: Vesper’s timeline has multiple row heights, and so there’s a VSRowHeightCache so those row heights don’t have to be recalculated.

But I don’t want VSRowHeightCache hanging off the app delegate. Rather, what I want is for each timeline view controller to get its own cache via [VSRowHeightCache new].

But I also want all VSRowHeightCache instances to use the same data, and I want the cache to be able to watch for notifications that trigger removal from the cache.

Here’s how I make that work.

Static Cache

Inside VSRowHeightCache.m, there’s a static rowHeightCache that all instances use.

static NSMutableDictionary *rowHeightCache = nil;

(The dictionary is created inside +initialize, inside a dispatch_once block.)

I don’t particularly like static variables, but at least it’s limited to the file it’s in. I consider this better than attaching a VSRowHeightCache to the app delegate.

There are also some static C functions that get, set, and delete items from the cache. Those wouldn’t have to be C functions, but I like using C because it signifies that they’re the private, low-level, shared-across-instances functions for manipulating the cache. Only the C functions are allowed to access the rowHeightCache dictionary directly.

Functions look like this:

static void cacheHeightForUniqueID(int64_t uniqueID, CGFloat height) {
  rowHeightCache[@(uniqueID)] = @(height);
}

So: problem halfway solved. Multiple VSRowHeightCache instances share the same data.

Notifications

The cache also needs to know when it should delete items. It watches for insert and update notifications from the model layer. When a note changes, its row height should be removed from the cache so it can be recalculated later.

It also watches for changes to typography settings, and removes all cached heights when typography settings change.

But having separate instances of VSRowHeightCache register for notifications means it could be doing extra work. And of course there are times when there’s no VSRowHeightCache instance, and then the un-caching wouldn’t happen, which would be bad.

This is easily solved by having the class itself register for notifications.

Inside +initialize, inside a dispatch_once block, the VSRowHeightCache registers for insert and update notifications and typography-settings-did-change notifications.

It then has class methods that handle the notifications, that then call the C functions that directly manipulate the cache.

Problem solved

I can instantiate as many VSRowHeightCaches as I want to, and the cache will continue to work as expected: all instances will share data. And the underlying cache dictionary will be maintained even when there are no instances.

All this goes on inside the file. The public API is super-simple:

- (CGFloat)heightForTimelineNote:​(VSTimelineNote \*)timelineNote;
- (void)cacheHeight:(CGFloat)height timelineNote:​(VSTimelineNote \*)note;

A couple interesting links:

Graham Lee discusses OCMockObjects in Making a mockery with Mock Objects on the Big Nerd Ranch Blog:

In this scenario, the mock object is acting like a VCR, but without the chunky 1980s styling and the mangled ribbons of tape. During your test, the mock records every message you sent to it. Then you ask it to play that back, and compare it with the list of messages you wanted to send. Just like with a VCR, if you were expecting Gremlins 2 but actually recorded the news and the first half of an episode of Cheers, that’s a disappointing failure.

Ole Begemann explains NSProgress:

Apple is setting a new standard here that will hopefully be widely adopted by the open-source community. If you write code that would potentially benefit from reporting progress of the work it does, you should strongly consider adding support for NSProgress. As we will see, you wonʼt have to modify your codeʼs public API to do that. In most cases, it is enough to add just a few lines of code to the method that performs the actual work. By using an established standard, you will make your code easier to use for other developers and easier to integrate with other components.

I try to add category methods sparingly, on the principle that creating my own personal dialect of Cocoa is a bad idea. (That’s not to say I never do. Here are some.)

Much of the time I can get by without a map method by using either valueForKeyPath: or filteredArray​UsingPredicate:.

But today I ran across one that required both methods.

I have an array of notes (for Vesper’s timeline). Each note has a thumbnailID property — which is nil when there’s no thumbnail.

I wanted an array of non-nil thumbnailIDs.

I could get all the thumbnailIDs, including nils, which turn into NSNulls, with valueForKeyPath:.

NSArray *thumbnailIDs = [notes valueForKeyPath:​VSThumbnailIDKey];

And then I could get non-nil thumbnailIDs:

NSArray *thumbnailIDsMinusNulls = [thumbnailIDs filteredArray​UsingPredicate:​[NSPredicate predicateWithFormat:​@"SELF != nil"]];

And that’s not terrible. But I looked around for a better way and didn’t find anything. So I finally broke down and wrote a map method.

So now the code looks like this:

NSArray \*thumbnailIDsMinusNulls = [objects qs_map:^id(id obj) {</br>   return ((VSTimelineNote \*)obj).thumbnailID;</br> }];

If thumbnailID is nil, the block returns nil and it doesn’t get added to the array.

The map function (in an NSArray category) looks like this:

typedef id (^QSMapBlock)(id obj);

- (NSArray \*)qs_map:(QSMapBlock)mapBlock {</br> </br>   NSMutableArray \*mappedArray = [NSMutableArray new];</br> </br>   for (id oneObject in self) {</br> </br>     id objectToAdd = mapBlock(oneObject);</br>     if (objectToAdd) {</br>       [mappedArray addObject:objectToAdd];</br>     }</br>   }</br> </br>   return [mappedArray copy];</br> }

It’s surprising to me that there isn’t a built-in map method for NSArray. I may be the last Cocoa programmer to write my own.

Because I’m busy I haven’t read the release notes for Xcode 5.1 yet. Then I saw Keith Harrisons’s post Xcode Debugger Quick Look.

Want to look at a color, image, NSAttributedString, or whatever while debugging? You can. This is really cool.

PS You should subscribe to Keith’s RSS feed.

I mentioned RubyMotion and Nu the other day, but missed Objective-Smalltalk.

I like the syntax in that it gets rid of a bunch of brackets, braces, and parentheses. Lines of code look more like sentences — they even end with a period.

But I wonder if, in practice, I could get used to it. Some extra punctuation may actually help readability.

Check out the Examples page. You see things like self window makeKeyAndVisible.

Is that easier to read than [self.window makeKeyAndVisible];? I’m not sure. It may be that, with practice, it’s at least as easy to read.

But: all that aside, what intrigues me most is that this is Smalltalk, and I’ve never written in Smalltalk, yet I’m aware that Objective-C is heavily influenced by it. It’s like Objective-C’s mom or dad.

Also: it has Higher Order Messaging.

self view subviews do setNeedsDisplay is way cooler than [self.view.subviews makeObjectsPerformSelector:​@selector(setNeedsDisplay)].

Charles Parnot looks at David Smith’s data and concludes that yes, people who don’t upgrade their OS are the kind of people who don’t buy apps.

Even were this not true, I maintain that it’s too much work to support older versions of the OS for a pool of people that shrinks every day. It’s better to spend time on awesomeness over compatibility.

My earlier post about WebScript and JSTalk got me thinking: what if I could write part of my app using a scripting language?

First let’s imagine a language: Shasta, named for the mountain in California. It works with Objective-C — in fact, its objects are Objective-C objects, and Objective-C objects are Shasta objects. You can, for instance, subclass UIViewController. The Cocoa APIs are almost all available to Shasta.

It has blocks, because otherwise we wouldn’t use it.

You can use the language on a file-by-file basis, kind of like how can turn ARC on/off per file.

Another thing: it’s interpreted. The code itself is included with the app. Anyone can look in the bundle and see the code.

And because it’s a scripting language, you can’t use C. There would probably have to be replacements for common things (like for loops, GCD, and enums) and they may be syntax-compatible with C. (And maybe not.) But there’s no C in Shasta.

Let’s assume it’s fun to write Shasta code and it makes some things easier. Block syntax is simpler, and perhaps there’s never a need to declare __block variables and do things like the weakself dance. Imagine whatever syntax you like that’s easy to read, easy to write, and expressive.

Let’s also assume it works with the debugger, auto-complete, static analysis, Instruments, and all the tools we’re used to.

Would I use this in my app?

Consider that I’d still use Objective-C when needed. But there’s a whole bunch of code that could just as well be done in a scripting language. And perhaps be done better and faster — perhaps it would make for better apps that are easier to develop.

The rub, it appears to me, in this scenario, is that the actual code is included in the app’s bundle.

One way around it would be to pre-compile the scripts to bytecode. But let’s assume that’s not available. Still worth it?

Yes. (For this imaginary language, at least.)

In thinking about this, I realized that most of the code in my app — in almost any app — is just some code. Some view controllers doing their things. Animations. Layout. Action methods. Caches. Web views.

If that’s true, then why not make our apps open source?

Well, we’re not going to. But I can’t help but wonder what it would be like if developers — even developers of for-pay apps — routinely made their apps open source. (Perhaps with a license that says you can’t just re-skin it and sell it as your own.)

People would still buy the apps.

But I stop there, at wondering. I’m not advocating anything — just thinking about what would happen. It could be interesting. But it sounds scary.

Update 8:15 pm: People have mentioned nu and RubyMotion and others. And those are cool, no doubt, and I’m tempted to take a closer look.

However, what I’d really want is a peer of Objective-C — that is, something that works with Xcode’s debugger and Instruments just as well as Objective-C. It would also allow me to mix Objective-C and scripting language files.

But, more importantly, this post (despite the title) isn’t really about a scripting language. That just provided me a way to ask the question: why not make our apps open source? (No matter what language they’re written in.)

Partly it’s just because it’s extra work. Partly it’s the fear that people will run the open source version rather than buy our app. Partly it’s the fear that someone else will rip off our code and easily make a competing app.

I wonder if the fears are overblown. I wonder — just wonder, not state — if we’re not missing out on a good idea.

Whenever Twitter’s down (as it is at this writing), I get annoyed, then think of Anil Dash’s article on The Web We Lost.

The technology industry, like all industries, follows cycles, and the pendulum is swinging back to the broad, empowering philosophies that underpinned the early social web. But we’re going to face a big challenge with re-educating a billion people about what the web means, akin to the years we spent as everyone moved off of AOL a decade ago, teaching them that there was so much more to the experience of the Internet than what they know.

Twitter may be down, but blogs (and RSS) are up. While an individual blog may go down, blogs in general are always up.

Nicolas Bouilleaud linked to the WebScript programming guide.

I never worked with WebScript (or WebObjects), but I find this document fascinating.

Types are not permitted — everything’s an id. There is a literal syntax for arrays and dictionaries, but only constants are permitted. And the syntax is a little different. Example: myDictionary = @{"key" = 16};.

Selectors are just quoted, as in selector:"timeOfSession".

About halfway-through there’s a section about “Modern” WebScript Syntax. (“Modern” is quoted in the original.) It’s “a variation of WebScript designed to appeal to programmers more familiar with such languages as Visual Basic and Java.” It’s weird. (To you and me.)

It does make me think — which I’ve thought before — that there’s a scripting language living inside Objective-C just waiting to get out. And then I remember that JSTalk is pretty darn close.

Marcel Weiher reminds us that id is the default type, and that it’s perfectly legal to write code like this:

-objectAtIndex:(NSUInteger)anIndex
{
  if ( anIndex < [self count] ) {
    return objects[anIndex];
  }
  return nil;
}

Marcel also writes:

There is a subset of the language, which I’d like to call the id subset, where all the arguments and returns are object pointers, and for this it was always safe to not have additional type information, to the point where the compiler didn’t actually have that additional type information…

What’s great about the id subset is that it makes incremental, explorative programming very easy and lots of fun, much like other dynamic languages such as Smalltalk, Python or Ruby.

Me on Twitter last Saturday, when asked about ReactiveCocoa:

It’s a research experiment, and I’m glad it’s happening, but I myself wouldn’t use it in production code at this point.

This may sound dismissive, though that wasn’t my intention. ReactiveCocoa is important, and I’m glad it’s happening for two reasons:

1. Functional reactive programming looks more and more like an important part of the future of programming.

2. Research that can change how we build apps should happen not just inside Apple but outside too.

Some extremely intelligent people are working on this. I applaud and encourage them to find better ways to do what we do.

Because ReactiveCocoa is more ambitious than the average open source project, because it seeks to change at a fundamental level how we write Cocoa apps, because it’s applying the principles of functional reactive programming to a language and framework that’s only somewhat cooperative, I consider it a research project.

And I won’t use it in my production code. Not yet. (I’m the weirdo who can’t even quite bring himself to use Core Data.)

Here’s why. It has nothing to do with code quality. Rather, there’s a trade-off involved in using ReactiveCocoa that I’m not ready to make.

Remember that there are two ways to write bug-free code: 1) use techniques that make bugs less likely, and 2) write code that you can read and verify for yourself is bug-free.

(Obviously you should use both.)

I’ll borrow an example from NSHipter’s article on ReactiveCocoa (which you should read if you haven’t already).

Here’s some conventional code:

- (BOOL)isFormValid {
  return [self.usernameField.text length] > 0 &&
    [self.emailField.text length] > 0 &&
    [self.passwordField.text length] > 0 &&
    [self.passwordField.text isEqual:self.​passwordVerificationField.text];
}

#pragma mark - UITextFieldDelegate

- (BOOL)textField:(UITextField \*)textField
  shouldChangeCharactersInRange:(NSRange)range
  replacementString:(NSString *)string
{
  self.createButton.enabled = [self isFormValid];

  return YES;
}

Here’s the same thing using ReactiveCocoa:

RACSignal \*formValid = [RACSignal
  combineLatest:@[
    self.username.rac\_textSignal,
    self.emailField.rac\_textSignal,
    self.passwordField.​rac\_textSignal,
    self.passwordVerificationField.​rac\_textSignal
  ]
  reduce:^(NSString \*username, NSString \*email, NSString \*password, NSString \*passwordVerification) {
    return @([username length] > 0 && [email length] > 0 && [password length] > 8 && [password isEqual:passwordVerification]);
  }];

RAC(self.createButton.enabled) = formValid;

After so many years of writing Cocoa code, I can read and understand the conventional version easily. The ReactiveCocoa version is very different from the code I’ve been writing all along, which makes it much harder for me to verify by reading that it’s correct.

That’s a choice every developer has to make. You may be willing to take the time to learn this so well that the ReactiveCocoa version reads naturally and easily. Me, I’m not willing, at least not yet.

(This assumes I’ve taken step one, which is to research functional reactive programming well enough so that I’m convinced of its benefits. I haven’t done so, though I’ve done enough reading to lean toward thinking it does have benefits.)

Here’s what may happen: ReactiveCocoa gains popularity, and it demonstrates that functional reactive programming is, in important respects, a better way to write apps. And then Apple — because only Apple can do this part — adds framework and language support, and does so in a way that makes for easily-readable code.

That’s my hope.

I most emphatically do not believe progress has stopped. There are better ways to do what we do, and everybody who’s trying to find these better ways has my thanks.

PS Highly recommended: Rob Rix on Postmodern Programming. I’ve read it twice already.

Jaanus, Chaining execution:

Sometimes I’ve seen these animations chained by timing: the engineer calculates how long each animation takes, and delays the next one by that amount. Not only is this brittle and looks ugly in code, but it may introduce crash bugs, as the context may change during execution: the objects may go away without knowing it themselves, and happily try to run animation when it’s no longer valid to do so.

The solution to both of these things is to use block-based APIs.

Over the years I’ve come to like subclassing less and less.

You can’t get away from subclassing NSObject, of course, and if you’re writing a view controller it has to subclass UIViewController. That’s how the frameworks work, and I don’t fight or even complain about this.

But in my own designs I’ve been stepping farther and farther away from subclassing. I like the flexibility, and the lessened brain-hurtiness, of protocols over subclasses.

Which brings me to a related topic: sometimes, at night, when I’m just reading random stuff on my iPad, I like to read about Go (golang). I haven’t used it for anything, but I find it fascinating.

(I’d most likely use it for server-side work, but it’s interesting to imagine as a client-side language.)

Here’s Rob Pike, one of the Go designers, in a talk from June 2012:

My late friend Alain Fournier once told me that he considered the lowest form of academic work to be taxonomy. And you know what? Type hierarchies are just taxonomy. You need to decide what piece goes in what box, every type’s parent, whether A inherits from B or B from A. Is a sortable array an array that sorts or a sorter represented by an array? If you believe that types address all design issues you must make that decision.

I believe that’s a preposterous way to think about programming. What matters isn’t the ancestor relations between things but what they can do for you.

That, of course, is where interfaces come into Go. But they’re part of a bigger picture, the true Go philosophy.

If C++ and Java are about type hierarchies and the taxonomy of types, Go is about composition.

One of the things I like about Cocoa is that it seems (am I right?) less subclass-happy than Java and C++. With protocols, the delegate pattern, and now blocks, we do tend to prefer composition over inheritance more than many object-oriented systems.

(But note that we still do subclass NSView, UIViewController, NSManagedObject, NSOperation, and so on.)

There are a bunch of other cool things about Go. It’s worth reading about, even if it only stretches your software developer’s muscles a little bit.

Another interesting article: Go at Google: Language Design in the Service of Software Engineering, which is a text version of another Rob Pike talk from 2012:

Type hierarchies result in brittle code. The hierarchy must be designed early, often as the first step of designing the program, and early decisions can be difficult to change once the program is written. As a consequence, the model encourages early overdesign as the programmer tries to predict every possible use the software might require, adding layers of type and abstraction just in case. This is upside down. The way pieces of a system interact should adapt as it grows, not be fixed at the dawn of time.

Tim Bray talks about his new Mac setup. My favorite line:

I’m too old now to not use Unix.

Jonathan Grynspan writes on Twitter:

In the general case, requiring single-threadedness is a code smell or worse.

He was reacting to my previous post mentioning the requirement that an API is main-thread-only.

I disagree with Jonathan. I’ll describe why and what I do.

The ideal Cocoa app

In the best-case scenario, that exists only in our dreams, everything runs on the main thread. We don’t need queues or threads because everything is so fast.

In this ideal world we never have to think about concurrency because there’s no such thing.

I’ve never written an app this way and I’m sure I never will. (As computers and devices get faster, apps will be expected to do more.)

But it’s still worth keeping this ideal in mind.

The UI runs on the main thread

There’s no escaping this. The main thread has gravity — code paths tend to start there and end up there.

There’s nothing wrong with recognizing the special-ness of the main thread.

Thread-safety is difficult

You can use a mix of queues, immutable data, and locking, and still get it wrong. Thread-safety is notoriously difficult.

The way to deal with concurrency is not to make everything thread-safe. (That may not be true for server apps, but it’s true for client apps.)

Making everything thread-safe is a lot of effort, and it’s easy to make mistakes. Due to the nature of concurrency bugs, some of those mistakes will show up only as intermittent bugs that are hard to diagnose. The developer may not be able to reproduce them.

What I do

I start with the ideal assumption that everything will run on the main thread.

Once I find that a queue is needed, I keep that queue private to the object that uses it. That object’s public API is main-thread-only, even though internally it uses a queue.

That object’s API may take completion callbacks, and those tend to be called on the main thread.

(I make an exception for objects that work very closely together. That’s fairly rare.)

A typical example:

- (void)notesWithUniqueIDs:​(NSArray \*)uniqueIDs fetchResultsBlock:​(QSFetchResultsBlock)​fetchResultsBlock;

The method triggers a fetch from the database on a background serial queue. Once complete, it calls fetchResultsBlock(notes) on the main thread.

Behind the scenes that object has to deal with concurrency: it fetches notes and updates a cache of uniqued notes. But those concurrency issues are small, well-defined, and limited to the scope of that object — and the caller never, ever has to think about it.

This system works wonderfully well. It doesn’t block the main thread because it does use background queues. And it makes dealing with concurrency as mistake-free as possible because most of the code can assume, correctly, that it’s running on the main thread.

I try to avoid the following pattern, but it’s the best call from time to time.

Foo does some startup in a background serial queue. Parts of Foo’s public API don’t work until that startup has completed, so I want those methods to block callers until startup has completed.

I have not found a great way of dealing with this.

I do something like this:

- init
  create_lock()
  lock()
  startupOnSerialQueue()
    completion: unlock()

- someMethod
  lock()
  do_the_thing()
  unlock()

Ugh.

With this method, someMethod is blocked while startupOn​SerialQueue() is happening. This is good.

But someMethod also gets a lock all the rest of the time. That’s not a big deal, because, in practice, someMethod is super-fast. But it wouldn’t actually have to have that lock except right at startup.

This feels inelegant to me. It’s simple enough, and it works, but I don’t like it.

Do you know of a better solution?

My first thought is that dispatch_once would help — but it wouldn’t. (Not that I can see.) It would make sure startupOnSerialQueue is called once, but doesn’t solve the problem of blocking use of someMethod until startupOnSerialQueue has completed.

Another possibility: someMethod could put do_the_thing() inside a dispatch_sync block that runs on the background serial queue. But that also seems inelegant. Needlessly complicated.

This reminds me of the old joke. Man: “Doc, it hurts when I go like this!” Doctor: “Well don’t go like that!”

Except that I will go like that when everything else about the design is perfect. Since I can keep the async startup internal to Foo, it’s able to implement its public API without infecting anything else.

So my only question is if there’s a better pattern for handling this.

Update 4:30 pm: To be clear: I believe in serial queues. Absolutely love ’em.

I’ve had varied feedback on Twitter and email.

In the end, this is what I’m doing.

1. Using an NSLock property — self.startupLock — instead of a pthread_mutex_t. (I always forget about NSLock. But it’s easy and cleaner-looking than pthread mutexes.)

2. Not just unlocking the lock upon startup completion but nilling it out (as Justin Miller and others have suggested) since it’s no longer needed once unlocked. (In my specific case all access is main thread only, so this is fine.)

In the end this still isn’t beautiful, but it’s simpler and less code than everything else and it solves the problem of making sure that 1) startupOnSerialQueue() doesn’t block the main thread, and 2) someMethod blocks until startupOnSerialQueue() completes, and is fast the rest of the time.

The most interesting of the feedback suggested that this is a job for continuations and futures. So: something to learn. I’ll start with Mike Ash’s article on futures.

Update 10:30 am the next day: Here’s what I really ended up doing.

Note that the previous solution had a deadlock. Sheesh. It was dumb anyway.

Instead I’m just doing startupOnSerialQueue via dispatch_sync. No lock needed.

So it blocks the main thread, but it’s not noticeable. (It’s about as fast as reading a 5K binary plist from disk, which I hadn’t realized at first. That’s not what it’s doing, but performance is comparable.)

Were startupOnSerialQueue slower, I’d have to think of something else. But since it’s fast enough, it’s fine.

So it looks like this:

- init
  dispatch\_sync (startupOnSerialQueue())

- someMethod
  do\_the\_thing()

Issue #10 of the awesome objc.io is all about Syncing Data.

Which means it’s Christmas for a certain type of masochistic geek. (Well, me.)

waffle:

It’s true; other platforms do suffer from not having quite a few elegant solutions present in Cocoa. (Oh, to have every object solve key-based archiving/coding abstractly, so that you can get universal coverage with your own serialization engine; NSCoding is the most underappreciated piece of design in all of Cocoa.) And it is very neat when it all works. But what about when it doesn’t?

I should explain why I chose not to use Core Data this time. (Note that I have shipped apps that use Core Data.)

This time it wasn’t the pathological cases like marking 10,000 items as read. Instead it’s two things.

The first is concurrency. I’ve been working with multi-threaded database systems since the mid ’90s, and I understand the issues very well.

I understand the issues well enough to know that every opportunity I can take to simplify concurrency issues to the point where concurrency isn’t an issue is worth taking. (As long as it performs well and doesn’t harm the user experience.)

This becomes even more important when you add something complex — such as syncing — to the mix.

The second reason has to do with my enduring love of plain-ol’ Cocoa. I like regular Cocoa objects. I like being able to implement NSCoding, override isEqual: and hash, and design objects that can be created with a simple init (when possible and sensible). I especially like being able to do those things with model objects. (Which totally makes sense.)

And I prefer APIs like this…

- (VSTag \*)existingTagWithName:(NSString \*)name;
- (VSTag \*)tagWithName:(NSString \*)name;

…to APIs like this:

- (VSTag \*)existingTagWithName:(NSString \*)name context:(NSManagedObjectContext \*)context
- (VSTag \*)tagWithName:(NSString \*)name context:(NSManagedObjectContext \*)context error:(NSError \*\*)error;

Implementing the MetaWeblog API shouldn’t be difficult — but I’ve heard from some people that it can be.

I mentioned that I write for my static blog using MarsEdit, via a MetaWeblog API implementation. (That runs only on my local machine, via WEBrick.)

I just dug up the script. It’s insane Ruby, clearly written by a newbie — but it has also worked with zero issues for five years.

It’s also not something you can copy-and-paste, since it works only with my personal system.

But if it’s enough to help anyone else do a MetaWeblog API implementation, then cool. (It’s really not difficult.)

Here’s metaweblog.rb as a gist.

I’ve written three blogging systems. At UserLand I wrote Manila, which powered editthispage.com and a few other sites way back in the ’90s and early ’00s.

I’ve written two more just for my own use. The first, PHP/MySQL-based, powered this site for seven years. The second, which I still use, is a Ruby static blog generator. I’ve been using it for five years.

I’m a big fan of static sites. In 2011 I wrote A plea for baked weblogs.

But lately I’ve been writing apps in Node.js, which I like, and I can’t help but wonder how I’d do a blog system. (Yes, I’m aware of Ghost. It’s probably quite cool, but I’m too impatient to sit through a video, so I don’t know.)

Performance

A Node site could out-perform a static site — in theory, at least.

Here’s how I’d do it:

Blog posts would be stored in a calendar-like folder structure on disk. Files would be markdown files. I would need some convention for storing metadata at the top of each file. (My current blog system uses lines starting with a @ character at the top. Maybe MultiMarkdown has something better for this.)

When a page is requested, the server would render it (using Express, Sass, Jade, whatever: standard stuff) and return it. It would also cache the rendered page in memory, in a dictionary — and so the next time the page is requested it would be served from memory.

Apache and Nginx have to hit the file system for each request to a static site. (Correct? I would think, at least, that they have to hit the file system to check if a file has changed before returning from its own cache. I could be wrong.)

But this Node blog could return a rendered page from memory almost all of the time.

The programming for this is so simple. It begs me to try it. (But I’m resisting.)

Details

One potential problem is not having enough memory to hold all the cached pages. I doubt that would be an issue since usually it’s just recent posts that get hits. Still, though, you’d have to monitor memory use and adjust as needed.

Another possibility would be to use a Least-Recently-Used cache for the pages. Limit the cache to 20 pages, say, and you’d never have memory issues, but the cache would be a little more expensive to maintain.

Another issue is static assets. Node can deal with those okay, but that’s not what it’s best at. I’d want to put images and similar files on S3 instead. Complicates things a little bit, but with good tools and scripts dealing with S3 is fine.

Editing

I wouldn’t even bother with an editing UI — instead I’d support the MetaWeblog API so I could use MarsEdit. (I sure wish for MarsEdit for iOS. But perhaps there is, at least, something on iOS that works with this API.)

(My static blog system has no editing UI. I write in MarsEdit. I run a small webserver on my machine that implements the MetaWeblog API.)

I’d also want all the posts to be managed by SCM. (Preferably Mercurial, but Git is okay.) On adding or editing a post it would have to commit the changes — I have to assume there’s a Node module for this somewhere. I’d keep local copies of the repository on my machines at home. (Also means I could use my local copies as a staging server, which is nice.)

When a post is edited or added, the cache would potentially (probably) contain one or more pages that need to be re-rendered. Rather than figure out a dependency system, I’d just wipe the entire cache. Editing and adding pages doesn’t happen so often that this would be a problem. (Sometimes the easy way to solve a hard problem is the best answer.)

What I like about this

Again, in theory this would be fast. Given a reasonable host and no dumb programming mistakes, it should stand up to a Fireballing at least as well as a static site.

I also like that the system would be portable. In the old days the only really portable thing was static sites — you could zip up a folder and move it somewhere else. Easy. (PHP is portable too, since it’s widely-deployed. But often with a PHP site you have the issue of moving a MySQL database, which is a bit of a pain.)

But these days there are so many different Node hosts (Joyent, Heroku, Azure, Nodejitsu, etc.), and the systems for deploying and running Node are so standardized, that you could consider it very portable. Not as portable as a static site, but portable to an important degree.

So now I’m not using Core Data with Vesper. I hope the people who (quite rightly) like Core Data are not disappointed. I like Core Data too and recommend it.

Consider the below not as criticism of Core Data but as a description of what I personally like. Also consider all the things I’m giving up: faulting, NSFetchedResultsController, the Core Data modeler (I’m using a plist instead), and plenty more.

Designing for what I want

Starting over means I could think about what’s important to me in my persistence layer. At a high level, in order, it’s correctness, performance, simple concurrency, low memory use, ease of programming, and flexibility.

The main goal of the design, in other words, is to make it impossible for me to screw up the data. The last few days (since Saturday) I’ve spent writing a new system. This is how it works:

Main thread model objects, background serial queue database

Model objects live on the main thread. This makes it easy to use VSNote, VSTag, and so on in view controllers and in syncing.

There is one exception: you can create a “detached” copy of a model object to use with API calls. A detached model object exists on one thread of execution only, is short-lived, and is disconnected from the database. Detached objects aren’t a factor when it comes to concurrency.

When a model object is added, changed, or deleted, updates to the database are placed in a background serial queue.

Similarly, all fetches happen in that same background serial queue.

This way the model objects and database are always in sync, though the database lags slightly behind until its queue is caught-up.

Concurrency is therefore never an issue, and I never, ever have to worry about the main thread being blocked for database access.

The implicit merge policy is always the same: main thread wins.

Why such a simple concurrency model?

Because sync is hard. There’s a lot of data-merging going on, on the clients and in the web app. Merging is the awful part of sync, but it’s unavoidable.

Since I can avoid yet another case of merging (merging across threads), I will. It’s an entire area that can’t have bugs because it doesn’t exist.

Multiple object types per table

Vesper is typical of sidebar/timeline/detail apps in that a timeline view object needs only a subset of what the detail view needs.

So I have two objects — VSTimelineNote and VSNote — which both come from the same notes table.

VSTimelineNote has five properties, while VSNote has 14 properties and two relationships.

This is all specified in the data model. (Here’s a screen shot of the data model.)

Primary keys

One requirement: each model object must have a uniqueID property. It can be a 64-bit integer, NSNumber, or NSString.

That uniqueID is also the primary key (unique, not null) for the corresponding table. I vastly prefer this to a system where objects have a local primary key that’s separate from its uniqueID.

The problem with systems like that is that duplicates are too easy to create. I want to make it impossible.

An example: a tag’s uniqueID is the lower-case version of its name. (A tag’s name can be edited case-wise, but if it changes otherwise it’s actually a separate tag.)

There’s no chance of creating duplicate tags because their uniqueIDs would be the same, and since uniqueID is also the primary key, I wouldn’t be able to insert that duplicate tag.

Another example: a note is assigned a 64-bit integer uniqueID on creation on your day phone. That same integer its primary key. On the sync server, that note’s primary key is compound (uniqueID, userID). When the note later syncs to your night phone, it still has that same uniqueID, and the night phone uses it as the primary key. So it can’t create duplicate notes.

Relationships

All relationships are ordered. (Via a lookup table with parentID, childID, ix, where ix is the order.)

My system doesn’t do inverse relationships, but for my uses that’s not an issue. VSNote has a to-many relationship to VSTag and to VSAttachment, but to get all notes for a tag I have to do a fetch. Which I don’t mind.

When objects are fetched, their related objects are also fetched. This is done as efficiently as possible: for instance, if three notes are related to a given tag, that tag is fetched just once. (If the tag is already cached, it’s not even fetched once. And, in fact, I’m caching all the tags at startup on purpose, since they’re small and there aren’t many of them.)

Uniquing

Some objects are uniqued and some aren’t. Not using uniquing is a performance benefit, except when I could end up with lots of instances of what should be the same object.

In Vesper, VSNote and VSTags are uniqued, but VSAttachments aren’t, because a tag can be related to many notes while an attachment can be related to only one note, so it’s unlikely that a single attachment would have multiple copies at once.

(The data model has a per-object uniqued boolean.)

Caching

Uniqued objects are cached in an NSMapTable with weak references. That means a note is cached as long as there’s a reference to it outside the cache.

Optionally, an object can be cached permanently, and all objects can be fetched and cached on startup. (There are keys in the data model for this. I do both with tags.)

Deleting

Deleting n objects (of the same class) takes one SQL call. If the object has relationships, the entries in the lookup tables are also deleted. To delete 10,000 objects, where that object has two relationships, takes 3 SQL calls. (One for its table and one for each relationship’s lookup table.)

Note that this can lead to orphans — there’s nothing like Core Data’s cascading deletes. Orphans are my responsibility.

Syncing

Most of syncing happens off the main thread. Networking doesn’t block the main thread, and parsing the JSON return data also happens off the main thread. Fetching data to send to the server happens in the database’s background queue.

Merging data happens on the main thread, but the database fetches to get that data and the database updates to save that date happen in the background.

So, even though model objects are updated on the main thread, the impact is low. It’s all fast, in-memory operations. If I find later that the impact is not low, I can optimize by merging only cached objects on the main thread and otherwise merging data in the background queue. (I doubt this would be necessary.)

Flexibility

Model objects implement the QSDataObject protocol. They must have a uniqueID property and may optionally implement awakeFromFetch. They should be init-able via init. They should actually have the properties claimed in the data model.

Otherwise they can be whatever. They are not subclasses of some model object class. Creating an object doesn’t hit the database on the main thread, and I can create an object without ever saving it to the database.

Though there is a data model defined in the plist, table and index creation is done via hand-specified SQL. This allows me to exactly tune how the database is constructed. (I especially like being able to specify which things are unique.)

And it allows me to add tables that don’t appear in the data model and that aren’t used by this system. (An example is the deletedNotes table, which is just a single-column table of uniqueIDs that isn’t backed by a model object.)

Finally, because it’s all just FMDB and SQLite, I can do things that step outside of the basic system. I can solve the “RSS reader problem” — I can mark 10,000 items as read all at once with one SQL call. (The only caveat is that cached objects also have to be updated. Which is easy: just a few lines of code. I could easily add built-in support for this kind of thing to the system if needed.)

Reuse

I mentioned that the RSS reader problem isn’t an issue with this system: bulk deletes are built-in, and bulk property changes are easy to do.

While I don’t have plans for another app, I like knowing that the system would work well for these two scenarios:

1. User-created and synced data.

2. Web data with some user-created and synced data. (RSS, podcast, and Twitter clients, for instance. Apps like Glassboard and MarsEdit.)

All the apps I’m likely to ever create fall into these two categories.

I never got into Flipboard. Great UI, but nothing I want to read.

Zite, on the other hand, has been a steady favorite. Zite’s UI is a little annoying, but the main thing is that it does its job well: it finds me interesting articles in topics I’m interested in.

But now Flipboard has acquired Zite, and the Zite app will go away.

Every acquisition of a favorite app brings its share of nervousness. But if this works out, we’ll have Flipboard’s great UI and Zite’s articles.

Reading Zite has been a habit for a certain period of my life, and I can’t help but be sad to see it go. Even if it’s going to a nice farm where it can run around all the time in the sun.

My fellow Q Dave Wiskus writes in Macworld: Where Apple design is headed in 2014:

So how do you apply the lessons learned in iOS 7 to the Mac? For one possibility, look at the interesting facelift iCloud.com’s Web apps have undergone. The animations, curves, and layout feel like a desktop-friendly reimagining of iOS 7. It’s not hard to imagine this as a testbed or preview of how iOS 7’s design ethos could apply to the Mac — and it’s a good-looking translation.

Rich Wardwell talks about the weakSelf dance:

The best way to handle retain cycles in blocks is to prevent the cycle from occurring in the first place. Using the weakSelf dance is easy.

Like Rich and probably everybody else, I love blocks. I couldn’t go back.

The weakSelf thing is the least lovely part of blocks, though. The second least-lovely is the syntax: I rely on auto-complete and fuckingblocksyntax.com, which I’ve bookmarked. (Neither of these puncture my big red balloon of affection for blocks.)

Some things I’ve noticed about my own use of blocks:

• Almost all of my blocks are stack-based. I rarely copy a block and assign it to a property. (Not never, though.)

• I can often avoid the weakSelf dance by thinking about what’s really needed. My first thought is always that self is somehow needed — but that often turns out not to be true. (Not always.)

• I never use -[NSNotificationCenter addObserverForName:​queue:​usingBlock:].

Update 12:25 pm: Kyle Sluder says:

I’d say the least lovely part about blocks is the automatic temporary shadow variables clobbering data: http://llvm.org/bugs/show…

NYTimes: The Sound of Philadelphia Fades Out.

(Via Daring Fireball. Which I realize means you’ve seen it already.)

When I was growing up (in Newark, DE and south Jersey) I took the accent as a sign of low class and low education. My accent was pretty thin: I sounded more like television. (I was a jerk, in other words.)

But to this day I retain some parts. It’s arnj juice for orange juice, sarry for sorry, and, occasionally, oh-wun for on. (But never wooder for water. I actually got in trouble as a kid for saying wooder.)

One of the things I love about going back east to visit family — all of whom live within about an hour’s drive from Philadelphia — is getting to hear the accent again.

On a recent visit my brother-in-law and I were at a sports bar in Wilmington at happy hour. Guys at the bar were talking fantasy football, and I was enjoying listening to them. Their accents weren’t thick, but they were noticeable to someone who doesn’t live there.

I explained to my brother-in-law that I was listening to the accent and enjoying it.

“I don’t hear it,” he said.

But it’s still there. I can’t tell from my visits if it’s dying out or not: it seems to be alive and well. I don’t hear it much in movies, but then I never did.

I’d miss it like crazy if it disappeared.

Ole Begemann: How I Learned to Stop Worrying and Love Cocoa Auto Layout:

If you find yourself in a situation that is difficult to solve with Auto Layout, just don’t use it for that particular view. You can freely mix the constraint-based layout with manual layout code, even within the same view hierarchy.

Good advice and simple. You should subscribe to Ole’s RSS feed, by the way.

The [360

intersect conference](http://www.360intersect.com/) is April 28-29 in Seattle. It’s by the same folks who do 360

iDev — but this conference is at the intersection of technology and liberal arts.

It’s about “awesome technologists talking about the things they do away from their keyboards that makes the things they do at their keyboards even more incredible.”

To get $75 off, use the discount code XCoders. And I’ll see you there. (I’m not speaking, but I’m attending.)