Vesper Sync Diary Follow-up
This is probably the last of the Vesper Sync Diary posts — there’s not much left to write about the design. But I should go back and note which things I wrote about are no longer true, and why. I’ll go article-by-article.
I especially like this article because it introduced the day-phone/night-phone thing as a way to talk about syncing. Please feel free to use that yourself, if you like.
This article describes tags syncing exactly. No changes.
I didn’t end up using Core Data.
This article is accurate except for a few details. The main one is that we don’t have a DeletedObjects table — we have a DeletedNotes table. The schema on the server for deletednotes:
id bigint, userID bigint, noteID bigint, serverModificationDate datetimeoffset(3)
On the client:
On the server, attachment metadata is stored in a column in the notes table. Stored as a JSON string.
On the client, attachment metadata is stored in a separate table, called attachments:
uniqueID TEXT UNIQUE NOT NULL PRIMARY KEY, mimeType TEXT NOT NULL, height INTEGER, width INTEGER
This article had many follow-up articles:
Vesper Sync Diary #8 part two - More about Unique IDs
Vesper Sync Diary #8 part three - Unique IDs and Hashing
Vesper Sync Diary #8 part four - Random IDs
Vesper Sync Diary #13: Unlucky Numbers
Vesper Sync Diary #13 part 2 - Maybe It’ll Be UUIDs After All
In the end: unique IDs for notes are random 53-bit positive integers. The client checks for collisions. (A note ID only needs to be unique for a given user, so this works out.)
The notes table on the server has a primary key that’s a monotically increasing bigint. It has a constraint that each noteID/userID pair has to be unique.
This article references Core Data, but we’re using FMDB + SQLite. (I write the model layer of my dreams. If I had plans for other apps, I could use it. It’s exactly what I want.)
Instead of storing a boolean that says whether or not migration happened, I do this instead:
If the v1 database exists, run the migration. On completion, rename the v1 database. (Don’t delete it.) That’s it.
This is more about performance than scaling.
(In Scaling and Performance I talk about scaling.)
The article mentions Amazon S3 — but we ended up using Azure blob storage instead. (Which is very similar.)
Otherwise the article is accurate.
In this article I considered switching from SQL to table storage for account data. Table storage is much cheaper, and everybody likes NoSQL these days.
I did not make that switch. I stuck with SQL — for the reasons the article talks about, but also because all my experience in the last ten years or so has been with SQL, and it’s worth sticking with something I understand well.
I’m still writing tests.
I haven’t had to do much debugging — but it sure is nice to control all parts of the system.
It does make for craziness where I have Xcode open with the client app, BBEdit open with the API server, several Terminal tabs open, and several Safari tabs opens (server log, for instance). And the app running in the simulator and the app running on my iPhone.
It may be a distributed system, but I can see and affect all the parts on my computer. Sure beats getting frustrated with someone else’s server bugs.