Vesper Sync Diary Follow-up
This is probably the last of the Vesper Sync Diary posts — there’s not much left to write about the design. But I should go back and note which things I wrote about are no longer true, and why. I’ll go article-by-article.
Vesper Sync Diary #1 - Syncing Tags
I especially like this article because it introduced the day-phone/night-phone thing as a way to talk about syncing. Please feel free to use that yourself, if you like.
This article describes tags syncing exactly. No changes.
Vesper Sync Diary #2 - Core Data
I didn’t end up using Core Data.
Vesper Sync Diary #3 - Immutability, Deleting, and Calculated Properties
This article is accurate except for a few details. The main one is that we don’t have a DeletedObjects table — we have a DeletedNotes table. The schema on the server for deletednotes:
id bigint, userID bigint, noteID bigint, serverModificationDate datetimeoffset(3)
On the client:
uniqueID integer
Vesper Sync Diary #4 - In Another Country
I didn’t write about my biggest frustrations with JavaScript. That deserves its own post.
Vesper Sync Diary #5 - Sync Tokens and Efficiency
Accurate.
Vesper Sync Diary #6 - Merging Notes
Accurate.
Vesper Sync Diary #7 - Audibles
Partly accurate.
On the server, attachment metadata is stored in a column in the notes table. Stored as a JSON string.
On the client, attachment metadata is stored in a separate table, called attachments:
uniqueID TEXT UNIQUE NOT NULL PRIMARY KEY, mimeType TEXT NOT NULL, height INTEGER, width INTEGER
Vesper Sync Diary #8 - The Problem of Unique IDs
This article had many follow-up articles:
Vesper Sync Diary #8 part two - More about Unique IDs
Vesper Sync Diary #8 part three - Unique IDs and Hashing
Vesper Sync Diary #8 part four - Random IDs
Vesper Sync Diary #13: Unlucky Numbers
Vesper Sync Diary #13 part 2 - Maybe It’ll Be UUIDs After All<br/ >
In the end: unique IDs for notes are random 53-bit positive integers. The client checks for collisions. (A note ID only needs to be unique for a given user, so this works out.)
The notes table on the server has a primary key that’s a monotically increasing bigint. It has a constraint that each noteID/userID pair has to be unique.
Vesper Sync Diary #9 - Tutorial Notes Edge Case
Accurate.
Vesper Sync Diary #10 - Data Migration
This article references Core Data, but we’re using FMDB + SQLite. (I write the model layer of my dreams. If I had plans for other apps, I could use it. It’s exactly what I want.)
Instead of storing a boolean that says whether or not migration happened, I do this instead:
If the v1 database exists, run the migration. On completion, rename the v1 database. (Don’t delete it.) That’s it.
Vesper Sync Diary #11 - Scaling
This is more about performance than scaling.
(In Scaling and Performance I talk about scaling.)
The article mentions Amazon S3 — but we ended up using Azure blob storage instead. (Which is very similar.)
Otherwise the article is accurate.
Vesper Sync Diary #12 - Let’s Make This Change No Matter How Late
In this article I considered switching from SQL to table storage for account data. Table storage is much cheaper, and everybody likes NoSQL these days.
I did not make that switch. I stuck with SQL — for the reasons the article talks about, but also because all my experience in the last ten years or so has been with SQL, and it’s worth sticking with something I understand well.
Vesper Sync Diary #14 - Keys
Accurate.
Vesper Sync Diary #15 - Server Testing
I’m still writing tests.
Vesper Sync Diary #16 - Debugging Syncing
I haven’t had to do much debugging — but it sure is nice to control all parts of the system.
It does make for craziness where I have Xcode open with the client app, BBEdit open with the API server, several Terminal tabs open, and several Safari tabs opens (server log, for instance). And the app running in the simulator and the app running on my iPhone.
It may be a distributed system, but I can see and affect all the parts on my computer. Sure beats getting frustrated with someone else’s server bugs.