GCDWebServer is an embeddable and lightweight http server for Mac and iOS. On iOS it runs as a background task — it keeps running even when your app isn’t in the foreground.
Picture this: app X wants to send some data to app Y.
App Y is running GCDWebServer, so app X wraps up the data as JSON and talks to app Y via http. Just as if it were talking to some server on the web, only it’s a local app.
Two tutorials from Dawid Woldu fascinate me. I keep wanting to make time to get into Quartz Composer.
Quartz Composer isn’t included in Xcode these days, but you can still download it from Apple. (Find “Graphics Tools for Xcode.”)
A two-part tutorial from Michael Katz is a good place to get started writing services. Node makes a great API server. And it’s fun.
The tutorials use MongoDB. I haven’t had a good reason for a NoSQL database myself lately — but my early career, back in the ’90s, was all about schema-less databases, and I have a major soft spot for them.
(I’m digressing now.)
Frontier’s database was a hierarchy of tables. Each table could contain anything, including other tables — including even your scripts.
To run a script named myScript inside the bar table which was inside the foo table, you’d write foo.bar.myScript(params).
If that script took a string as a parameter, say, you could use a local variable or reference any string anywhere in the database: myApp.data.settings.username, for example. This was all presented with a user interface, navigable and editable.
I haven’t seen a database like that anywhere else since then. So easy and intuitive. Great for productivity. (It was within this laboratory that such things as templated and scripted websites, blogs, RSS, OPML, and XML-RPC were invented and/or fleshed-out.)
More so, unless you have huge sales volumes, it’s near impossible for a company selling $2.99 apps (our lowest cost product and a fee which, in the app store world, is considered wildly exorbitant) to make enough money to support their team, pay for office space, keep up to taxes, cover fees and look after all of the financial requirements that it takes to run a real business.
What is making money for them? A web app.
We’ve done next to zero promotion of this software-as-a-service (SaaS) product since we started building, using and supporting it two years ago. Instead, we’ve been slowly adding to the system, improving the platform and signing up paying customers on a regular basis. As a result, the product has been making money — real money — and long ago surpassed the combined revenues of our other commercial software applications in a big way.
I’m not sure that the choice is between web apps and app store apps — I think it’s actually about standalone apps versus apps-plus-services. But, still, read the post.
Manton Reece explains how tint color is misused:
The problem is the implementation in apps that use tint color anytime they want to highlight something, whether it is tappable or not.
The board is for questions, feedback, and Mac nerd nostalgia. Join with the invitation code APUME.
Here’s the initial design: the text of notes is encrypted in the database. The key is not stored in the source code. (The source code could get out and you wouldn’t be able to decrypt notes.)
That design is a no-brainer, and I thought I was finished at that point. But then I did a design review with some security folks, and they suggested I revise it like this:
The text of notes is encrypted in the database. They key is not stored in the source code. The key should change from time to time. To make this work:
Always encrypt using the latest key.
Add some token to the text that lets me know if it’s been successfully decrypted.
Try decrypting using the latest key first. If the token doesn’t appear in the right place, use the previous key. Repeat if necessary until decrypted.
Add a new key regularly. (Twice a year, for example.)
Be prepared with a script that re-encrypts all note text with the latest key, in case there are any security concerns at all.
This is sensible. I can’t think of any reason not to do it this way. Is there anything I’m missing?
Keith Harrison shows how to run custom Clang analyzer builds:
The open source build is updated frequently with bug fixes and extra checks. This makes it more likely to spot an error in your code that would go undetected with the Xcode bundled version.
I went through Twitter’s impersonation process and was able to get the @inessential username.
The way it worked is that Twitter renamed @inessential_com, which I created a few days ago, to @inessential.
So if you followed @inessential_com, you’re now following @inessential, and there’s nothing you need to do.
Justin Williams: Refactoring in the Cloud:
Given that I’m not a fan of the C# threading model currently implemented, the amount of code used to power something that’s relatively simple, and the homegrown nature of many of the push wrappers we’re using, I made the decision to rewrite this portion of Glassboard’s architecture using Node.
Justin’s use of Azure is quite different from mine. Glassboard is more complex than Vesper syncing, which should be no surprise.
(If you think about it, you realize that Glassboard does syncing, but it’s silly to call it that because it’s a natural part of what’s expected of a group messaging app.)
Vesper’s architecture is smaller. There are two main components: an API server (Node.js Mobile Services app) and a reset-password site (Azure web site, also Node.js). There are no architecture-level refactorings for us to do at this point.
We’re at about 2,000 lines of server-side code. Small.
More than one person has suggested I’m guilty of violating the law of premature optimization when it comes to my server work.
Here’s the thing, though: when it comes to database schema, I really, really want to get it right before shipping.
Making code changes in a client app is normal. Making database schema changes in a client app is a pain, but not the worst thing.
Making code changes on the server is normal too, though a little hairy. But the hairiest of all is database schema changes on the server. I’m designing so that I don’t ever need to do that. (I may not reach that goal. Time will tell.)
Even though Brian Reischl wrote up how to do data migration, and so I have a good plan if I ever need to go there, I just don’t ever want to go there.
In other words: getting the server-side database schema right right now isn’t premature — it’s exactly the right time.
My SQL Server genius pointed me to this article by Kimberly Tripp about the problem with GUIDs as primary and/or clustering key. (GUIDs and UUIDs are the same thing in this context. Microsoft folks often call them GUIDs.)
Another article suggests this isn’t a big deal with Azure SQL because a network write is slower than a page split anyway.
But the advice still seems to be that UUIDs don’t make the best clustering key. You want something narrow to keep down index size.
So I’ve slept on it. Do I still like this layout for the notes table?
id - auto-incrementing integer clustering primary key
noteID - UUID, unique
userID - int
Yes, I like it.
Cesare Rocchi argues that the Heartbleed problem isn’t C — it’s the API.
My counter-argument: people make mistakes. People make dumb APIs. With C, combine a dumb API and a mistake and you get Heartbleed.
That’s far less likely with another language.
Here’s the thing: we will always have dumb ideas and mistakes. We can and should do our best to eliminate them, but we’ll never succeed entirely. Because we know that, we’re negligent if we don’t do our best to minimize their consequences.
Graham Lee notes that computing turns 100 probably within your lifetime and proposes a book about programming that I want.
There are two pieces of advice I’ve been getting:
One is that I’m thinking too much about this. It’ll be fine if I have a properly normalized schema and I use the appropriate indexes. After that, don’t worry.
(I admit that I’m prone to going down every performance rabbit-hole I can find.)
The other — partly related — is that the right way to deal with the notes table is to do create a clustered primary key as userID + noteID. This way all notes by a given user will be together.
And this is the default behavior. It’s good. Smarter people than I am have thought about this.
And I can drop the integer identity column.
Update a few hours later: No. Wait. The best database guy I know tells me to do it the way I was thinking: surrogate key integer identity column as clustering key.
He also suggests I don’t need to add a unique constraint for noteID + userID, since noteID is a UUID. A unique constraint on noteID is all that’s needed.
I helped Ondřej with some of the wording on the home page for SettleApp.
Some advice for developers in general, off the top of my head:
Use words that humans use. (No “algorithms.”)
Use verbs. Verbs are vivid. Verbs trigger people’s imaginations — they picture themselves using the app.
Use curly quotes.
Don’t talk about how you rewrote any part of your app. Nobody cares.
Don’t talk about how it’s native or about how it’s ready for the latest version of the OS. These things are assumed.
Don’t be repetitive.
Cut, cut, cut, cut, cut, cut.
I slept on the thing about using 53-bit integers as note IDs. I woke up and it felt weird — slightly but detectably. I listen to those feelings.
As previously mentioned, I’m still learning my new environment. Any new environment takes time. The thing to do, when things seem weird, is more research.
UUID data type
SQL Server has a UNIQUEIDENTIFIER type. (Sorry for the shouting, but database people are shout-y.)
I assume this saves space over just using a 36-character UUID string — I figure it strips the dashes, at least.
That’s wider than a 64-bit integer, and I don’t love it, but it’s not the worst thing ever to happen. The great thing about UUIDs is that they can be created on the client and guaranteed (for practical, real-world values of “guaranteed”) not to collide.
(Remember that, in Vesper’s case, a note ID need be unique only for a given user.)
I really wanted 64-bit integers. But I’m willing to accept UUIDs because it’s not weird and is a standard practice.
Azure SQL Server insists that each table has a clustered index — that is, an index which tells the database how to physically lay out the data.
The primary key is by default the clustered index. The primary key for the notes table is noteID + userID.
Recall that UUIDs look like this:
There’s no order to them. So, to insert a new note could mean inserting it anywhere in the table rather than at the end.
(Yes, I know about NEWSEQUENTIALID(), but I’m generating IDs on the client.)
But here’s the thing: I had this exact same problem even before using UUIDs, because I was using random integers.
What I want instead is a way to add a row at the end, like adding a page to a book. For that I’d need to create a separate identity column with a monotically-increasing integer — like adding a page number — and then make that the clustering key. (The primary key and clustering key do not have to be the same.)
Having primary key as noteID + userID, and a separate identity clustering key, is just about the same thing as this layout:
Primary key: identity integer (page number style)
Constraint: noteID + userID is unique
This alternate approach feels more right.
My goal is to make noteID + userID unique, and to enable fast lookups. Adding a constraint (which adds an index) makes that work. I don’t also have to make it the primary key.
Does my code ever need to see the identity integer? The clients don’t — it’s a server-side detail only.
You know what? I’m fine with that. To hit that number, every human on Earth would have to create over a million notes. I won’t worry about it. I’d be a billionaire long before then, and my blog posts will be me figuring out which island I should buy.
(With that much money this problem would be solvable, in other words. Solvable by other people who I pay very, very well.)
Summary of changes
I think it’s a plan.
Client will have to make these changes:
- Generate note IDs as UUIDs.
- Store note IDs as UUIDs. (Change database schema and data model objects.)
- Fix code that expects note.uniqueID to be a 64-bit integer.
Server will have to make these changes:
- Update notes and deletednotes tables to have an integer identity clustered primary key.
- Update notes and deletednotes tables so that noteID is a UUID.
- Updates notes and deletednotes tables with a unique constraint on noteID + userID.
The bad news: there’s more client-side code to change than I’d like. Normally a schema change in a development version isn’t a rough thing, but it does mean going into the database migration code again, which I don’t like doing.
The other good news: but the database migration will actually be simpler, since I was using UUIDs for note IDs in Vesper 1.0. I can preserve those UUIDs across the migration, where until just now I was changing note ID to 64-bit integers. So that’s cool. Simpler data migration is better.
PS The only thing that gives me pause — and something always gives me pause — is the idea that I could make userID + noteID the primary clustered key. (userID first.) This would mean that notes don’t get added at the end of the table — but all notes for a given user would be stored together. I think. I don’t have an easy way to test this, but my gut says that adding notes at the end is the better way to do this, and I should just rely on that unique constraint index to make lookups fast.
This is where the value of knowing your environment comes in. If I had five years, say, of SQL Server experience, I’d probably just know the best answer.
Well, the only way to learn an environment is to work in it, and that’s what I’m doing. Later I may look back and laugh. (Wouldn’t be the first time. Not even close.)
PPS If it seems like I’m working things out by writing them up here, it’s because I am.
Also, if it seems like I sometimes have to try everything before I get to an answer I like — well, that’s often also true.
PPPS Why did 53-bit integers seem weird enough to bug me? I have to plan for all options. There’s always the chance that another developer helps with this code — web app, for instance. Or we could make the API open. (Doubtful, but I have to plan for craziness.) Would the 53-bit limit be observed? It’s a data corruption bug waiting to happen. Even though it’s one little thing — an upper limit on note IDs — it’s too much complexity. Sync needs to be as simple as possible.
Ensembles 1.0 ships:
Today, Ensembles is a framework worthy of consideration if you want to add sync to a Core Data app. As you would expect from a 1.0 release, it is robust and stable, and is already shipping in a handful of apps.