Some Mobile Services Tips

Here are a few tips, in no particular order, on using Mobile Services. I have months of experience, but not years, so treat as such.

Custom APIs Only

I don’t use the scripts attached to the tables. Instead I create all custom APIs. This lets me define exactly what callers can do.

Create SQL Tables Using Command Line

If you create a table using the portal, you get an id column that’s a string, plus some extra columns that you don’t need if you’re using custom APIs only.

Make sure you have the azure command line tool installed. (It’s a Node package.) Do azure mobile table create --help to see your options.

Note the --integerId flag — that’s what you want. If you use that flag when creating a table, you get a table where the id column is a bigint, and no extra columns.

Database queries and updates

It’s pretty easy to use table.where, table.insert, table.update — but I use request.service.mssql.query for almost everything, for reasons of efficiency and control.

Advanced Database Admin

If you have to do more than the portal will let you, use the Azure SQL Server command line.

Hopefully you won’t need to — but it’s not scary. (It’s much like using the sqlite3 command line.)

You May not Need the iOS Framework

If you’re using Facebook or Twitter authentication, you need the framework.

But since we’re not, we don’t. Here’s the thing: connecting to the service is really easy without the framework.

You call your methods as you’d think, with URLs like https://yourappname.azure-mobile.net/custom/api — and you need to do just two extra things:

  1. Add an x-zumo-application header with the app’s public access key.

  2. For authenticated calls, add an x-zumo-auth header with the JWT that Mobile Services returns.

After that it works like any other REST + JSON interface.

Create Two Mobile Services

One for production, one for staging.

Set up Git

You don’t want to paste your scripts in the browser. Set up SCM (Git) instead and deploy that way.

(I’ve asked for Mercurial support. I realize that doesn’t matter to most people.)

Use Shared Scripts

You can have shared scripts that your API scripts call. When you set up Git, you’ll note that there’s a shared folder. Include those (via require) from your API files, as in:

var something = require('../shared/something.js');

If you have a function somefunction you want made available, in something.js add a line like this:

exports.somefunction = somefunction;

This will make it available from the outside as something.somefunction.

Don’t Store Binaries in the SQL Database

Use blob storage instead.

Before Deploying

Make sure your files compile. (You should have installed Node.js on your system.) Type node filename. It’ll tell you if there are errors.

To Read

Josh Twist has a bunch of blog posts worth reading: The Twelve Days of Zumo.

If you need an associated website

You can create an Azure Node.js site. You don’t get all the nice parts of Mobile Services, but it’s still Node. That associated website can make API calls to your API server.

Mark on DHH on TDD

Mark Bernstein, Driven:

Methodological fundamentalists descend on technology from time to time. Hell, TDD and agile started as a rebellion against an older tech fundamentalism. As a profession, we’re suckers for The Word and want to know The True Way. Sometimes, the methodologies do help: structured programming, literate programming, strong types, weak types, objects, patterns, functional, agile: it’s all been good for us.

But, perversely, these helpful little ideas often become modes of punishment.

Surviving UI Programming

I’ve often wondered why UI programming is less fun than everything else.

My theory is that it’s because so much of it is arbitrary, single-use programming — I’m making a specific screen or view work the way it needs to, and there isn’t much that can be re-used. There are only rarely generic problems to solve. (Those generic problems are mostly solved — that’s what UIKit is for.)

Luckily, UI programming has gotten easier. These days we have view controller containment, custom view controller transitions, auto-layout, and, especially, appearance controls.

One of the nicer additions is editing static table views in storyboards. I’m doing that right now.

To make UI programming as smooth as possible I try to use all the things designed to make it easier. Stick with standard controls as much as possible. Use storyboards.

But it seems inevitable that there will be some point where I get frustrated. Right now it’s this: I have a grouped table view, a static table, and I want to add a section footer. (Not a table footer.) There’s nothing built-in to storyboards for this. (Please correct me if I’m wrong.)

To make matters worse: I don’t know how big the footer will be. It should be a container view that holds a UILabel with text that uses a custom font. The UILabel, and the container view, should be the exact height needed and no taller or shorter.

I think that the modern way is to add auto-layout margin constraints to the UILabel and the container view will tell me its proper size. But I worry that I’ll spend half a day on this, not get it to work (even if there is a way to get it work), and I’ll end up punting: doing a view that draws text in drawRect, and getting its size by measuring the text. I don’t want to do that, but I’ve done it plenty of times before, and I know it works.

So I think my problem is really anxiety. I expect to get frustrated.

And I have reason. Just last night I was attempting to add a button to a navigation bar. I created the button, then made a UIBarButtonItem with the button as the custom view, then added it as leftBarButtonItem.

The button worked — you could tap it and the right thing would happen — but the text of the button was never actually visible. I spent an hour, and I couldn’t explain it. (I finally just gave up.) (I even tried embedding the button in a container view and adding that instead. I gave the container view a background color. So I ended up with a red rectangle in the navbar but still no button text. Yes, the button was a subview of the container view.)

I’m not new to this. I had an app on day one of the App Store. And if you look at Vesper I don’t think you’ll think, “Oh, this app was clearly made by someone who gets anxious when he writes UI code.” There’s a ton of custom UI in Vesper, and I’m very proud of it.

But still: here I am, today, looking at something that should be simple — some text as the footer of table section — and wondering how long it’s going to take me, and wondering if I’ll have to give up on the right way and do it the old-fashioned way.

It’s possible that this just means I should spend more time doing UI programming.

Update 3:50 pm: John Siracusa on Twitter:

@brentsimmons The problem with UI prog. is you’re building on top of the deepest, most opaque stack of code you didn’t write and can’t see.

Exactly right. I think the only remedy is experience. But, the thing is, I have a bunch of experience.

Sponsorship Changes

I made two changes to make inessential sponsorships a better value:

  • Volume discount: $650/week for two or more weeks.

  • Sidebar link that appears on every page for the week. (Averages about 40,000 views per week.)

If you’re interested, get in touch.

Storyboard Advice

It’s a couple years old, but UIStoryboard Best Practices still has good advice.

I’m trying to use storyboards for the first time. At first I had one big storyboard that was really hard to work with, even on a 27" screen. Now I’m breaking it up into separate storyboards.

Testing Singletons

The iOS Unit Testing blog, in Testing Singletons, suggests making singletons by choice — not enforcing singleton-ness by overriding allocWithZone or init.

Agreed. Good practice.

App Store Improvements

David Smith has good ideas Towards a Better App Store.

I’m worried about a possible future where the only way to make money (outside of the super-rare mega-hit) is with 1) websites with recurring revenue and 2) Mac apps sold outside the Mac App Store.

The iOS app, in this future, would be a checklist item. Something you have to make but that doesn’t directly make money.

The problem with that future is that making a good iOS app is expensive and takes time. If quality can’t be justified with revenue, then quality will suffer.

(There are some people who’d argue that this future is already here.)

A better App Store would mean (by definition) that quality apps do better. And the future I worry about could be avoided.

NSNotificationCenter and Threads

Lap Cat Software: NSNotificationCenter is thread-safe NOT.

(Via Michael Tsai.)

I have a simple rule: posts notifications on the main thread only.

That’s not a guideline — it’s a rule. No exceptions.

Mobile Services and Blob Storage

This is a quick write-up of how I got this working.

Here’s the situation:

  • I want to store images and other binaries in Azure blob storage (which is like Amazon S3) rather than in a SQL database or NoSQL table.

  • I want the client to upload directly to blob storage — I don’t want the image data to pass through the API server.

I struggled with this. For a while I was looking at the REST API and building a SharedKey. It wasn’t until I found Chris Risner’s page on this from last year that I found what I needed. (Thanks tons to Chris for writing this up.)

So this tutorial exists in case you’re in the same boat.

I’ll assume you already have a Mobile Service and know how to set up custom APIs. I’ll talk about the server side mostly, since you already know how to use NSURLSession and friends.

Get a blobService

Set up a storage service if you haven’t already. (New > Data Services > Storage.) A storage service is NoSQL table storage, queues, and blobs.

Add the storage service’s name and primary access key to your Mobile Service’s key storage (see the Configure pane, under app settings). Key names such as BLOB_ACCOUNT and BLOB_ACCESS_KEY are fine. Whatever you want.

In your code:

var azure = require('azure');

The azure module is available without you having to do anything. Just require it to use it.

The azure module gets you a blobService:

var blobService = azure.​createBlobService​(accountName, key, host);

accountName and key should come from your Mobile Service’s key storage — request.​service.​config.​appSettings.​BLOB_ACCOUNT, for instance.

The host is accountName + '.blob.core.windows.net'.

Whenever you’re going to do something with blob storage, get a blobService.

Creating a container

blobService.​createContainerIfNotExists​(containerName, function(err)…

This creates a private container. (You can create public containers too, but that’s outside of the scope here. It’s easy, though.)

Listing files

blobService.​listBlobs​(containerName, function(err, results)…

The results are in JSON. You could return those to the client, or filter them to just filenames or whatever the client needs.

Deleting a file

blobService.​deleteBlob​(containerName, filename, function(err)…

Uploading

Here’s the fun part. You don’t want the image binary to go through the API server. Instead, the API server generates a temporary shared access URL that will allow the client to upload directly.

First use createContainerIfNotExists. (I assume you have some way of deciding what the container names should be. They could be based on a person’s user ID, for instance. Depending on the security needs of your app, it may be a good idea to hash the container names, so they’re not personally identifiable.)

You need a sharedAccessPolicy.

var sharedAccessPolicy = {
  AccessPolicy: {
    Permissions: 'w',
    Expiry: azure.date.​minutesFromNow(5)
  }
}

Then you generate the URL to return to the client:

var sasURL = blobService.​generateSharedAccessSignature​(containerName, filename, sharedAccessPolicy);
var accountName = request.​service.​config.​appSettings.​BLOB_ACCOUNT;
var urlForUploading = 'https://' + accountName + '.blob.core.windows.net' + sasURL.path + '?' + qs.stringify(sasURL.queryString);

(Somewhere above this you need var qs = require('querystring').)

Return this URL to the client.

The client can use an NSURLSession​UploadTask. Use the PUT verb. Add Content-Type and Content-Length headers. (It’s a good idea to add Content-MD5 too, but not required.)

Downloading

It’s almost exactly the same as uploading, except that sharedAccessPolicy​.AccessPolicy.​Permissions is 'r' instead of 'w'.

I have the server respond with a redirect to the URL.

That’s it. Pretty easy.

Glassboard Design Review

Jared Sinclair looks at Glassboard’s design:

When starting a new design project, the hardest part is finding the project’s voice. There are no easy solutions, and more than one right solution. You discover the voice through a repeating cycle of experimentation and refinement, one following the other, over and over. You try lots of things that turn out to be dead ends. With time and effort you accumulate enough familiarity with your project until one of the possible paths seems obvious.

Atom XHTML Content Considered Jerky

Dr. Drang writes about varying support for Atom’s content type="xhtml".

I strongly disliked this part of the Atom spec.

The thing about this feature is that the HTML tags are included as naked tags in the XML. That is, they’re not escaped or placed in a CDATA section.

From the good doctor’s example:

<content type="xhtml" xml:lang="en-US" xml:base="​http://lancemannion.typepad.com/​lance_mannion/">
<div xmlns="​http://www.w3.org/​1999/xhtml"><p><em>Barnes & Noble. Thursday. April 17, 2014. Six forty five p.m.</em></p>…
</content>

Just look at it. This feature is a giant invitation to screwed-up feeds. The HTML — which is probably a blog post, typed by a human — has to be valid XML. People writing scripts to generate these feeds have to make sure they can turn that HTML into valid XML.

Feeds are already screwed-up often enough. This could only make it worse.

The second issue: how can a parser handle this? What an RSS/Atom parser really wants is everything between <content> and </content> as a string.

I remember writing this code for NetNewsWire, which still supports this feature. (I checked. I have no idea if they’ve touched my code or not.)

I’m going from memory, but I’m pretty sure this is what I did.

NetNewsWire used libxml2’s SAX parser, which meant it gets called when the XML parser encounters the beginning and end of a tag and when it encounters a range of characters. There was no easy way — that I know of; correct me if I’m wrong — to tell the parser to treat everything inside a tag (<content> and </content>) as a string when there are XML tags inside that tag.

So I wrote the Atom parser to rebuild the HTML. It maintained a string and pushed stuff on it. If it encountered a tag and (optionally) attributes, it would create a string version and push it. When it encountered characters it would push those. When it encountered the end of a tag it would create a string version and push that. Once the </content> was reached then it had the entire string.

The end result was equivalent HTML, but not necessarily character-for-character the same, since little things like quote characters could change.

It worked. But boy was I coding angry that day.

PS Luckily I didn’t see this feature used very often. Still had to write the code, though.

PPS I still wonder if there’s an easier way. (Using a SAX parser. No way would I give up SAX. Performance and memory use considerations require SAX.)

PPPS For a great list of other ways feeds get screwed up, see Brian’s Stupid Feed Tricks.

Azure SQL Server Command Line

Database stuff is easy using Mobile Services — it’s a piece of cake to do almost everything you need via the browser or via the Azure command line.

Almost everything. It’s great until you hit the things that aren’t supported. Then, well, then there’s this weird browser-based SQL Server portal you could use.

But that’s not true anymore. Now there’s sql-cli, a Node module that provides a command-line interface to SQL Server. It’s so much like using the sqlite3 command-line interface that I’m at home right away. (It’s written by Hasan Khan, who’s been a huge help. Just about daily, lately.)

UIActivityViewController and Things

NSHipster looks at UIActivityViewController, introduces IntentKit (which I hadn’t heard of before), and reminds us to hope for remote view controller APIs.

App Camp For Girls in Seattle

App Camp For Girls is coming to Seattle. Cool news for my town.

CocoaRadio One

I’m on episode one of Justin Williams’s new podcast CocoaRadio. We talk syncing.

[Sponsor] Microsoft Azure Mobile Services

Mobile Services is a great way to provide backend services — syncing and other things — for your iPhone, iPad, and Mac apps.

If you’ve been to the website already, you’ve seen the tutorials where you input code into a browser window. And that’s an easy way to get started.

But don’t be fooled: Mobile Services is deep, too. A mobile service is a Node.js site, and you can write JavaScript in your favorite text editor. Deploy via Git. Write unit tests using Mocha. Administer via the command line. All on your Mac.

You can get started today with a free trial.

Vesper Sync Diary #15 - Server Testing

I don’t write many tests as I go along, because I know I’ll change the design and do a bunch of refactoring. Tests would have to be rewritten and rewritten again.

I like the idea of test-driven development, but in practice it means more wasted time than I’d like. (For me. I’m sure it works great for other developers, but not every developer’s the same.)

I just ran cloc on the API server. It’s 1,545 lines of code. It’s small, which I take as a good thing. By the time I’ve finished writing tests it should be closer to 3,000 lines of code.

(If writing a Mac app is like writing a novel, and writing an iOS app is like writing a short story, then writing an API server is like writing a poem. It has to be compact, and every line has to have a reason to be there and has to do its job perfectly. In an iOS app you can slightly screw up an animation somewhere and just fix it later. On the server nothing can be slightly screwed up.)

My process

First step is to go through every single file and add all the easily-testable functions to a to-test list. That’s most of them.

As I’m going along I find some small refactorings that can be done that will make more things testable, so I jot those down and then do those after going through all the files. And update the list of functions to test.

Next step is to write the tests — using Mocha — from the ground up. Do the low-level simple utility routines first.

(I used Josh Twist’s post on unit testing Mobile Services scripts to help me get started.)

Live tests

Unit tests are indispensable, but there are two issues:

  • They don’t actually hit the server itself.

  • There are some things that are very difficult to test — because they need all the context of the server.

For instance, it’s easy to test the logic for merging notes, but it’s hard to test end-to-end — sending some notes to the server and making sure they really end up merged and stored in the database.

I’ve been thinking about what to do here and I don’t see any existing frameworks that fit the bill. (Tell me if I’m wrong.)

So here’s my thinking. I’ll write code in the client app that calls the server and runs tests with some test data. Ideally these would be XCTestCase tests — with special junk for doing async tests — but if that doesn’t work, if it’s a special configuration of the app instead, I won’t stress about it. (Well. Weeeeell. I might stress about it. And bang on it until it works. Because it really ought to work. It bugs me so much that XCTestCase doesn’t have built-in support for async tests.)

This is made just a little more complex by the fact that I want to run tests against both the production and staging servers. That’s a small complexity, though. The bigger thing is the code to set up the environment on the server so that it can run the tests.

Here’s what I like, though: every line of code is testable. All the logic is testable. This is so different from client apps, where it’s hard-to-impossible to test everything.

Given the complete lack of margin for error, given my need to not constantly worry about the server, this is a wonderful thing.

PS I counted. First step is to write unit tests for 53 functions.

Scaling and Performance

In the recent Accidental Tech Podcast John Siracusa pointed out that I had talked about scaling when I was really talking about performance.

John’s right, and I know better.

In the vernacular sense of scaling the two things are related, but I should use more precise language.

So I’ll be more precise: my main goal is to maximize performance and minimize the amount of resources used. My second goal is to design so that I have scaling options if needed.

(Performance is related to scaling only in the sense that it lessens the need to scale, but it doesn’t solve the problem of scaling itself.)

Here’s what I’m doing for actual scaling:

  • Using a system that allows for multiple web servers to be automatically created as needed.

  • Using a SQL Server database with a maximum of 150 GB.

  • Using blob storage for binaries (images), which can scale forever (presumably).

The weakest link here is, I think, the database. If I have to, I can split up the data into separate databases. There are just four tables — accounts, deletednotes, notes, and tags — and each of those could be moved into a separate database. (There are no joins and no foreign key constraints.)

I don’t expect to ever come anywhere near doing that, so I’m not actually planning the migration steps. But it’s in the back of my head that there’s a non-zero probability.

And if I have to go even further, I can. The biggest of the tables, by far, is the notes table. I can break that table out into separate databases, separated by userID. (Which is an integer. Each database would store notes for a range of userIDs.)

And, if that’s not enough, the remaining tables (accounts, deletednotes, and tags) could also be broken out the same way.

I don’t think I’ll ever have to do any of this — but I can, if I need to, with only small code changes.

The web server may also be a weak link. The way to solve this one — not that I think I’ll need to, because I can run a bunch of instances of the server — is to create separate API servers. Each endpoint could be a separate server with its own set of instances. This would require a small code change on the client, but I’d see this coming and make sure it gets done in plenty of time.

PROCEDURE

Check out the code listings in More Macintosh Toolbox (for instance) for a reminder of the role Pascal used to play in our world.

Pascal’s cool.

Craig on X 10.10

Craig thinks Helvetica Neue will be the next Mac system font, and suggests you start testing your apps with it now.

Archive