“App: The Human Story” Screening in San Jose

Here’s the scoop. It’s Sunday, June 4 at 5 pm. There’s a panel afterward with a bunch of people from the movie (including me).

You can get tickets. You should get tickets — the event benefits App Camp for Girls.

Plus I think you’ll enjoy it. :)

JSON Feed

I was hesitant, even up to this morning, to publish the JSON Feed spec.

If you read Dave Winer’s Rules for standards-makers, you’ll see that we did a decent job with some of the rules — the spec is written in plain English, for example — but a strict application of the rules would have meant not publishing at all, since “Fewer formats is better.”

I agree completely — but I also believe that developers (particularly Mac and iOS developers, the group I know best) are so loath to work with XML that they won’t even consider building software that needs an XML parser. Which says to me that JSON Feed is needed for the survival of syndication.

I could be wrong, of course. I admit.

Feed Reader Starter Kit

See my RSXML repository for Objective-C code that reads RSS, Atom, and OPML. I’ve done the work for you of supporting those formats. Go write a feed reader! Seriously. Do it.

I planned to have a JSON Feed parser for Swift done for today, but other things got in the way. It’s coming soon. But you probably don’t actually need any sample code, since JSON is so easy to handle.

Feedback so far

Feedback has been interesting so far. Some questions on the GitHub repo need answering.

Some people have said this should have happened ten years ago, and other people have said that they hate how developers jump on the latest fad (JSON).

And some people really like the icon:

Microformats

One of the more serious criticisms was this: why not just support the hAtom microformat instead? Why do another side-file?

My thinking:

My experience as a feed reader author tells me that people screw up XML, badly, all the time — and they do even less well with HTML. So embedding info in HTML is just plain too difficult. In practice it would be even buggier than XML-based feeds.

And there are other advantages to decoupling: a side-file can have 100 entries where there are only 10 on an HTML page, for instance. A side-file can have extra information that you wouldn’t put on an HTML page. And yet, despite the extra information, a side-file can be much smaller than an HTML page, and it can often be easier to cache (since it’s not different based on a logged-in user, for instance).

Microformats sounds elegant, but I don’t prize elegance as much as I value things that work well.

Frontier Diary #8: When Worlds Collide

I spent the weekend making a bunch of progress on the compiler. It has two pieces: a tokenizer, which I created by rewriting the original C code (langscan.c) in Swift, and a parser.

The parser in OrigFrontier was generated by MacYacc, which is similar to Yacc, which is similar to Bison, which is on my Mac. The thing about the parser is that it’s C code, and the rest of the app is Swift.

How do you bridge the two worlds? Easy answer: with Objective-C, which is a superset of C and which plays nicely (enough) with Swift.

So I renamed langparser.y — the rules file that the parser generator uses — to langparser.ym so that Xcode would know to treat the generated parser source as Objective-C. I edited it slightly, not to change the grammar rules but to change how nodes are created (as return values rather than via inout).

I also made my CodeTreeNode class, written in Swift, an Objective-C class so that it would be visible to my Objective-C code.

And then, finally, I started a build…

…and then it stopped with an error because the parser places my CodeTreeNode in a C union, which isn’t allowed in ARC.

Crushed.

* * *

I think I have three options:

  1. Go down the rabbit hole of figuring out how to get the parser to work with ARC.
  2. Go with the flow: have the parser generate nodes that are, as in OrigFrontier, C structs. The last compilation step would be Objective-C code that translates that tree of C structs into a tree of CodeTreeNode objects, and then disposes the C-struct-node-tree.
  3. Write the parser by hand, in Swift.

My thinking:

I could waste a ton of time on #1, and bending tools in that way can be pretty frustrating work when they refuse to bend.

With #2 I’d feel a bit weird about the redundancy: building a tree and then building a copy of that tree with a different type of object.

My heart tells me #3 is the answer. After all, I’ve already done the tokenizer. How hard would it be to parse those tokens into a code tree? I could skip C and Objective-C altogether and stay in Swift. And it would be so fun. (Because that’s precisely the style of weirdo I am.)

* * *

But the real answer is #2. Writing a parser by hand would take way longer than I think. Given enough tests, it shouldn’t be a huge source of bugs, but still.

The thing about #2 is that yes, it’s redundant, it’s doing more work than it needs to, ideally — but my bet is that it would still be so fast that you wouldn’t be able to tell the difference. Computers are so good at this kind of thing. It’s not like reading files or networking; it’s just in-memory traversal and creating/releasing things.

You remember in Indiana Jones that guy with the twirling swords, and Indy gives that look and then just shoots him? The second option is the Indiana Jones solution.

Update 2:05 pm: Two people have already written me to recommend ANTLR. So I will definitely give that a look. It might be exactly what I need.

Frontier Diary #7: Pretty Much Everything Throws

A script can throw an error, either intentionally (via the scriptError verb) or by doing something, such as referencing an undefined object, that generates an error.

OrigFrontier was written in C, which has no error-throwing mechanism, and so it worked like this: most runtime functions returned a boolean (for success or failure), and the return value was passed in by reference. If there was an error, the function would set a global error variable and return false. The caller would then have to check that global to see if there was an error, and then do the right thing.

This was not unreasonable, given the language and the times (early ’90s) and also given the need to be very careful about unwinding memory allocations.

But, these days, it seems to me that Swift’s error system is the way to go. There’s just one downside to that, and it’s that I have to do that do/try/catch dance all over the place, since pretty much any runtime function can throw an error.

Even the coercions can throw, so last night I changed the Value protocol so that asInt and so on are now functions, since properties can’t throw (at least not yet).

The extra housekeeping — the do/try/catch stuff — kind of bugs me, but it’s honest. I considered making script errors just another type of Value — but that meant that all those callers have to check the returned Value to see if it’s an error, and then do the right thing. Better to just use Swift’s error system, because it makes for more consistent code, and it makes sure I’m catching errors in every case.

It also means I’m not multiplying entities. A Swift error is a script error, and vice versa.

* * *

Working on this code is like applying the last 25 years of programming history all at once.

A completely different type of error is a bug, and I’m certain to write a bunch of them, because that’s how programming goes.

That’s where unit tests come in. Frontier has long had a stress-test suite of scripts — you’d launch the app, run that suite, wait a while, and see if there are any errors. This was critically helpful.

But OrigFontier didn’t have unit tests at the C code level. The new version does. (Well, I’ve started them anyway.) This means I can more easily follow Rule 1 — the no-breakage rule — and can also more easily follow Rule 1b — the don’t-break-Dave rule.

PS I’ve added a collection page for the Frontier Diary, as I did with earlier diaries. There’s a link to it in the footer of every page on the blog.

Frontier Diary #6: Ballard, from the Parallel Universe

In another universe I didn’t decide to port Frontier — instead, I started over from scratch on an app inspired by Frontier.

In that universe, the new scripting language, descended from UserTalk, is called Ballard. And it’s documented.

My Microblog

I’m on Manton‘s cool new microblogs system. Here’s where you can follow me, once you’re on the system: http://micro.blog/brentsimmons.

And here’s my microblog: http://brent.micro.blog/. (Which you can read using RSS, whether you’re on the system or not.)

I wrote about three-quarters of my own single-user microblog system — and then stopped because I didn’t feel like running a server and because Manton’s service is so good.

Frontier Diary #5: Values and Progress on the Language

I put the Frontier repository up on GitHub.

(The build is currently broken. This is bad discipline, but since it’s still just me, I forgive myself. Sometimes I run out of time and I just commit what I have.)

The repo has my new code and it also contains FrontierOrigFork, which is the original Frontier source with a bunch of deletions and some changes. The point is to give me 1) code to read and 2) a project that builds and runs on my 10.6.8 virtual machine.

The original code is in C, and the port is, at least so far, all in Swift. In the end it should be almost all in Swift, but I anticipate a couple places where I may need to use Objective-C.

Here’s one of the Swift wins:

Values

Since Frontier contains a database and scripting language, there’s a need for some kind of value object that could be a boolean, integer, string, date, and so on.

Original Frontier used a tyvaluedata union, with fields for the various types of values.

This is a perfectly reasonably approach in C. It’s great because you can pass the same type of value object everywhere.

Were I writing this in Objective-C, however, I’d create a Value protocol, and then create new value objects for some types and also extend existing objects (NSNumber, NSString, etc.) to conform to the Value protocol. This would still give me the upside — passing a Value type everywhere — while reducing the amount of boxing.

But: this still means I have an NSNumber when I really want a BOOL. Luckily, in Swift I can go one better: I can extend types such as Bool and Int to conform to a Value protocol.

This means passing around an actual Bool rather than a boxed boolean. I like this a ton. It feels totally right.

Other topic:

Language Progress

I’m still in architectural mode, where I’m writing just enough code to validate and refine my decisions. A couple days ago I started on the language evaluator — the thing that actually runs scripts.

It works as you expect: it takes a compiled code tree and recursively evaluates it. It’s not difficult — it’s just that it’s going to end up being a fair amount of code.

I’ve done just enough to know that I’m on the right path. (The Swift code looks a lot like the C code in OrigFrontier’s langevaluate.c. See evaluateList, for instance.)

The next step is for me to build the parser. I thought about writing a parser by hand, because it sounds like fun, and it would give me some extra control — but, really, it would slow me way down, so forget it.

OrigFrontier generated its parser by passing a grammar file — langparser.y — to MacYacc (there was such a thing!), which generated langparser.c.

I’ll do a similar thing, except using Bison (which is compatible with Yacc). Or, possibly, using the Lemon parser generator instead. Either way, I’ll want the generated code to be Objective-C. (Well, mostly C, but with Objective-C objects instead of structs.) (I don’t know of a generator that would create Swift code.)

This is completely new territory for me, and is exciting.

(Almost forgot to mention: I’ll need to write a tokenizer. This means porting langscan.c. I’ll need to do this first, since the parser generator needs it. So this is the real next step.)

Save $300 on CocoaConf Next Door

My pals at CocoaConf asked me to remind you that the Early Bird sale ends in two weeks for CocoaConf Next Door — the one taking place in San Jose during WWDC.

I’ll be there. At least in the afternoons.

Check out the speakers list. Yummy, chewy, nutty speakers list.

Frontier Diary #4: The QuickDraw Problem and Where It Led Me

In my fork of Frontier there are still over 600 deprecation warnings. A whole bunch of these are due to QuickDraw calls.

For those who don’t know: QuickDraw was how, in the old days, you drew things to the Mac’s screen. It was amazing for its time and pretty easy to work with. Functions included things like MoveTo, LineTo, DrawLine, FrameOval, and so on. All pretty straightforward.

These days we have Core Graphics instead, and we have higher-level things like NSBezierPath. QuickDraw was simpler — though yes, sure, that was partly because it did less.

* * *

I was looking at all these deprecation warnings for QuickDraw functions and wondering how I’m ever going to get through them.

I could, after all, convert all or most of them to the equivalent Core Graphics thing. But sheesh, what a bunch of work.

And, in the end, it would still be a Carbon app, but with modern drawing.

* * *

So I thought about it from another angle. The goal is to get to the point where it’s a 64-bit Cocoa app. All these QuickDraw calls are in the service of UI — so why not just start over with a Cocoa UI?

The app has some outlines (database browser, script editor, etc.), a basic text editor, and a handful of small dialogs. And all of that is super-easy in Cocoa.

Use an NSOutlineView, NSTextView, and some xibs for the dialogs, and we’re done. (Well, after some work, but not nearly the same amount of work as actually writing an outliner from scratch.)

In other words, instead of going from the bottom up — porting the existing source code — I decided to start from the top down.

I started a new workspace and started a new Frontier project: a Cocoa app with Swift as the default language.

Then I looked at the existing source and thought about how to organize things. I came up with this:

  • Frontier — App UI
  • UserTalk.framework — the language
  • FrontierVerbs.framework - the standard library
  • FrontierDB.framework — the object database
  • FrontierCore.framework — common utility functions and extensions

I like using frameworks, because it helps enforce separation, and it helps in doing unit testing. And frameworks are so easy with Swift these days.

Hardly any of this is filled-in yet. I’ve got the barest start on FrontierVerbs. Ted Howard, my partner in all this, is taking UserTalk.framework and FrontierDB.framework.

In the end, it’s possible that no code from the original code base survives. Which is totally fine. But it also means that this is no quick project.

At this point I should probably put it up on GitHub, since it’s easier to write about it if I can link to the code. I’ll do that soon, possibly on the weekend.

Frontier Diary #3: Built-in Verbs Configuration

Frontier’s standard library is known as its built-in verbs. There are a number of different tables: file, clock, xml, and so on. Each contains a number of verbs: file.readWholeFile, clock.now, and so on.

Most of these verbs are implemented in C, in the kernel, rather than as scripts. At the moment, to add one of these kernel verbs, you have to jump through a few hoops: edit a resource, add an integer ID, add to a switch statement, etc. It’s a pain and is error-prone.

So I want to re-do this in Swift, because I’m all about Swift. And I want adding verbs to be fool-proof: I don’t want to remember how to configure this every single time I add a verb. Adding a verb needs to be easy.

My thinking:

  • Give each table its own class: ClockVerbs, FileVerbs, etc.
  • Have each class report the names of the verbs it supports. These need to be strings, because we get a string at runtime.
  • Run a verb simply by looking up the selector, performing it, and returning the result.

To make things easy and obvious, I think it should work like this: the selector for a given verb is its name plus a parameter. Then there’s not even a lookup step.

Each verb will take a VerbParameters object and return a VerbResult object.

dynamic func readWholeFile(_ params: VerbParameters) -> VerbResult

The flow goes like this:

  1. We have the string file.readWholeFile.
  2. We see the file suffix and so we know we need a FileVerbs object.
  3. We check fileVerbs.supportedVerbs (an array) to see if readWholeFile is in the list. It is.
  4. We construct a selector using the readWholeFile part of the string and we add a : character: NSSelectorFromString(verbName + ":")

This is great! We’re almost home free. Then we run the verb:

if let result = perform(selector, with: params) as? VerbResult {
    return result
}

That doesn’t work. We get:

Cast from 'Unmanaged<AnyObject>! to unrelated type 'VerbResult' always fails

Nuts.

* * *

It was so close.

In Objective-C this would have worked. And obviously, apparently, I still think in Objective-C.

I investigated some other options. At one point enums were abused, because there’s always, in Swift, an enum-abuse step. But everything I tried was more code and was more error-prone, and my goal here is to improve the situation.

I think, in the end, I’m going to do something that looks kind of ugly: a switch statement where the cases are string literals.

switch(verbName) {
case "readWholeFile":
    return readWholeFile(params)
…
}

“Nooooo!” you cry. I hear ya.

My experience as an object-oriented programmer tells me this: if I write a switch statement, I blew it.

And my experience as a programmer tells me that string literals are a bad idea.

But the above may actually be the easiest to configure and maintain. Each string literal appears only in that one switch statement and nowhere else in the code. And the mapping between a verb name and its function couldn’t be more clear — it’s right there.

(Yes, instead of using a string literal, I could create a String enum and switch on that. But that’s actually more code and more room for error. I’m going to have to type those string literals somewhere, so why not right where they’re used?)

It does mean that readWholeFile appears three times in the code (the string literal, the call, and the function itself), and in an Objective-C version it would appear only twice (in a supportedVerbs array and the method itself).

But. Well.

I’m torn between shuddering in abject and complete horror at this solution and thinking, “Hey, that’s pretty straightforward. Anybody could read it. Anybody could edit it.” Which was the plan all along.

And I get to stick with Swift, so there’s that.

But, sure as shootin’, some day someone’s going to come across this code and say, “Brent, dude, are ya new?” And I’ll send them the link to this page.

* * *

Update the next day: well, the performSelector thing would work, if only I’d known about Swift Unmanaged objects.

Joe Groff told me how this works.

Here’s the gist: the Unmanaged<AnyObject> just needs to be unwrapped by calling takeRetainedValue or takeUnretainedValue. Once unwrapped, it can be cast to VerbResult.

All this means that I can use my original design, which is great news.

* * *

Update April 25, 2017: I ended up using enums after all. See MathVerbs.swift for an example.

Archive