Git replay

I recently had a need to run a script on every revision along the master branch of a git repository. Not finding any simple way of achiveving this with the builtin git tools, I knocked up the following script, which I call “git-replay”. The gist, which will be updated as I find problems/add features, is at

#!/bin/bash

# Takes a repository and script from the command line
# and executes the script for each git log entry in reverse chronological order

# Use temporary files of the following format
TMP_TEMPLATE="/tmp/gitreplay-XXXXXXXX"

# Validate command line parameters
if [ -z $1 -o -z $2 ]; then
echo "Usage: gitReplay.sh REPOSITORY SCRIPT"
exit
fi

if [ ! -x $2 ]; then
echo "'$2' does not exist as an executable file"
exit
fi

REPOSITORY=$1
SCRIPT=$2

# Generate and move to a temporary folder
tmp_loc=`mktemp -d $TMP_TEMPLATE`
pushd $tmp_loc > /dev/null

# Checkout the repository, and bail out if it fails
git clone $REPOSITORY . 2>&1
if [ $? -ne 0 ]; then
echo "Could not checkout repository"
popd > /dev/null
exit;
fi

# Determine all the hashes of the repository history
hashes=`git log --reverse --pretty=format:%H`

# For every hash, move to that revision and pass it to the repository
index=0
for hash in $hashes; do
git checkout $hash
bash $SCRIPT $hash $index $tmp_loc

if [ $? -ne 0 ]; then
echo "Early terminating based on script return code"
break
fi

let index=$index+1
done

popd > /dev/null
rm -rf $tmp_loc

Six to Eight development: What you might think is simple…

This is part two of my series on writing Six to Eight, an iOS Stack Exchange client. If you haven’t read part one, I urge you to head to it now. In this part, I’ll discuss creating the first part of the “Core functionality” that users of Six to Eight will see. We’ll look at why I made it the way it is, and how it uses the StackExchange API.

Six to Eight supports all the Stack Exchange sites in the network. You “subscribe” to the sites you’re interested in, optionally focusing your attention on one person by “tracking” them. As such, the first view seen by new users of Six to Eight will be one requiring them to setup an initial subscription. You might think this should be easy, and perhaps it should be. However, for me, it turned into one of the most complex pieces of design and development in the app.

This “Subscription creation” view, which also allows them to edit existing subscriptions, has to accomplish three things:

  1. Let users pick a Stack Exchange site
  2. Let users decide if they want to track somebody, and allow them to pick that person
  3. Just get out of the way as quickly as possible for users who just want to play with the app

Users have also got to be able to experiment with this setup a bit. If they’re tracking a user on one SE site, and want to switch to another, it’d be annoying if they had to re-select that user (assuming that the user is also using the newly selected site).

Let’s have a look at my UI attempt (see on Vimeo):

The UI uses a normal iPhone table view to present the site list, and user tracking toggle. This will, hopefully, present a large number of sites in an familiar fashion. The user selector allows people to easily find their own user within the selected site, and the re-association function saves them from having to repeat this task when they experiment. The UI is nearly all non-modal, so users can change their minds and toggle options without needing to wait for data. Importantly, if they just want to close the UI and start playing, they need only wait for the sites list to load, then tap “Save”.

So, to the implementation. For what looks like it should be a reasonably simple presentation of data, the code that runs this view is the most complex in the entire app. This is mostly to do with the number of possible states the UI can be in, coupled with the desire to keep things happening in an asynchronous fashion – lots of background work is occurring to update the data and verify it whilst the user selects their options.

Initially, the view turns up in a “Loading” state – there are no sites known, no user selected. We can’t do anything until we know what sites exist in the StackExchange network. This information comes from the StackAuth API endpoint at http://stackauth.com/1.0/sites, and Six to Eight fetchs it via my CoreStack CSAuth class. We request the data, and place the UI into a “Loading” state whilst we wait for it to arrive. Once our delegate is called to indicate that the CoreStack background loader has the data, we can update the UI with it, and transition into a “Select a site” state, where we default to the first available site and to tracking no user.

The entire UI is stitched together from this type of state transition, so we might as well represent it as a finite state transition diagram. In this…

  1. Grey indicates the UI is fully operational to the user
  2. Red indicates an error state, with suitable modal error message
  3. Blue indicates a modal state, where the UI is displaying a “Working” dialog (like the “Look for John Doe on SESite”).
  4. Green indicate “complete” states, where you can return having created a new subscription
  5. Purple indicate other view controllers
  6. Solid arrows are state transitions activated by the user
  7. Dashed arrows are state transitions activated by the controller

Six to Eight Site subscription state diagram

The goal is to start in the left hand purple state (where site subscriptions are listed), and get back there via one of the green states. For clarity, I’ve omitted all the “Cancel” links back to the Site selector state – all grey and green states can get there.

Moreover, these are not all the possible states of the view controller, only the UI. Internally, the view controller can be both not waiting for or waiting for data from the Stack Exchange API, including the sites list, user check data, and site icons. This isn’t even all the UI – this is just the core view (and even then, excluding each “Waiting for icon” state). The slide down user selector has it’s own internal UI state, and is activated from the two grey states in the centre of the diagram.

In all states other than blue, the UI is responsive to user interaction. Ideally, there would be no blue states, however in v1.0, I had no time to expand the state management to feasibly handle the feedback and changes needed to make user checking and re-association non-modal.

From the perspective of the API, re-association is easy. Whenever we start to track a user, the user selector dialog returns a structure obtained from the API endpoint api.sesite.example.com/1.0/users/filter?SEARCH, (CSUserDescription from CoreStack) that indicates the user name, their identifier and an association GUID. This association identifier is unique across all accounts tied together in the Stack Exchange network. When the user toggles the selected site, if we have such an association GUID, we display a modal dialog whilst we use the stackauth.com/1.0/users/GUID/associated method to find all the sites this user has associated themselves with. We search this list for the newly selected site, and either fail to find it (and so disable user tracking and display an error), or we find it and update the tracking data. The modal display is then hidden.

So, how did I do with respect to the 3 point goals initially set above, and the more general goals set out in part one?

As shown by the lack of solid arrows in the diagram between the two states and the first green state, the user can get a site up and running by just waiting a few seconds then pressing “Save”. This will setup the first reporting Stack Exchange site from the StackAuth API, without tracking a user. Similarly, the user is in control of the UI in all non-blue states, which represent the vast majority of interaction time with the UI. They can can cancel the setup, or change their settings. I think this “super fast initial setup” and responsive UI are key points in creating a good app.

The re-association feature is another subtle but important consideration for a touch based UI. We could leave it out, and whenever a user who’s tracking someone changes site, they could just re-search for their user. However, this requires more fiddly typing and network access – both best avoided on the iPhone. The StackAuth re-association API lets us save them the effort. Despite the fact that this feature is, in truth, rarely going to be used, I think the extra effort was well worth it. The fact that the Stack Exchange API allowed it with reasonable ease is testament to the quality of the feedback/design process used by the Valued Associated over at Stack Overflow Inc.

Now, if you’ll excuse me, I’m going to re-associate with an ale.

The development of Six to Eight

For the last couple of months, my spare time “relaxation” programming has been the development of an iPhone app for the Stack Exchange platform. For those not in the know, Stack Exchange is an expert community presented as a Q & A site. Users gather reputation for asking and answering questions. Examples you might know are Stack Overflow and Super User – though with the advent of Area 51, they’re adding new sites at a fair clip.

They’ve recently release a JSON based API for accessing their sites, and as an avid Stack Overflow user and Apple fan, I’ve wanted a pocket client for ages. Thus, Six to Eight was born. It’s my first “Release quality” iOS development, and I thought some people might like to know how it went – both other Stack Exchange API users, and other iPhone developers.

I set out to provide a good quality application. One that felt at home on my iPhone, was useful, and provided a good foundation on which I can build in the future. As such, I focused on 4 key developmental areas:

Core functionality: It might seem obvious, but way too many apps are little more than web browsers without a decent set of browser controls. With Six to Eight, the two key ideas are “Your Stack Exchange, in your pocket” and “Your Stack Exchange user, in your pocket”. I’ve tried to design an app that provides those things in a fast and iPhone consistent experience. Everything should feel like it belongs on the iPhone – iOS like UI idioms, user input via taps, most input data being derived automatically. Users also expect a certain level of polish, which I hope I’ve delivered.

Network error handling and recovery: Coding for the iPhone, you’ve got to be ready to handle, at any time, the loss of network connectivity or the degradation of network performance to that of a 14.4k modem. Just finished parts 1 and 2 of a critical 3 part request? You can bet on a decent number of your users losing their connection just then. Six to Eight underwent a lot of testing to ensure it remaines usable even on a GPRS connection, handles network drops gracefully, and reports them to the user as best it can.

Asynchronous UI: Blocking your UI whilst you wait for data is a cardinal sin. It makes your app feel unresponsive, and renders it unusable on a slow connection. Six to Eight performs all data collection and processing in the background, streaming it into the UI as it arrives. This way, you get to see what’s been downloaded so far, without waiting for every request to complete. You can also cancel any request you’re bored of waiting for.

State preservation: iPhone apps are often asked to quit with no notice, for inbound phone calls or because the user pressed “Home”. I tried to ensure that whenever you quit, both data and UI state are persisted so you reload back to the same state you left from.

Six to Eight is approximately 14,000 lines of code. Of course, this measure by itself is useless – but how much of this code is devoted to each part of the application is, to me, interesting. The following percentages are just my estimates based on how much time each feature took and the rough code footprint, as one can’t easily untangle the mix of the above ideas.

Core functionality: 40%. Less than half the program is needed to provide the functionality I wanted. This includes talking to the Stack Exchange API, processing the data, presenting it in the UI and responding the UI events. If I was programming for a desktop machine with a guaranteed fast network connection, I could have basically stopped here at around 5000 lines.

Error handling and recovery: 10%. This includes propagating errors around the system, presenting them to the user, and designing the UI so it can present partial data where missing data is in an error state.

Asynchronous UI: 25%. This includes designing the UI as a state machine that can present data incrementally no matter what order it arrives in, as well as the concurrent download and processing of data from the Stack Exchange API.

State preservation: 25%. This includes the ability to serialise and de-serialise all forms of Stack Exchange data and the state of each view within the UI.

In my next post, I’ll talk about one part of the core functionality I rather like, from both the Stack Exchange API and iPhone development perspectives.

Apple (hates) Developers?

As anyone who trawls the technology sites knows by now, Apple have as of OS 4.0 banned the use of anything other than C/++ or Objective C/++ for iPhone or iPad development. Effectively, this bans Adobe’s Flash to iPhone targeting technology, MonoTouch and several other techniques. I can think of only two (tenuously) justifiable reasons for this.

  1. Perhaps, during the beta, they want code written in C (and derivatives) to aid in debugging framework bugs. Rather than “My App doesn’t WORK!!!!! (Using MonoTouch Vxxyy)”, they can get a repro using a stack they know and control top to bottom. In this case, I can see it as justified.
  2. If it’s specifically to target cross-compilers as a business decision, to ensure that iPhone only gets “exclusives”, then I suppose it’s their prerogative. It will probably backfire long-term, as their market share isn’t that dominant. “Browser, Palm, Android + all else” or “iPhone” makes the iPhone a secondary port target versus a first class platform.

Gruber gives a third, pretty sensible conclusion – that they just don’t want to hand control of their platform to another vendor, and become a commodity target – just another checkbox inside Flash alongside “Web” and “Android”. If developers started using these meta development tools, all of the iPhones advantages vanish – it’s just another smartphone that runs Flash based applications.

For Apple the platform holder, this makes sense. For developers, any of the reasons are overly restrictive. This is, at best, ethically dubious without inconveniencing real development (which is all going on in Objective C already). At worst, it’s indicative of an ongoing schizophrenic pathology within Apple management where, despite winning ongoing customers through applications, they are happy to slap developers around just to see how far they can go before everyone decamps to Android.

We already exist with the capricious Application Review policy whereby you’re playing Russian Roulette with your development time – there’s no way to know that tell which perfectly innocent development choice will lead to rejection. Add this new restriction, and one has to wonder what choice made today will be retrospectively banned tomorrow. It’s an odd strategy, as there are plenty of innovative developers who are now scared of targeting the platform, whilst coding sweat shops keep flooding the channel with $0.10 an hour developer dross; developers who will use whatever they’re paid to use, be it Objective C or Flash. Pump enough stuff in, and some of it leaks through.

I can take two solaces from this personally. First, it literally can’t ever sensibly apply to OS X – the historical baggage makes this kind of restrictive move possible only on the closed and dominant iPhone platform. Even if they ever decided to make a “Mac App Store”, there would always have to be a standard development path lest they lose the thousands of platform portable apps that they current have (and really, OS X is still missing large swathes of desktop software that Windows has). Second, it makes my Objective C skills more valuable.

I still think it’s a poor move technically and ethically, but the iPhone platform has always been rife with these land-mines. Whether it turns out to be a good move from a business perspective, the market and time will tell.

A few notes on Windows Phone 7 Development comparisons

I’ve exchanged a few e-mails with Shawn Burke, who’s writing the “iPhone SDK vs. Windows Phone 7 Series SDK Challenge” series on his blog (which I highly recommend you read). Those reading my previous posts will know I’ve taken him to task for a few of his comments concerning the “number of lines” comparison between the iPhone and WP7. I just want to clarify a couple of my comments – not at his request, just to set the record straight.

I mentioned that Expression Blend is not a free product, but he’s mentions that “A basic version of the tooling for WP7 (including Blend) will always be free”, which is good news – the XAML designer in Visual Studio frankly sucks, and having to pay for a UI designer would be unwelcome. The main point of our brief exchange has been the (lack of) quality of Apple’s documentation, which I entirely agree with. The Interface Builder techniques I used in my implementations are not widely known, being documented in an obscure corner of the IB Manual, and it’s understandable that someone just arriving on the platform would think they didn’t exist. It’s also undeniably true that the iPhone tutorials are more complex than they need be to achieve the target application.

To clarify my opinion on this, I believe the tutorials are not going for the “ideal” MoveMe application – they’re covering a gamut of techniques people will need whilst developing for iPhone. Is the pacing of the tutorials quite right? Would a more “drag and drop” IB approach in the first few be preferable? I don’t know. Given the number of applications in the App Store, it certainly seems most people are getting to grips with iPhone development. Though given the quality of most of them, it’s fair to say there’s scope for improvement.