Usability Study:

We had 10 participants perform the following 3 use case tasks:

Use case 1:
"For each of the following residences, determine the distance to the nearest subway terminal:"
735 Dufferin st
200 Harbord st
600 Huron st

Use case 2:
"Of the following residences, which offers the closest combination of subway terminal, 
pharmacy and bank?  Also, I care twice as much about the distance to the subway terminal 
as I do the distance to other services."
50 Millicent st
200 Harbord st
600 Huron st

Use case 3 (extends 2):
"I want to add another address, 111 Davisville Ave, and I want to remove pharmacy from the 
services and add public school.  Also, I think the distance to the subway should be 3 times 
as important as the distance to other services.  Now which residence is ranked best?"

These use cases tested users abilities in:

1. Address entry
2. Service selection
3. Weighting selection for a service
4. Browsing set of results
5. Browsing top result
6. Revising service selection
7. Revising service weighting
8. Revising address input
9. Mousing over residence for detailed result information

We used screen capture software to capture their interaction with our application, as well as recorded audio. We present the low-resolution captures here (we are somewhat distant from the microphone, so you may have to turn up volume to hear):

Participant 1: Canadian, single, no children	Participant 2: Canadian, married, no children	Participant 3: Canadian, single, no children
Participant 4: Canadian, single, no children	Participant 5: Canadian, single, no children	Participant 6: Canadian, single, no children
Participant 7: Canadian, married, no children	Participant 8: Canadian, married, children	Participant 9: Not Canadian, married, children
Participant 10: Not Canadian, single, no children

Participants unanimously found the system easy to operate, and of potential use. The following bullet points summarize the more common suggestions for improvement (this list will be current as of the time of the study, November 28, 2009):

• A great idea we missed - Service Clustering: it is disadvantageous that the closest services may be spread far apart from each other (someone may be walking and wants to minimize the distance they have to travel amongst some set of services, thus the distance between the closest services themselves may be of importance to the user)
• The tab structure as well as the BACK/NEXT buttons - confusing, should do one or the other - while the back/next is "setup wizard"-like, the application layout is not, and further the tabs promote a random-access type of navigation instead of being purely sequential
• Upon address entry, camera should move to frame the residences, instead of leaving the task to the user
• Relative importance sliders: need labelling really badly, highly unintuitive to a first-time user
• Results table: not obvious results were sorted, "what does 'Value' even mean?", better headings (e.g. "rank")
• 3D Map interaction: perhaps use more familiar interaction for map, instead of the present Maya-like controls (e.g. left drag does a pan instead of orbit, mouse wheel to zoom)
• A "CLEAR" button, to start fresh
• Highlighting the house when the user hits "GO", as moving the camera to centre on it works most of the time except when there is more than one house close to the centre of the screen
• Using different colours for the arcs emitting from the houses, based on the service type
• The best result should have an additional visual cue (a gold star, a trophy hovering above it, etc.)