User testing: Guerilla Methodologies
Over the years, I have become a strong advocate of “guerrilla” user testing. While your testing methodology needs to meet your objectives, the benefits of guerilla testing are obvious. Most importantly you can do more testing more often. This allows the UX designer to present iterations to users and gather feedback through out the process.

Ultimately I like to aim for 5-6 users based on Jakob Neilson’s research into the optimum number of user to test for. This is also a really nice number – because from a management perspective you can test this many users in one day. I like to aim for a 20-30 minute test once an hour. This gives me time to consolidate my notes, check my emails before the next participant arrives. But don’t be fooled even this kind of testing takes time to prepare, recruit users, agree with stakeholders on testing objectives, reporting and respective design iterations.

Why leaving my notes until tomorrow just didn’t work
Despite being a very good note taker, I was finding that by the end of the day, there was a lot of information to digest. Plus the overwhelming sense that I wanted to get as much of the information down while it was all fresh, before sleeping on it. However, we all know that user testing is a long and tiring day. So it became clear to me that there needed to be a better way to streamline this. It seemed clear to me that this would be to do some of the work as I went along.

Introducing the analysis wall
So I created an analysis wall which I would go to and work through after each session while the content was fresh in my mind. The analysis wall, pictured, should be out of the view of your user group. I hid mine behind a white board, but if you have a corridor, or separate meeting room available that would work well. As you can see I’ve simply printed out the interface onto separate A3 sheets with plenty of room to write.

User Testing Analysis Wall

Also importantly, coloured textas….
The first time I created my analysis wall, it became clear that my blue biro for all users was not particularly helpful. It became a mush. It also became really difficult to connect the user to the finding. However, when using coloured textas I was able to highlight that all of the dark green findings are from a 35 year old woman, with plenty of internet experience, but no mobile web experience and no iphone experience. This is a really important aspect of the finding.

Using coloured textas for documenting user findings on the analysis wall

Outcomes/ benefits
The first time I conducted the analysis this way – I was blown away by how much time it actually saved me in pulling together my report. Using the analysis wall had meant all I had left to do is pull the findings together, offer recommendations and report in. The time saving was considerable.

Secondly, I was blown away by how I was able to engage and illustrate the “big things” to the project team immediately by inviting them to come and check out the analysis wall that afternoon. While this doesn’t replace the need for documentation to share with stakeholders, it gives an immediate and powerful overview of the user feedback. In the past I had found that stakeholders tended to observe only one session, leaving them focused on the findings of that one session only.

Most importantly, what you end up with from the analysis wall is a visual illustration of each user’s feedback among the group, and you can easily see where the views of each user align and where they diverge.

Links:
Jakob Neilson: Why You Only Need to Test with 5 Users
http://www.useit.com/alertbox/20000319.html
Andy Budd: Guerrilla Usability Testing
http://www.slideshare.net/andybudd/guerilla-usability-testing