Last week I organized a Testing Dojo. I wanted to try a dojo for a long time and I was quite disappointed that Markus Gärtner was ill at Agile Testing Days so the testing dojos were cancelled. What do you do when you want to experience a testing dojo? Right! You just get a bunch of people in a room and try it yourself! To make this dojo a success I read the website Testingdojo.org from start to finish and scanned testing-challenges.org for a good challenge. I also skyped with Markus for an hour to discuss the dojo. For me the goals of the dojo were:

  • Experience what a testing dojo is (useful? fun?)
  • Learn from group
  • Practice stuff we learned in RST last year

At our dojo 20 people showed up: mostly testers but also a business analysts and a developer. After ordering pizza for all, I did a short intro presentation. I explained what a testing dojo is, what the rules are for the dojo and the mission for the evening. I wrote down the rules on a flip-over and put that on the wall:

  • Work in pairs
  • We recognize 3 roles:
    1. Driver / tester
    2. Recorder / note taker
    3. Observers (audience)
  • Switch roles after 5 minutes:
    1. Tester –> audience
    2. Recorder –> tester
    3. Audience –> recorder
  • Everybody in audience (observers) take notes using checklist
  • Audience does not interfere with testing, unless they are asked

Mission:
Test parking calculator Gerald R.Ford International Airport using SFDPOT and the Test Heuristic Cheat Sheet.

Observer checklist:

  • What happened?
    • Testing
    • Communication
    • Collaboration
  • How do they communicate?
  • Are they listening to each other?
  • Who is in charge?
  • Also note emotions
  • What path do the testers take?
    • depth
    • breadth
    • Explore first: what do we have?
    • When does what happen
    • What happens if they find a bug?

I gave the group a sneak preview of the parking calculator. I also showed them the webpage with the information on the car parking and the parking rates. While preparing the dojo I thought the participants would like to do some preparation while eating pizza. I figured they could think about their approach and maybe write down some test ideas or other stuff they thought that could be useful. But the group wanted to start right away. There were some questions about the exact purpose of testing this application. I told the group, while playing the role of product owner, the goal is to inform me if this calculator needs adjustments.

In the room the participants were sitting in a u-shape all facing the screen were the beamer projected the display of the laptop in front of the room. The pair behind the laptop was facing the group. I think it’s not easy to test when 20 people are watching you and you are sitting in front of the room. In the back of the room were 2 flip overs with the parking rates. We decided that everybody would have 3 minutes of testing time and after 3 minuted we would switch roles. After three pairs had finished their testing we stopped for a short retrospective to discuss the progress so far. We noticed that nobody was in charge really and communication between the pairs was minimal. They were trying stuff, without having a clear strategy or approach. The first pair could get away by telling they were exploring the application to see what they had in front of them, what it could do, etc. The other pairs had some sort of strategy but couldn’t explain. A discussion on the further approach took place and the group decided to create a list on a flip over with items that needed testing. I noticed that SFDPOT wasn’t used explicitly and the strategy followed by the group was almost fully focussed on testing functionality.

After a flip over with the “test strategy” was created by one of the participants, testing was more focussed. But still I noticed that it was difficult for the participants to test in pairs (communication) and keep focussed without scripted tests (structure). None of the pairs debriefed or communicated with the next group. To fill this “gap” I asked every pair after their time was up, if I could “tick off” one of the items from the test strategy. One of the participants created a list of test ideas before he took his place behind the laptop. It was fun to see a lot of different styles and approaches. I also noticed that being observer is a rather difficult job. Not a lot of notes were taken on observations despite the checklist on the wall. Most of the participants focussed their observations on the testing and the application being tested, rather then on the people testing and their communication and collaboration.

After all pairs had their go testing, we did a retrospective. Below the feedback from the group.

Lessons learned:

  • Group was too big, a group of up to 10 people is better.
  • More time to test: with a smaller group it’s probably better.
  • Give the group a bit more structure. It is helpful when the starting point is clear.
  • Stop more often to evaluate. Idea: 1. Mission – 2. Test – 3. Self debrief – 4. Group feedback – 5. Test again

It was a great evening, although it didn’t went as smoothly as I hoped. But we learned a lot about testing dojos and I think it was a successful evening. I hope the attendees went home with the same feeling. In a small group we discussed the dojo somewhat further and the reactions were enthusiastic. There is still enough room for improvement, as always: practice makes perfect! For a first time I think this was quite a successful evening. A second dojo is already planned. I am looking forward to it.

Other notes made during the test dojo: