Category: Skills

Do we need testers?

May 17, 2020 / Huib Schoots / 2 Comments

The first version was first published on linkedin on February 5, 2020 titled “Do we need testers? No! Do we need skilled testing? Yes!”. I added my new insights to this blogpost.

Last week I did a talk at Agile, Testing & DevOps Showcase in Amsterdam. My topic was “Testing in modern times“.

In agile and especially DevOps approaches the motto is: automated everything! Companies like Facebook claim they do not have testers at all. Microsoft only has SDET (software development engineers in Test), other companies are T-shaping developers to do the testing. New kid on the block is AI and machine learning, that will definitely replace testing I hear people claim. What is really happening globally?

Do we no longer need testers? I am not sure anymore. Why? Because testers have a bad name. If you cannot automate nowadays, you are no longer valuable, people say. That makes me sad. I think skilled testing is super important! Testing informs decisions about value, quality and risk by learning about the product (and a serious professional tester also will give you insight in the status of the project or team they are working in). Modern technology and tooling is reducing the need for dedicated testers. Not by replacing testers, but by reducing certain types of risks we used to test. More and more people are getting obsessed by “automate everything”. Sadly even some testers I meet are obsessed by trying to automate everything they do…

How can we make valuable software for our clients? I believe that personal leadership and collaboration ultimately makes the difference. The quality of software is crucial nowadays. Therefore I like to focus on an integrated quality approach. As a tester, a mentor and a coach, I help people and teams learn effectively and continuously develop with attention to sustainable adaptability that leads to improving (team) results and way of working. It enables teams to create more customer value by building quality solutions!

In IT we need insights in risks. Risks and value. For that we need to learn continuously. And I think we need smart people who do skilled testing, determined to find problems that matter. Teams need to create insight and overview in the risks we take by creating, releasing and using (IT) products. Often people with excellent testing skills excel in doing this… I hear new job titles as “Quality coach” and “Quality Engineer”. Is this the way to go? Well if that solves the problem of “automation obsession” and a lack of testing skills in teams? I am game.

So far the original post on linkedin. Recent events make me doubt if we ever get to a point where we do not need dedicated testers.

Dan Ashby reacted on linkedin:

"It's a really interesting question. My take: yes, we need testers... because we need skilled testing, and the Dev communities haven't kept up with what skilled testing is and how to do it. One thing too: Facebook use offshore testers (lots of them via a tester contracting company - I know people that work at FB and at the 3rd party testing company). Also MS have done a U-turn too, and they also employ testers again as well as SDETs. They also have test manager roles again too. Maybe lots of the big companies have hit that realisation that they needed good testing (and hence needed the testers who have those skills)? If so, hopefully the smaller companies that tend to imitate the big companies will soon follow suit. 😁"

I like what he says here: “we need testers… because we need skilled testing.” Although there are some developers who have really good testing skills, many are not interested in testing nor learning to do skilled testing. So I think he has a point there. But dedicated testers alone do not solve the problem. We need better testers and better thinking about testing too!

I see a huge fixation on “test automation” and this is causing us to lose connection with the human, social purposes of software development and testing. The essence of software development is that during development, we learn about what we need, what the customer really wants and how the product we are building actually works. This is research and development, learning along the way, and needing sense making and feedback to get it right.

This “Vision on the Future of Software Testing” has nothing to do with skilled testing. This shows that even an institute like ISTQB does not understand IT in general nor testing. That makes me so sad!

Test approaches like TMap and ISTQB are neglecting the human aspects of testing. They try to approach testing with mechanistic thinking and are not dealing with the complexity and uncertainty that developing software and dealing with people brings. Leading test experts told me we cannot make testing too complicated or else testers will not understand.

Recently the new TMap book was published called “Quality for DevOps teams”. Reading how TMap deals with risk analysis and test strategy gives me goosebumps. The risk analysis is just a list of quality attributes with a simple calculation (possible impact x chance of failure) and based on the number we assigned a “risk class” (high, medium, low). And based on the risk class we assign the test intensity in dots. See for yourself what it looks like here and here. Now let’s look at some quotes from the book that made my eyes roll.

Chapter 47 “Experience-based testing”

“Exploratory testing is an experience-based approach of testing, the most important approach of experience-based testing in our opinion. We distinguish coverage-based and experience-based testing. Others use terms like scripted testing and free-style testing, but we prefer the division in focus on either experience or coverage.”

All testing that you do uses experience, because there is no way you can shut it off. And doing any test will give you some coverage. I guess they just do not know how to talk about coverage in a way that makes sense. Why make this strict distinction in only two categories? There are so many more ways to classify test techniques (note: what TMap calls “approach” is called “technique” in BBST). There is no strict distinction between test techniques. See slide 62 of BBST Test Design “Every test addresses all of these. A specific technique typically addresses 1 to 3 of them, leaving the rest to be designed into the individual test“. Personally I like the way BBST classifies techniques looking at the driving ideas behind the testing.

“The main downside of applying error guessing is the lack of documentation. Therefore, tests are not reproducible. This may result in a developer not being able to investigate an anomaly, the tester not being able to retest a fix, and the test cannot be added to a regression test set.”

Why do tests need to be reproducible? If a tester is capable of telling or showing the developer what goes wrong, you do not need any documentation. I think with enough product knowledge, it is not that difficult to retest. So we are solving a problem in the wrong way, aren’t we?

Chapter 48 “Is there any value in unstructured testing?”

“Any testing lacking a plan containing what to do and what to expect of a system, or lacking preparation of the test, is unstructured. This is also called ad-hoc testing. Some people see a great advantage in unstructured testing because, as they say: “You can start testing right away.” That is, without “losing” any time on preparation.”

The only structure TMap knows is plans and test cases. Structure is “the arrangement of and relations between the parts or elements of something complex”. Michael Bolton wrote about this here. Testing is about learning and learning involves mental models. TMap seems to have no attention at all about how people learn. Deep learning in the beginning is a confusing process, but it gets clearer along the way. By letting it rest (defocus), we give our brains the chance to process the learned information and integrate it with the models we have in our heads. So good (mental) models are important. These “mechanistic test approaches” forget the whole learning part.

“When you have an IT system that is of good quality, the testers do their unstructured testing and don’t encounter any faults or failures. Can they now say the quality is good and that there are no significant risks? Did they really measure the quality and risks? No, the only thing they can truly say is they did “some” testing and didn’t find any problems. However, they cannot explain which requirements or quality risks have been covered. They are not even sure which parts of the system have been covered.”

This is an interesting approach taken by TMap here which is called an “appeal to ignorance”. The testers cannot explain so the approach must be wrong! But is the approach the problem or are skills of the testers the problem here?

7 Rules for Positive, Productive Change – a book review

May 17, 2020 / Huib Schoots / 0 Comments

At Agile Testing Days 2019 during the keynote “I can’t do this… alone! A Tale of Two Learning Partners” by Lisi Hocke and Toyer Mamoojee I got inspired by their story about learning pacts. During the keynote Nicole Errante and I started a learning pact too. In our first call we created a plan. One of the books I added to our pact was “7 Rules for Positive, Productive Change – Micro Shifts, Macro Results” by Esther Derby. In this blog post Nicole and I share a summary and our learnings from the book.

Huib: The book is about change and from the first page on it resonated with me. Esther opens with: “People hire me because they want different outcomes and different relationships in their workplaces. My work almost always involves change at some level…” and that is exactly what I do and have been doing for many years now. While reading and discussing the book with Nicole I recognized so many things from my own experience. The introduction talks about change as a social process! Work and life in general is heavily influenced by social processes and everything we do has major social aspects to it. An aspect that unfortunately often is underexposed especially when people want to be in control. Best practices do not work in complex situations in which we find ourselves in IT often, we know that from the work of Dave Snowden and the cynefin framework. This is why I got so inspired by Context-driven testing years ago. Finally I found people who were taking the human aspects in testing and IT seriously. Not trying to approach (testing) problems with mechanistic thinking, not seeing IT as technology centered, not striving for certainty but being okay with uncertainty and a community where human interaction and feelings played a prominent role in solving problems. This book takes the same approach with change. Change is a social thing. Esther’s book hands you the interventions she calls rules to improve and help change to happen. The rules are heuristics or guidelines that will help change to happen.

Nicole: Change in life, whether work or personal, is inevitable. The philosopher John Locke said “Things of this world are in so constant a flux, that nothing remains long in the same state.” But we, as humans, tend to be resistant to change; we like stability, routines, and the known. However, maybe we would be less resistant to change if it was implemented in a way that considered this human side of it. That’s one of the things I love about Esther’s book – it constantly keeps people in the focus of the change process. The success of the change is not just about the process but about the people in it. I have been lucky enough to work at the same company for the past 13 years but that means my experience is a bit more narrow than Huib’s when it comes to experiencing change in the workplace. However, a few years ago our company wanted to make the move from a very waterfall-based software development process to one that was more agile. I think most people at my company would agree that the change was more painful and took more time than anyone expected. It is through the lens of that change that I read this book – what could we have done to make the change process better? And what lessons can we learn for future changes?

Summary: The introduction ends with Lessons Learned from a project Esther did. Many of those I recognized and made me want to read the book even more. The lessons are:

Skill and will aren’t always the problem
Training is useful and necessary, but it’s not sufficient
Standardizing nonstandard work may make matters worse
Long feedback loops delay learning and improvement
Observed patterns result from many underlying influences

The first chapter deals with change by attraction. If you try to force change upon people, they will react with resistance. Mandating change makes people feel a loss of control and they have no personal buy-in to the change. Esther says “At best, coercion, rewards, and positional authority result in compliance, not engagement…” You don’t want people just going through the motions, you want them actively involved and eager to give things a try. In order for change to happen, things need to be learned and other things need to be unlearned. There is no best practice that works in every situation in knowledge work, quite the opposite: we need to experiment to find out what works and what doesn’t. It is a matter of responding to people, adapting to their needs and attracting and engaging people instead of pushing and persuading.

Strive for Congruence

Congruence is an alignment of a person’s interior and exterior worlds, balancing the needs and capabilities of self, others, and context. Ignoring other people’s needs and capabilities is probably the most common cause of incongruence. When this incongruence happens, you are in a stress state. When people are stressed, it is hard to think, learn, or engage. You cannot have successful change when learning and engagement are suppressed. Congruence is essential for change by attraction. Congruence contributes to safety, which is essential for people to solve problems, to learn and to speak up about mistakes and things they don’t know. Being empathetic will help you understand where someone else is coming from and what they have to lose by changing, thus avoiding ignoring the context of others in the process of change. Empathy helps people feel safe and understood. Empathy and congruence go hand-in-hand and are essential for making long term changes. At the end of chapter 2, Esther lists a couple of questions you can use to be more congruent.

Honor the past, present and people

When implementing change, it is important to show respect to existing belief systems, the experiences and knowledge people have, and the effort people have made to keep things going with the system currently in place. Build trust and relationships before coaching others. People seldom think that they themselves are wrong. They also may want to improve but most do not want to hear from an outsider that they are doing it wrong. So we have to choose our language with care. Remember that while you have ideas of how things can be better, the people you want to change know things that you don’t know that will be important in this process. By acknowledging and exploring the negative space of change, we prevent unpleasant surprises along the way. Again Esther has a great list of questions to discover what lives in this negative space. People don’t resist change, they respond to its implementation. You can learn from reasons behind the responses to help adapt the change. Use Transformational Communication (inquiry, dialogue, conversation, understanding) instead of Convincing and Persuading (advocacy, debate, argument, defending) to gain openness, trust, and shared understanding. Finally, don’t take for granted what works by only focussing on the problem. Build upon what already works.

Assess what is

Every system is perfectly designed to get the result that it does (W. Edwards Deming). Change starts from where you are now and paying attention to the context increases the change problems will get solved. How did the existing conditions in the organization produce the current patterns and results? This chapter introduces 3 techniques intended to look beyond symptoms and find influencing factors:

Containers, differences, and exchanges (CDE), it describes the three conditions that determine the speed, direction, and path of a system as it self-organizes. This will help discover both the formal structures and invisible structures that factor into the behavior of the organizational system.
SEEM Model: Steering, Enabling-and-Enhancing, Making. This model shows the different perspectives of people in the organization on the basic set of concerns each company has: how to achieve clarity so people know what to do, what conditions are needed for people to do good work, and what productive constraints will streamline decision making and guide actions and interactions.
Circle of Influences: to see how the factors influenced one another and where to find virtuous and vicious loops. This method helps to find the many influencing factors that result in problems, instead of focussing too much on the problems itself. Find factors that have influence on several others as possible places to run experiments on.

Attend to networks

An organisation has a formal and an informal side. Informal social networks within the organisation have great influence and cannot be ignored, although they are not visible on the org chart. The most important social networks for change are those that people turn to and trust for advice. That is why Esther suggests to map the networks within the organisation to be able to use them productively and not to break them inadvertently. You can also enhance existing networks by reducing the number of hops between people that are not directly connected to reduce bottlenecks. Networks can also carry rumors. Esther suggests to capture them to find out what people worry about using a rumor control board. Finally, if people do not want to change something, just do it with other people that do (change by attraction). The resisters will probably follow when they see that other people are doing it. This is the fear of missing out in action.

Experiment

Solving big complex problems is not easy because many factors are involved and they cannot be solved independently. Trying to solve them in big changes will cause big disruptions and that’s risky. Small experiments foster learning and will engage the people you work with. Experiments are FINE (Fast feedback, Inexpensive, require No permission and are Easy). Landing zones make big changes small by defining intermediate states to which the organisation can evolve. After reaching the landing zone, you can reassess whether your bigger goal is still relevant and course correct as you need. Safe to fail probes are good examples of experiments. Find something that you can try without asking for budget or permission. Don’t worry about failing – keeping the experiment small means any risk should be contained and you can learn from what went wrong. Esther lists a great set of questions to assist in shaping the experiments and another set to test assumptions. Reflecting on what works in the experiments involves double-loop learning.

Guide and allow for variation

Knowledge work and complex organizations need to allow for variation and emergence to perform effectively. Unnecessary standardization will lead to inefficiency and suboptimal behavior. Coherence is more desirable than consistency and that is why Esther suggests using boundary stories: to help people focus on gaining a similar outcome. Boundary stories give people a guideline on reaching the outcome you want (and avoiding those you don’t) while allowing people to mindfully decide how to get there based on their unique situation. Also change will be evolutionary: small evolutionary steps towards the end goal. Landing zones are useful, so is a horizon map: a thinking tool where you start with the desired outcome and work from right to left filling in conditions and constraints needed for the change to take place. Since change is social, it requires changing habits of thought and cognitive frameworks. Change will happen if we manage to influence metaphors and narratives within the organisation. Explain the outcomes you want and why, then add some boundary stories as a guide to how to get there. This will allow people to refine the change based on their knowledge and experience, thus owning the change rather than being forced into it.

Use your self

People bring their personalities, characteristics, belief systems, and life experiences to work and this influences what they do and how they do it. This includes you, the person involved in bringing the change. Change is a social process which needs personal connection with the people involved. This works best using empathy, curiosity, patience, and observation. These skills can and must be practiced. A nice list of questions to help prompt empathy is given. Esther also supplies nice overviews with types of questions and how to focus questions to be curious and patient. These questions help to avoid why questions, which often make people defensive. The question and how you ask it, determines the answer you get. Making sense of your observations requires bias awareness and testing your observations. Be generous when trying to interpret the motivation behind what people do and the results they achieve.

Learnings Huib

It was fun to work with Nicole and read the book together talking about two chapters each time we met. Sharing our stories and experiences with change and discussing situations at work helped to understand what the book is about and to get ideas where we could try the things we were reading. The parts on empathy, curiosity and patience really resonated to me. Like I described in my blogpost “Mastering my mindset” I become more and more aware that growth, learning, improving and change needs empathy. I am working on that. This book gave me more tools and inspiration to get there. I already used several questions from the lists in the book and I enjoy using them. The landing zones are a great way to create small steps of change. I’m now working on a horizon map with a scrum master to get insight into what is going on in his team. I cannot wait to work with the team on the map.

Learnings Nicole

I agree with Huib that the method we used to read the book a couple chapters at a time and then discussing really was a fun way to read a book.It really helped reinforce the learnings of each chapter as well. Being able to share what we thought and our experiences helped not only make things clearer but also gave insight on where to apply it to our work. When our organization went through the big agile change, the main reason I thought it went rough was that people didn’t understand the reason behind the change. While that is true, Esther’s book also helped me realize there were other factors involved as well. The change we bit off was too big: we should have started where we were at and done smaller experiments to learn and adjust along the way. People’s knowledge, experience, and feelings should have also been considered in order to get them actively involved in the change. I look forward to being able to apply the lessons in this book to future changes in our organization. I also want to incorporate some of the questions from the book to help work through the day-to-day challenges that we face in our team.

Finally

The book is easy to read and has some great stories in there to illustrate the rules and lessons. It has many valuable and ready to use lists of questions and methods that will help in experimenting with change. Every chapter ends with a great set of takeaways which summarizes what you just read in different wording. We absolutely recommend this book to anybody dealing with change in their work.

The art of reflection

July 22, 2019 / Huib Schoots / 2 Comments

“Once is coincidence
Twice is striking
Three times is pattern”

October 19-21 2018 DEWT held their 8^th annual peer conference with “Developing expertise in software testing” as theme. I had the honour to open the conference with my experience report called “Mentoring and coaching to develop skills”. In the open season after my experience report and during the rest of the peer conference we talked about reflection on several occasions. I think one of the most important skills in learning is reflection.

My vision on learning
Learning is the process of acquiring new or adapting existing knowledge, behaviours, skills, values or preferences. Learning is much more than knowing: putting the learned into practice and gaining experience with it is important to truly internalise real knowledge and to gain skill. Learning must be linked to experiences from daily practice. By reflecting on knowledge and skills, so-called learning loops are created.

Learning is an ongoing process: the world is changing fast and to be excellent in your role requires many skills. So it is important to keep up! To learn effectively, learning should come from yourself, with intrinsic motivation and personal responsibility.

Learning requires a positive and open learning environment in which you can safely come out of your comfort zone to try new things. The safe environment gives confidence to make mistakes and to experiment with new knowledge and skills. It also demands a certain degree of challenge. How big the challenge is, is different for everyone. It is not that you always have to come out of your comfort zone radically. Right outside of your comfort zone, in your stretch zone, you learn best.

Effective learning requires focused learning with clear learning objectives and preferably evaluation criteria. It should be clear what you want to learn and how you can measure that. Where do you want to grow? And how do you know that you have made a step? By setting clear learning objectives, you focus on learning.

Levels of learning
There are different levels of learning: (ref: Joris van de Griendt)

In single-loop learning (improvement, behavioural improvement), it is about the visible and concrete behavioural level (what): you do something and that has a certain effect. This effect may be desirable or not, and based on that assessment, you can adjust your behaviour or not.
Double loop (reframing, behavioural renewal) If you want to change your behaviour permanently, there is double-loop learning: researching which patterns the behaviour involves (how). Which patterns and mechanisms are behind the behaviour? Which helpful or obstructing thoughts are involved? Insight into this can provide a more well-founded choice to change behaviour.
Triple-loop learning (transformation, behavioural development) goes even deeper. You involve your values, your purpose, the why question: why do I really want to change this behaviour? What important values in me support this and what stops me?

What is reflection?
Reflection is a process of exploring and examining ourselves, our perspectives, attributes, experiences and actions / interactions. It helps us gain insight and see how to move forward. (ref: University of Edinburgh).

When we reflect, we deeply consider something that we might not otherwise have given much thought to. This helps us to learn. Reflection is concerned with consciously looking at and thinking about our experiences, actions, feelings, and responses, and then interpreting or analysing them in order to learn from them. (ref: The Open University).

Reflection is looking back at your behaviour in a certain situation. You reflect on that situation by asking yourself meaningful questions to make you think about the situation. The difference between just thinking and reflection is the intention to learn. There is a difference between evaluation and reflection. Evaluation means making a judgement about something you did. For example: did you reach your goal? Did I do the right thing? While reflecting means creating a safe space to investigate behaviour without making a judgement with the intention to grow.

Like learning, there are different levels of reflecting:

Single-loop reflection focuses on behaviour and actions and is very close to evaluation.
Double-loop reflection means trying to get hold of underlying convictions that interfere with the adjustment of interaction and behaviour.
Triple-loop reflection is about motives and matters that touch on their own identity. There is often a relationship with issues at a higher level, that of the organisation or even of an entire system.

Iceberg

(image credit: Dutch Vision Institute)

Above the waterline behaviour is perceptible and statements are audible. But opinions, beliefs, feelings and emotions are not visible; they are below the waterline. These invisible elements, however, are often the motives for visible behaviour. An important part of the reflection will therefore consist of researching these deeper layers of the Iceberg.

By not addressing all layers within one’s competence management, you allow the coachee to act for incongruously (say A and do B, or vent a belief that contradicts his own motivation); just like external fragmentation (non-integration in the context), internal fragmentation (in the context of do-thinking motives) also leads to a real risk of energy loss.

Korthagen

(ref: How do I use the Korthagen reflection circle diagram?)

Korthagen’s reflection cycle is a tool or a strategy to be followed for learners to gain insight into their educational performance and to improve this. By applying this cycle step-by-step, one learns to systematically reflect a skill to be learned.

Phase 1: Describe the experience/situation you wish to reflect upon. What was the actual situation? You can do this by using the STARR(S) method: Situation-Task-Action-Result-Reflection-Strengthen (see appendix).

What did I have to do in this situation?
What action did I actually take?
What was the outcome of this action?

Phase 2: Looking back: What exactly happened?

What did I see?
What did I do?
What did I think?
What did I feel?

Phase 3: Awareness of essential aspects

What does that mean to me now?
What is the problem (or the positive discovery)?
What has all that caused? What does it involve?

Phase 4: Alternative methods

What alternative methods do I see (solutions or ways of making use of what I have discovered)?
What are their advantages and disadvantages?
What will I remember for next time?

Phase 5: Trial/action

What do I want to achieve?
What should I watch out for?
What do I want to try out?

Danger of thinking too much

Reflecting is an active activity that demands skills. It often happens that professionals think they reflect, while they are actually worrying.

This points out the important differences:

Worry

Involved in itself, looking from our own perspective, alone
Focused on mistakes
Focused on judging and condemning
Global approach
Mono causal approach

Reflection

Involved in the problem, also looking at a perspective outside of oneself, in contact with others
Focused on solutions
Focused on understanding
Analytical approach
Multi causal approach

Tips for reflecting

You can reflect on every situation and every problem that concerns you. You can learn a lot from that, but the pitfall is that you will be overwhelmed by the information and will keep looking back endlessly. Another danger is that you may feel that you are actually doing a good job – the work is going well, there is no criticism from colleagues or your supervisor – and so you see no reason to reflect. Yet it can also be very instructive to reflect on yourself and your way of acting.

The following tips can help you reflect:

Choose a concrete situation and look back on that specific moment and your course of action
Reflect regularly and ‘schedule’ at least once a week a reflection moment, preferably at a fixed time
Ask yourself open questions
Explain judgments about yourself; first see what really happened before judging yourself
Reflect in a methodical way, for example by going through a list of questions or use a reflection model
Reflect not only on problem situations but also on success experiences
Use feedback from others to reflect from that point of view
Read more about reflection to get inspired. This is a nice blogpost about reflection: How Self-Reflection Gives You a Happier and More Successful Life. For more inspiration read this: Tools to help you with self-reflection

Together you learn even more

It is already very instructive to consider your own functioning in this way, but it can be even more profitable if you do this together with others. Do not try to judge here either. Not about others, nor about yourself: you feel free to tell everything. For example, start doing intervision with peers or colleagues. More about intervision here: Intervision: what is it about?

Start a journal

Writing in a journal regularly (preferably daily on a specific time) helps to analyse your professional and personal growth. Journaling can give you a different perspective on things. Writing in your journal is a very useful tool to help you understand yourself and the world around you. Write down activities, thoughts, ideas, reasons, actions, techniques and reflections on specific topics or skills you want to improve. By writing in a journal you get an overview of your thoughts in which you can identify patterns. Journaling helps you to get thoughts and ideas out of your head but more important it enables making sense of things that happened. After doing something related to your learning goal, take notes on your observations, summarise facts and experiences. Also write down how it makes you feel.

More about reflective journaling in this article: Reflective Journals and Learning Logs.

Here are two helpful checklists to help you reflect:

Let’s stop talking about testing, let’s start thinking about value

October 18, 2018 / Huib Schoots / 6 Comments

This year Alex Schladebeck and I did two keynotes titled “Let’s stop talking about testing, let’s start thinking about value” at QA Expo in Spain and TestNet in the Netherlands. This blogpost has the most important points we made in our talk.

The keynote was inspired by some of our frustrations: “Testing is under appreciated” (Alex) and “Most testers are unable to explain what we do” (Huib). I wrote about my frustration back in 2016 already. This blogpost is about my frustration that most testers cannot come up with a decent definition of testing. And even worse: a big majority of the people who call themselves professional testers are not able to explain what testing is and how it works! They have trouble explaining what they are testing and why they are doing specifically the thing they are doing! How can anybody take a tester seriously who cannot explain what he is doing all day?

Alex’s frustration is that testing is not valued by others. Developers are seen as the rockstars of the project because they create the software that adds value. But why are testers often not valued?

Lowered expectations for testing expertise by stuff like ISO standards and ISTQB: I wrote about certification and standards before. ISTQB and standards put too much emphasis on process and documentation, rather than the real testing. By assuming there can be a standard, you say that there is one best way to organize and document your testing. But isn’t your test strategy heavily dependent on its context? When using standards we tend to focus on complying with the standard, and lose sight of the real goal. This sort of goal displacement is a familiar problem in many situations. Also, the idea that you can learn how to test is a couple of days of training is dangerous. Remember lesson 272: if you can get a black belt in only two weeks, avoid fights (Lessons Learned in Software Testing: A Context-Driven Approach by Bach, Kaner and Pettichord).
Avoiding controversy: nowadays more and more people advocate to be nice! I think that we confuse being nice, with being kind! An interesting article about this phenomenon is written by Marcia Sirota. Of course we need to respect other people, but to push the testing craft forward, we need to have firm discussions and disagree with others way more often. Being nice doesn’t help. Serious feedback does!
We devalue our own work by becoming tool jockeys: unfortunately there are too many testers (and teams) out there who focus too much on automation as much as possible. Why? Because they can! The testers in those teams are often so busy doing automation that they do not have the time to test anything…
We do not stand up for our craft: we do not fight back enough when other people say they do not need testers, or if they tell us how to do our jobs to name a few examples. We have to learn “testers self-defence: to stand up to people who try to dictate how do our jobs. We have to learn how to organize effective (and efficient) testing. And we need to learn how to talk about our work in a way others understand. This requires practice!
We do not learn or practice enough: testing is difficult! We have to deal with complexity, ambiguity, change and people. Testing is a craft, not something you do as a hobby. To become a craftsperson, you have to practice (also see my blogpost: a road to awesomeness).
We don’t know how to talk about testing: as said before: how can anybody take a tester seriously who cannot explain what he is doing all day? To be really valuable, testers need to learn to talk about their testing in a way others understand and find valuable.

So looking at these things, are we okay with this? I don’t think so. But what can we do about it? We are trapped in this vicious circle: we need to talk about testing! It is good for our soul to explain what I did and why, but we don’t know how to talk about our testing in a way that others understand.

Alex and I listed some traps:

Stories decay into Numbers: testing is about providing information to enable others to make informed decisions. The number of test cases or the number of bugs do not really matter. It is the story about the product and the risks involved. Those numbers might back up your story, but they do not tell the story!
A performance decays into Deliverables: testing is about finding problems, collecting information, exploring and experimenting to discover new information. Sure, documents and stuff sometimes help us, but testing is a performance. (James Bach talks about that here: a test is a performance and here: Test cases are not testing: towards a culture of test performance).
Test strategy decays into Test execution: when was the last time you saw a really good test strategy? In many cases I find master test plans where everything is described except the strategy. It is hard to create a test strategy and it is even harder to write it down or visualise it. Many testers I meet focus on test execution: creating test cases and scenarios and calling that the strategy.
Tool supported testing decays into Automation: testing using tools is a great idea. It gives us more opportunities to test and improves testability. But as said earlier: it becomes a problem when we focus too much on automation or even try to automate all our work. We cannot automate testing.
Many kinds of coverage decays into One kind of coverage: testing benefits from diversity! You find a certain type of bug with a certain test technique or approach. By using lots of different views, approaches and techniques, we find more problems.
Learning activity decays into Formalized static tasks: testing is learning about the product for our stakeholders. It’s not about verification and validation, there is much more to it. I like to replace such words with challenge the belief that (verify) and investigate (validate). Those activities provide the valuable information we need.
Balance risk and uncertainty decays into Certainty: people like to be comfortable and we like to give other comfort as well. But as testers we need to stay unsure, when others are sure. It is our job to keep asking critical questions. We are not here to give confidence or comfort, we are here to demolish unwarranted confidence! Also keep in mind that to find new unexpected problems, we have to go where nobody has thought of and nobody went before us. That will cause confusion which feels uncomfortable for many. I learned to be okay with confusion, since this is essential for learning new things.
Business Impact decays into Bugs: some testers are frustrated when bugs aren’t fixed. But that is part of the deal: some things that bug us, are just not important enough.
Product story decays into Testing jargon: I think this is the main problem for people not listening to testers. We talk jargon and about what we do in detail too much. We say stuff like: “We’ve executed 17 test cases in the system test, we’ve automated 50% of the test cases for area C and now have 30% code coverage. We found three major and five medium bugs”. And we are surprised that nobody will listen. We need to talk about the product! So you have found 8 bugs? Who cares? Talk about the risks involved, about the threats to the value of the product.

So maybe testers need to stop talking about testing?
Well, not exactly. We need to remember that the information from testing enables other people to do better work! So the testing itself isn’t always interesting, but the story about the results and the impact on the business is!

Just imagine a conversation between a tester and the PO.
Tester: The testing is going well!
PO: Okay, great. How is the product?
Testers: It sucks!

The role of testers

What is the role of testers? Testers see things for what they are. Testers help others make informed decisions about quality, because we think critically about software. This means creating awareness about the state of the product by staying sceptical when everybody else is sure. So we have to know what our clients want from testing. What information do they need to take these decisions? Project managers have one big question to be answered: are there problems that threaten the on-time, successful completion of the product?

Product Risk Knowledge Gap

I like to explain testing using the “Product Risk Knowledge Gap” like we teach in RST. Knowledge Gaps are the things that we need to learn in order to make good decisions. We need to learn about the product to close the knowledge gap. The more we know, the less risky our decisions will be. Testers should focus on questions like: what does the client need to know right now? What might hinder the successful completion of the product? What role do I need to take on in this situation to ensure we achieve our aims? Does this information matter? To whom?

But there is a way to avoid talking about testing. Just find enough questions and problems so that your stakeholders simply won’t have time to ask you questions back! Also, if you tell a credible story and give them the information they need, nobody cares how you got the information in the first place. In this case you need to stand your ground: tell people what they need to hear despite what they want to hear. Again: it’s your job to see things for what they are. If you give people the chance to doubt what you are doing, because you do not deliver the information they need, they will start asking questions about how you do your job. And if you have to talk about how you do your testing, then prepare to be able to tell a damned good story about your testing. Something they can understand and relate to.

The testing story

The testing story by Rapid Software Testing can help you tell that story. Tell a story about the product, what you saw, what you did to gather that information and how valuable that information is. (See “Braiding The Stories” by Michael Bolton). The testing story contains three stories that feed into each other:

The product story: a qualitative report on how the product can work, how it fails, and how it might fail in ways that matter to our clients.
The testing story: to make the product story credible, the testing story is about how we configured, operated, observed, and evaluated the product; what we actually did and what we actually saw.
The quality of testing story: to make the testing story credible, tell a story about the quality of the testing. Describe why the testing we’ve done has been good enough. It includes details on what made testing harder or slower and what we might need or recommend in order to provide better, more accurate, more timely information.

Modern testing
As testers we do way more than only testing. We are enablers of testing by doing all kind of other things to be a service to the team and our clients. Researching this, Alex and I found the Modern testing principles by Alan Page and Brent Jensen. There is a lot of good stuff in there, and yet we feel that there is not enough focus on the actual testing in their principles. Furthermore, we think that the seventh principle “We expand testing abilities and knowhow across the team; understanding that this may reduce (or eliminate) the need for a dedicated testing specialist.” is formulated too negatively. We do not talk about dedicated test specialist as a function. But we like to talk about testing skills. And although we think there should not be a need for a dedicated testing specialist, we see too many people in teams who do not like testing. Passion (or at least motivation) for what you do, is conditional to become good at anything. So we created our own testing principles (inspired by the modern testing principles of course):

Deliver insight into status of the product
Practice (and enact) critical thinking
Enable testing: lead, coach, teach, support
Discuss testability
Explore & experiment
Promote waste removal / avoidance
Help to accelerate the team
Advocate continuous improvement
Foster quality culture
Keep critical distance and close social distance

Stop talking about testing?

So do we need to stop talking about testing? Not really. But we need to talk about the product, risks and value more. We can talk about actual testing only to back our story up or if they ask questions. And even then, we need to make our story understandable and relatable to others. Make sure you are a service to the team. We created our own testing principles to explain what value we add. We also have a pretty clear story on what testing is and how it adds value. We did this by practicing our stories many times. But we also figured out our own testing paradigm. That makes is easier to talk about what we do and how we add value.

Software Development is research & development: a series of experiments that ultimately lead to a suitable solution. We are dealing with customers who do not know exactly what they want. Furthermore, we are dealing with the complexity, confusion, changes, new insights and half answers. That requires research. As we team we are looking for what works and what doesn’t. Testing is of great importance for this! Testing provides insight and overview. Testing shines a light on the actual status of product and project. These insights enable others to make better decisions and eventually make better products.

The slide are here.

Note: this is an Alex Schladebeck and Huib Schoots co-production and this blogpost was co-authored by Alex. So where you read I, you could read Alex and I.

A Road to Awesomeness

November 6, 2016 / Huib Schoots / 5 Comments

At TestBash Manchester I did a presentation titled “A road to awesomeness”. In this talk I tried to explain how testers can become awesome. I talked about learning and testing skills. To prepare this talk I created a mind map to list skills and ways to learn. The mind map is here and will be continuously updated in the future. Let me know if you have things to add.

The 4-hour tester

In another talk at TestBash Manchester Helena Jeret-Mäe and Joep Schuurkes talked about the 4-hour Tester Experiment. They took the the challenge of teaching new testers useful skills in only 4 hours. They focus on 5 skills:

Interpretation
Modeling
Test design
Note taking
Bug reporting

Their experiment is an interesting try to teach software testers essential skills in only 4 hours. Of course it is impossible to teach anyone to test in 4 hours. But the 4-hour tester has simple exercises that take at most one hour to complete. Interesting approach to help inexperienced testers to get started.

Learning

To become awesome, you have to learn a lot. But how do we learn? I like the 10/20/70 model: 70 percent of learning comes form job-related experiences, 20 percent from interactions with others, and 10 percent from formal educational. So training is only a small part of a learning journey. Interaction with others, like communities and networking, coaching and mentoring are a part of it as well. But the biggest part of learning takes place from work related experience. This doesn’t mean just working. Many years of working doesn’t let you necessarily learn effective if you do not get feedback, reflect on what you do in the right environment. Doing the same work for 10 years doesn’t give you 10 years of useful experience. If you never retrospect, reflect or get any feedback, I’d say you have 10 times 1 year of experience. To make learning effective in your job you need:

Concrete, challenging and achievable tasks
Realistic application of skills, processing and reflection
Personal interpretation, exchange with others and constructive feedback
A safe environment to experiment and make mistakes

But there is more to learning. In her TED Talk “Learning how to learn” Barbara Oakley talks about two fundamental learning modes of our brain: focused mode and the more relaxed diffused mode. When your are learning, you want to go back and forth between these two modes. When you are stuck, you defocus going into the diffused mode, generating new ideas. If you want to learn more about this, I recommend the coursera MOOC “Learning How to Learn: Powerful mental tools to help you master tough subjects“. Two of the things I took from that course is that active listening (e.g. by asking questions) is far more effective. Also learning by doing is a great way to learn fast.

Learning is the most important skill for a tester. And learning is closely related to thinking skills. I’ll come to that later.

What makes an awesome tester?

An awesome testers has many skills. In my blogpost “Heuristics for recognizing professional testers” I described 18 heuristics to recognize professional testers. This was the starting point for my talk at TestBash Manchester. The first steps to help me think about aspects to become awesome, was creating a mind map with a couple of things to focus on: what makes an awesome tester: who (characteristics), what (skills) and how (ways to learn and how to become an expert).

There are many skills that make an awesome tester. I think the most important skill is the ability to learn! Remember the definition of testing we use in Rapid Software Testing: “Testing is evaluating a product by learning about it through exploration and experimentation”. Besides learning I identified 5 categories of skills testers need:

Thinking skills
Testing skills
Technical skills
Domain skills
Soft skills

And of course there are many, many skills that make these categories. Have a look at the mind map to learn more about these skills. It is very important to recognise what skills are involved in being a great tester. If you want to learn, you need to know what to focus on. Not knowing which skill to train, will result in unfocussed and ineffective learning. I see many testers apply test techniques without knowing what skills are used and which methods are underneath the technique. This makes them apply techniques as recipes or trick. Being able to apply approaches, tactics and techniques effectively in any situation requires the right skills.

Thinking skills

Learning and thinking are closely related. While researching thinking and thinking skills, I stumbled upon a great TED talk by Dr. Derek Cabrera called ”How Thinking Works”. He talks about the schooling system and about 4 universal thinking skills. Schools nowadays are over-engineering the content curriculum: students do not learn to think, they learn to memorize stuff. Kids are learned to follow instructions, like painting by the numbers. To fix this, we need to learn how to think better and for that he suggests four thinking skills: DSRP. A simple set of rules to become better in systems thinking.

Making Distinctions – Any idea or thing can be distinguished from other ideas or things
Organizing Systems – Any ideas or thing can be split into parts or lumped into a whole
Recognizing Relationships – Any idea or thing can relate to other ideas or thing
Taking Perspectives – Any idea or thing can be the point or the view of a perspective

In his book “Systems thinking made simple“, Dr. Cabrera talks metacognition. “Systems thinking is a particular type of metacognition that focuses on and attempts to reconcile the mismatch between one’s mental models and how the real world works”, he continues to describe acts of metacognition: “Awareness that everything you experience is a mental model that approximates (either poorly or well) the real world”, “The ability to distinguish among cognition (thinking), emotion (feelings), and conation (motivations) and the awareness of how these different human capacities influence our mental models and behavior”. Recognizing that models are a big part of our thinking makes you a better tester. But also that your biases and emotions influence your thinking are great insights we have to take into account in our testing.

A second TED talk I recommend is “How to think, not what to think” by Jesse Richardson. He says: “The way we approach education needs to adapt! What’s different in teaching children how to think, we are involving them in the process of their own learning. Instead of just telling them to memorize the right answer, we ask them to engage their own minds, their own awareness by questioning things, attaining understanding not just knowledge. And that involvement, that engagement, is so important, because it keeps the spark of curiosity alive.”

He mentions the famous TED Talk by Ken Robinson: “Do schools kill creativity?“. One of my all time favorites.

Activities in testing
To fully understand what skills we use in testing, I made a list of testing activities. In Rapid Software Testing we teach the 9 Elements of testing and I took these as a starting point.

Model the test space and risks
Determine coverage
Determine oracles
Determine test procedures
Configure the test system
Operate the test system
Observe the test system
Evaluate the test results
Report test results

The next step was to zoom in on the different elements in the list. Zooming in on “Model the test space and risks” we can distinguish:

Context Analysis
Creating a product Coverage Outline
Test Plan
Test scope
Risk & Value Analysis

I did this with all 9 elements and came to a pretty long list of activities. Maybe you can think of more activities testers do, let me know if you do.

This (making distinctions or factoring) exercise resulted in a huge list of things we do in testing and products we make. The next step was to lists the skills that we use, doing these activities. Obviously the first thing I thought of was learning and thinking. Researching this, I stumbled upon a lot of interesting stuff, of which part I have described earlier. But there was more, way more as you can see in my mind map. And this mind map expands every time I visit it. Factoring the activities and skills helps me to see what is really going on while we are testing.

So how do you become awesome?

So now we have a map of activities and skills. But how do we learn them? And where do we start?

The most important part of becoming awesome is knowing what to learn and how. The first step is to know what to learn and focus on that. Stop trying to learn 10 things at the same time, it doesn’t work! A way to get to know what you want to learn is writing a Personal Development Plan answering 5 simple questions: Who am I? What are my skills? What do I want? What do I need? How do I get there? This can give you great insight in your skills, ambition and learning needs. The second step is to find mentors who can help you and give you feedback. After that it is “just” a matter of doing it and practice!

I learned to be awesome by doing many things:

Observing others and myself: by becoming manager and a coach I found out that observing others gives you great insight in how testing is done and what the common problems are for people who are learning how to test. Observing myself, which is quite difficult in the beginning, I learned what I was doing and I found out that this was a big eye-opener and learning enabler. Using a journal a writing down stuff every day on my way home, gave me great insights in what my problems were. Knowing your problems, is the first step to solving them.
Explaining, presenting, teaching & coaching: by explaining stuff to others, you learn to structure your thoughts. To be able to present or teach you really have to know your stuff. And still every time I teach, I learn new things about what I teach. New examples, new problems, new ways of doing things. Also answering difficult questions is very helpful learn deeply about topics. That is way I think we should encourage debate and questioning at conference way more.
Pairing: pairing is a fun and nice way to learn from others. Seeing how others do things is helpful and has many learning opportunities. Also the feedback you get from your pairing-partner is valuable.
Writing my blog: same thing as explaining and teaching. Writing this blog post made me think about what o write and how to explain. I needed to structure my thoughts. Great exercise.
Keeping a journal: see observing other and myself.
Always having a notebook with me: to practice note taking and it also helps me not to forget interesting stuff that others tell me. It is great to read back notes from conferences and courses from way back. It refreshes my memory and I often see new insights I didn’t see before.
Discussing & debating testing: see explaining.
Trying new stuff: experimenting is fun and also a way to learn about new techniques, tools, approaches, etc.
My coaches and mentors: I have had and still have many mentors. All these people help me to become better every day. It is great to have many mentors, each with their own expertise and specialties. I have a big international network of people I know and who I work with. So many sources of knowledge and skill help me quickly find an answer to almost every difficult testing problem I have.

My advise is to try the same exercise I did creating the mind map. Create your own model of test activities and skills. Observe what you do, how you do it and make your own mind map (or any other way you would like to model it). Come back and compare yours to my mind map.

So why is this important?

I see may similarities with the problems in the current schooling system and in testing: over-engineering the content curriculum, students do not learn to think, they learn to memorize stuff and to follow instructions (like painting by the numbers). In most testing classes testers learn tricks, procedures and the use of template like approaches in stead of the required skills to do an excellent job. In Rapid Software Testing (RST) we teach thinking skills. We learn testers how to effectively think and solve problems instead of using tricks, templates and standards to do their work. By learning to think better, you learn better. And isn’t testing all about learning? Also, if you are able to think, you are able to do better work, apply your skills in the right way. Adapt to changing context, find your own solution to problems that occur. We also teach testers to dynamically manage their focus: focus/defocus is similar to fundamental learning modes Prof. Dr. Barbara Oakley talks about in her TED Talk.

The DSRP model (distinctions, systems, relationships, perspectives) is very useful in testing. In RST we talk about models and learn our students how to model (distinctions, systems and relationships) and to think from different perspectives using a diverse strategy using heuristics. Great to see that the stuff we teach is actually backed up by scientists who study metacognition (thinking about thinking) and epistemology (the study of knowledge). Making a product coverage outline for instance helps you see distinctions and relations and helps systems thinking.

Good luck on your journey to awesomeness!

References

The slides I used in Manchester are here: slideshare.
The updated slides are here: pdf.

All talks at TestBash are recorded. My talk is here.

Too controversial?

June 21, 2016 / Huib Schoots / 3 Comments

On May 11 2016 TestNet (*) held her spring conference with “Strengthen your foundation: new skills for testers” as the central theme. The call for papers that was send out made me frown. It said:

“In the final keynote of the TestNet autumn event, speaker Rini van Solingen referred to the end of software testing as we know it. ‘What one can learn in merely four weeks, does not deserve to be called a profession’, he stated. But is that true? Most of our skills, we learn on the job. There are many tools, techniques, skills, hints and methods not typical for the testing profession but essential for enabling us to do a good job nonetheless. Furthermore the testing profession is constantly evolving as a result of ICT and business trends. Not only functional testing, but also performance, security or other test varieties. This presses us to expand our knowledge, not just the testing skills, but also of the contexts in which we do our jobs. The TestNet Spring Event 2016 is about all topics that are not addressed in our basic testing course, but enable us to do a better job: knowledge, skills, experience.”

I think that there are a lot of skills that are not addressed in our “basic testing course” where they should have been addressed. I am talking about basic testing skills! So I wrote an abstract for a keynote for the conference:

The theme for the spring event is “Strengthen your foundation: new skills for testers”. My story takes a step back: to the foundation! Because I think that the foundation of most testers is not as good as they think. The title would then be: “New skills for testers: back to basics!“

Professional testers are able to tell a successful story about their work. They can cite activities and come up with a thorough overview of the skills they use. They are able to explain what they do and why. they can report progress, risk and coverage at any time. They will gladly explain what oracles and heuristics they use, know everything about the product they are testing and are deliberately trying to learn continuously.

It surprises me that testers regularly can’t give a proper definition of testing. Let alone that they are able to describe what testing is. A large majority of people who call themselves professional testers can not explain what they do when they are testing. How can anyone take a tester seriously if he/she can not explain what he/she is doing all day? Try it: go to one of your testing colleagues and ask what he or she is doing and why it contributes to the mission of the project. Nine out of ten testers I’ve asked this simple question, start to stutter.

What do you exactly do if you use a “data combination test” or a “decision table”? What skills do you use? “Common sense” in this context does not answer the question because it is not a skill, is it? I think of: modeling, critical thinking, learning, combine, observe, reasoning, drawing conclusions just to name a few. By looking in detail at what skills you are actually using, helps you recognize which skills you could/should train. A solid foundation is essential to build on it in the future!

How can you learn the right skills if you do not know what skills you are using in the first place? In this presentation I will take the audience back to the core of our business: skills! By recognizing the skills and training them, we are able to think of and talk about our profession with confidence. The ultimate goal is to tell a good story about why we test and value it adds.

We need a solid foundation to build on!

My keynote wasn’t selected. So I send it in as a normal session, since I really am bothered by the lack of insight in our community. But it didn’t make it on the conference program as a normal session either. Why? Because it is too controversial they told me. After applying for the keynote the chairman called me to tell me that they weren’t going to ask me to do a keynote because the did want a “negative” sound on stage. I guess I can imagine that you do not want to start the day with a keynote who destroys your theme by saying that we need to strengthen our foundation first before moving on.

But why is this story too controversial for the conference at all? I guess it is (at least in the eyes of the program committee) because we don’t like to admit that we lack skills. That we don’t really know how to explain testing. I wrote about that before here. It bothers me that we think our foundation is good enough, while it really isn’t! We need to up our game and being nice and ignoring this problem isn’t going to help us. A soft and nice approach doesn’t wake people up. That is why I wanted to shake this up a bit. To wake people up and give them some serious feedback … I wrote about serious feedback before here. But the Dutch Testing Community (represented by TestNet) finds my ideas too controversial…

(*) TestNet is a network of, by and for testers. TestNet offers its members the opportunity to maintain contacts with other testers outside the immediate work environment and share knowledge and experiences from the field.

A new level of testing?

January 24, 2015 / Huib Schoots / 9 Comments

Yesterday I saw this awesome video of Lars Andersen: a new level of archery. It is going viral on the web being watched over 11 million times within 48 hours. Now watch this movie carefully…

The first time I watched this movie, I was impressed. Having tried archery several times, I know how hard it is to do. Remember Legolas from the Lord of Rings movie? I thought that was “only” a movie and his shooting speed was greatly exaggerated. But it turns out Lars Andersen is faster than Legolas. My colleague Sander send me an email telling me about the movie I just watched saying this was an excellent example of craftsmanship, something we have been discussing earlier this week. So I watched the movie again…

Also read what Lars has to say in the comments on YouTube and make sure you read his press release.

This movie is exemplar for the importance of practice and skills! This movie explains archery in a way a context-driven tester would explain his testing…

0:06 These skills have been long since been forgotten. But master archer Lars Andersen is trying to reinvent was has been lost…

Skills are the backbone of everything being done well. So in testing skills are essential too. I’ll come back to that later on. And the word reinvent triggers me as well. Every tester should reinvent his own testing. Only by going very deep, understand every single bit, practice and practice more, you will truly know how to be an excellent tester.

0:32 This is the best type of shooting and there is nothing beyond it in power or accuracy. Using this technique Larsen set several speed shooting records and he shoots more than twice as fast as his closest competitors…

Excellent testers are faster and better. Last week I heard professor Chris Verhoef speak about skills in IT and he mentioned that he has seen a factor 200 in productivity difference between excellent programmers and bad programmers (he called them “Timber Smurf” or “Knutselsmurf” in Dutch).

0:42 … being able to shoot fast is only one of the benefits of the method

Faster testing! Isn’t that what we are after?

0:55 Surprisingly the quiver turned out to be useless when it comes to moving fast. The back quiver was a Hollywood Myth…

The back quiver is a Hollywood myth. It looks cool and may look handy on first sight, since you can put a lot of arrows in it. Doesn’t this sound like certificates and document-heavy test approaches? The certificates looks good on your resume and the artifacts look convenient to help you structure your testing… But turn out to be worthless when it comes to test fast.

1:03 Why? Because modern archers do not move. They stand still firing at a target board.

I see a parallel here with old school testing: testers had a lot of time to prepare in the waterfall projects. The basic assumption was that target wasn’t moving, so it was like shooting at a target board. Although the target proved always to be moving, the testing methods are designed for target boards.

1:27 Placing the arrow left around the bow is not good while you are in motion. By placing your hand on the left side, your hand is on the wrong side of the string. So you need several movements before you can actually shoot..

Making a ton of documentation before starting to test is like several movements before you can actually test.

1:35 From studying old pictures of archers, Lars discovered that some historical archers held their arrow on the right side of the bow. This means that the arrow can be fired in one single motion. Both faster and better!

Research and study is what is lacking in testing for many. There is much we can learn from the past, but also from social science, measurement, designing experiments, etc.

1:56 If he wanted to learn to shoot like the master archers of old, he had to unlearn what he had learned…

Learning new stuff, learning how to use heuristics and train real skills, needs testers to unlearn APPLYING techniques.

2:07: When archery was simpler and more natural, exactly like throwing a ball. In essence making archery as simple as possible. It’s harder how to learn to shoot this way, but it gives more options and ultimately it is also more fun.

It is hard to learn and it takes a lot of practice to learn to do stuff in the most efficient en effective way. Context-driven testing sounds difficult, but in essence it makes testing as simple as possible. That means it becomes harder to learn because it removes all the methodical stuff that slows us down. These instrumental approaches trying to put everything in a recipe so it can be applied by people who do not want to practice, make testing slow and ineffective.

2:21 A war archer must have total control over his bow in all situations! He must be able to handle his bow and arrows in a controlled way, under the most varied of circumstances.

Lesson 272 in the book Lessons Learned in Software Testing: “If you can get a black belt in only two weeks, avoid fights”. You have to learn and practice a lot to have total control! That is what we mean by excellent testing: being able to do testing in a controlled way, under the most varied of circumstances. Doesn’t that sound like Rapid Software Testing? RST is the skill of testing any software, any time, under any conditions, such that your work stands up to scrutiny. This is how RST differs from normal software testing.

2:36 … master archers can shoot the bow with both hands. And still hit the target. So he began practicing…

Being able to do the same thing in different ways is a big advantage. Also in testing we should learn to test in as many different ways as possible.

3:15 perhaps more importantly: modern slow archery has led people to believe that war archers only shot at long distances. However, Lars found that they can shoot at any distance. Even up close. This does require the ability to fire fast though.

Modern slow testing has led to believe that professional testers always need test cases. However, some testers found that they could work without heavyweight test documentation and test cases. Even on very complex or critical systems also in a regulated environment. This does require the ability to test fast though.

3:34 In the beginning archers probably drew arrows from quivers or belts. But since then they started holding the arrows in the bow hand. And later in the draw hand. Taking it to this third level. That of holding the arrows in the bow hand, requires immense practice and skill and only professional archers, hunters and so on would have had the time for it. … and the only reason Lars is able to do it, is he has been years of practicing intensely.

Practice, practice, practice. And this really makes the difference. I hear people say that context-driven is not for everybody. We have to accept that some testing professional only want to work 9 to 5. This makes me mad!

I think professional excellence can and should be for everyone! And sure you need to put a lot of work in it! Compare it to football (or any other good thing you want to be in like solving crossword puzzles, drawing, chess or … archery). It takes a lot of practice to play football in the Premiership or the Champions League. I am convinced that anyone can be a professional football player. But it doesn’t come easily. It demands a lot of effort in learning, drive (intrinsic motivation, passion), the right mindset and choosing the right mentors/teachers. Talent maybe helps, and perhaps you need some talent to be the very best, like Lionel Messie … But dedication, learning and practice will take you a long way. We are professionals! So that subset of testers who do not want to practice and work hard, in football they will soon end up on the bench, won’t get a new contract and soon disappear to the amateurs.

4:06 The hard part is not how to hold the arrows, but learning how to handle them properly. And draw and fire in one single motion not matter what methods is used.

Diversity has been key in context-driven testing for many years. As testers we need to learn how to properly use many different skills, approaches, techniques, heuristics…

4:12 It works in all positions and while in motion…

… so we can use then in all situations even when we are under great pressure, we have to deal with huge complexity, confusion, changes, new insights and half answers.

5:17 While speed is important, hitting the target is essential.

Fast testing is great, doing the right thing, like hitting the target is essential. Context-driven testers know how to analyze and model their context to determine what the problem is that needs to be solved. Knowing the context is essential to do the right things to discover the status of the product and any threats to its value effectively, so that ultimately our clients can make informed decisions about it. Context analysis and modelling are some of the essential skills for testers!

There are probably more parallels to testing. Please let me know if you see any more.

”Many people have accused me of being fake or have theories on how there’s cheating involved. I’ve always found it fascinating how human it is, to want to disbelieve anything that goes against our world view – even when it’s about something as relatively neutral as archery.” (Lars Andersen)

Category: Skills

Do we need testers?

7 Rules for Positive, Productive Change – a book review

The art of reflection

Let’s stop talking about testing, let’s start thinking about value

A Road to Awesomeness

Too controversial?

A new level of testing?

Quotes on Software Testing

Recent Posts

Categories

Recent Comments

Archives

My website