Tag Archives: Awesome Women Testers

Specification By Example and Product Quality

Target audience: Product People, Testers, Developers

I’m sure that in your workday with software development you’ve heard words like BDDATDD, and Specification By Example. Most of us link these terms to just the test portion of an application but do not observe how powerful the practices are.

Rather than just contributing to the test of what your team is developing, they help ensure that it is actually doing the thing — what your customer needs!

To try to clarify a little better where these words came from and how we got here, I set up a timeline based on several different authors. These authors have written books covering the breadth of software testing and its principles. In the end they’re all basically saying the same thing, but they’re also presenting the theme according to their point of view.

A Little History


Figure 1

1999 — Kent Beck (Extreme Programming Explained: Embrace Change)

In this book, Kent Beck presents the values, practices, and principles of XP. Although he doesn’t talk about the term “Specification By Example”, he talks about continually specifying and refining during development, so that customer learning is reflected in the developed software.

In the cycle he proposes, the stories are specified immediately before they are implemented. After which, they become extracted tests of those specifications. In the cycle that follows —

  • The interface is designed to meet test needs
  • The code is written to match the tests and the interface
  • The design is refined to match the needs of the written code

This leads to a decision about which story to specify next. Meanwhile, the rest of the stories remain stationary, until they are chosen for implementation.


Figure 2

2004 — Martin Fowler — Specification by Example

In this article, Martin uses the phrase “Specification By Example” to refer to what Kent Beck had called specs in the XP part of testing. Instead of dealing only with behavioral specifications (thinking about preconditions and postconditions), the word example is added to make us think in a more humanized way in terms of actual use.

An important point that he raises is that Specification By Example alone does not solve all problems. You need to always have the support of more than one tool to achieve your goals and solve problems.

He also reinforces that this only works in collaborative environments — where all the people involved in software development are together building the best product.

“Always have the support of more than one tool to achieve your goals and solve problems”

2005 — Lisa Crispin — Customer Test-Driven Development

Communication and collaboration are two key words in this article by Lisa. For each story, she asked the client how he would know that the story was complete — in the form of tests and examples.

Just like in TDD, before writing any code, the tests are written first — and then if they pass, it means that the code meets the requirements. The tests should ideally be written in a way that can be understood by an automation tool. It should also be at a high level so that they can guide the exploratory tests.

“Write tests that can be understood by an automation tool”

2006 — Dan North — Introducing BDD

During his journey using TDD, Dan North noted some difficulties, such as — where to start, what to test and what not to test, how to call the tests, and how to understand why a test fails.

BDD is an agile practice that allows better communication between all technical and non-technical people involved during a software project. It describes a cycle of iterations with well-defined outputs and resulting in the delivery of tested and functioning software.

“BDD allows better communication between technical and non-technical people involved in a software project”

2008 — Elisabeth Hendrickson — Driving Development with Tests: ATDD and TDD

In this material, Elisabeth talks about executable requirements, where you create tests before coding. These tests represent the behavior that the software should have. It reminds you that it is very important that this work be done together with the stakeholders.

According to her, the specifications give feedback on how close the team is to the “done” of a story, giving even an idea of progress toward the goal.

2009 — Gojko Adzic — Bridging the Communication Gap: Specification by Example and Agile Acceptance Testing

The most relevant point here in this Gojko publication is — specification is not a programming technique, but a communication technique. It brings people involved in software development closer to the project.

“Specification is not a programming technique, but a communication technique”

2010 — Lisa Crispin — ATDD X BDD X Specification By Example

At Agile 2010, during a workshop on Functional Testing, about 20 people from the international software community were discussing what ATDD really meant.

As you can see in the summaries of each of the sessions above, several people were saying the same thing but with different names and some particularities — (examples, acceptance tests, scenarios, user tests, behaviour, functional tests, executable specification …).

As a result of this workshop they came up with a definition of Specification By Example that would be described in 2011 in Gojko’s book of the same name.


Figure 3

2010 — Craig Larman — Specification By Example

When Craig talks about Less, he devotes a part to techniques that help achieve technical excellence. And guess what — the Specification For Example is there (as well as Integration and Continuous Delivery, Clean Code, TDD and others).

According to him, tests should be regarded as requirements and requirements as tests. The book reminds you that if you have a test that describes the behavior, and at the end of development this test passes — it means that you have reached the goal of that functionality.

In addition, he cites other elements we have seen in previous blocks, such as — doing workshops to collaborate and discovering the requirements, working on short development cycles, and focusing on problem prevention rather than finding.

“Tests should be regarded as requirements, and requirements as tests”


Figure 4

2011 — Gojko Adzic — Specification by example: How successful teams deliver the right software

It was through this book that I came to know the subject. Two years ago as I entered Concrete, it was the first book I received as a reference on QA’s work.

In this book, Gojko brings together a series of standards that have been observed in several successful projects. He defines Specification by Example as a set of these standards that help to develop the right software.

Since then I have been using the references in this book by Gojko when talking about Specification By Example. But in doing the research for this post, I saw that his book is a summary of what many other people were talking about the world.

Get Down to Business

Going back to the beginning of this text, many people associate Specification Examples with only the testing part of the application. But they do not use all of their tool-ing to effectively build quality products — both in terms of meeting functional specifications, and actually being what the customer needs.

I consider Specification By Example as a very focused approach to discovery. Here collaboration between those involved in the development process (product and engineering) is extremely important in order to discover the requirements needed to meet the customer’s need. It helps to exemplify them in a way that all of these people can understand what is being said (so it uses a domain language) and using engineering support (with automated testing and Continuous Delivery and Integration practices). This ensures the delivery of quality software.

Applying SBE in Practice

For each of the practices of the Specification By Example, I have examples of how we can apply it:

#1 Derive the scope from the objectives

Find out what the customer really needs. Then, give the development team the freedom to suggest the best way to do it. This is simpler to implement, faster and even cheaper.

Our focus, then, should be to understand what problem the customer wants to solve, or the goal he wants to achieve. And from there, the whole team collaborates to arrive at the solution design.

#2 Business Objectives -> Derive the Scope -> Create the Stories

For example, if we aim to solve a problem of customers who want to pay an account quickly, we can derive the following scope:

The bank account holder will pay a ticket through the barcode. He can type the barcode, which is slow, or can use the camera of the mobile phone to capture the image of the barcode. The application must interpret this image and fill in the ticket data to be paid without the need for user intervention. The user can review the data and confirm the payment.

The faster option should be offered as the first option to the user, while the slower option should be offered as an option.

Story 1: I, as a checking account holder, want to use my cell phone camera and identify a barcode, so that my ticket data is filled up quickly.

Story 2: I, as a current account holder, want to manually fill in the barcode number of a ticket, to pay it even if the bar code is unreadable.

Story 3: I, as a current account holder, want to review the completed data of a ticket informed via barcode, to confirm them and to effect payment.

#3 Specify Collaboratively

Here is where some people who try to implement Specification By Example end up getting lost. Often only one person within the team (usually the one who plays the role of QA) writes the specs, and the rest of the team just looks. If this is what happens, we totally lose the sense of collaboration.

The idea is that — all involved should participate, and this can be through workshops, or writing and review. The important thing is that the specifications are written by the team including the client. Language has to be part of the business domain and be understood by everyone.

#4 Illustrate Using Examples

Human beings better understand practical examples. This helps avoid ambiguity and communicates accurately. It is important to use concrete business examples and think about expected outputs.

In addition, the examples help to generate discussion and eliminate doubts. Here’s one —

Functionality: Free Delivery
1: Offered to VIP customers, once they buy a certain number of books;
2: It is not offered for ordinary customers or for VIP clients who buy anything other than books;
3: The minimum number of books for free delivery is 5


Figure 5

What do good examples look like?

  • Must be precise: it helps to avoid ambiguity, it should clearly inform the context and how the system should behave in each case
  • Must be complete: it is important to combine different inputs and think about the expected results, as well as to experiment with data
  • Must be realistic: always think of real user situations (caution with sensitive information)
  • Easy to understand: everyone involved should read the examples and understand without much difficulty. You can use abstractions — for example, use “car” as an example instead of describing exhaustively the characteristics of the car: four-wheel car, two-door etc., unless these characteristics influence the final result.

You can also use examples for non-functional requirements.

  • Performance: “The system needs to import X records into Y minutes by consuming CPUs.”
  • UI: Screen prototypes are good examples as well.

#5 Automate Specifications

Today we have several tools available that allow you to automate specifications. The best known is Cucumber. Here, all the scenarios you wrote using the Gherkin format are interpreted and implemented from any other automation framework. Some examples are — Appium or Selenium.

With each new feature or change to an existing one, you can add or modify its specifications and ensure that the regressive continues to work.

#6 Validate Frequently

It’s no good getting several automated tests that do not run continuously. To ensure that your specification is alive once you have implemented them, you need to use mechanisms such as Continuous Integration for these tests.

It helps to remember that they are tests built from the specifications — they ensure that what has been developed meets the business needs. They need to be run frequently with each new change. This ensures that if there is a problem, the tests give you quick feedback and indicate what needs to be fixed. Here you can use tools like JenkinsBambooCircleCI, among others.


The purpose of this text was to show that developing from specifications is not new. We have been talking about this for a long time in software engineering.

But the most important point of the specification as we know it today is the constant discovery of requirements related to the product. It’s also about the use of that knowledge to generate a business domain language, which will be used in the writing of live and executable documentation.

Image Credits:

Figure 2: Extreme Programming Explained: Embrace Change — Kent Beck

Figure 3: Results from Workshop in Agile 2010

Figure 4: https://less.works/less/technical-excellence/specification-by-example.html

Figure 5: https://www.amazon.com.br/Specification-Example-Successful-Deliver-Software/dp/1617290084

References if any


Author bio:

Author's photo

Linkedin: https://www.linkedin.com/in/samantacici/

Twitter: https://twitter.com/samantacicilia

QA Manager at Concrete (an Accenture Company), helping teams to define quality strategies to create incredible digital products. Today I lead 40 QAs in the Rio de Janeiro, São Paulo, Recife, Belo Horizonte (Brazil) and Colombia offices.

I am passionate about agility. I believe in empiricism, continuous improvement and diversity.

Co-founder of the Carioca Testing Group that is now part of Ministry Of Testing (https://www.meetup.com/pt-BR/Ministry-of-Testing-Rio-de-Janeiro/)

Articles of QA published in https://medium.com/@samantacicilia/ and https://code.likeagirl.io/portugues/home

What is your upcoming strategy for Test Automation?

Implementing test automation is critical to build and deploy robust applications in quick time. However, the business should have a strategy in place to make test automation functional, cost effective and easy to maintain.

In an Agile and DevOps environment, the focus is on the quick detection of glitches in an application before its eventual deployment. This is due to the fact that a quality validated application stands a good chance in securing customer approval than the one devoid of it. It is better to have a robust and secure application with less number of features than having one with umpteen features but with glitches galore. To ensure this, the software must undergo a series of tests namely, unit, functionality, performance, usability, security, and regression test among others. However, should these battery of tests be conducted manually, the eventual outcome may not be to the satisfaction of everyone due to the inherent issues with manual testing.

Test automation provides the best way to validate an application in terms of quality and ensure its quick deployment. The automated tools can run scripts that test repetitive tasks to shorten the SDLC. This way, QA automation testing can achieve speed and reliability – important deliverables of the Agile-DevOps setup. Even though the test automation framework adopted by businesses has reduced the number of test cases, there are occasions where businesses have failed to implement their automated testing strategy like in regression testing. The reasons for the failure in implementing test automation can range from the difficulty in writing test scripts with a longer lifespan to maintaining them. Let us chart a robust automated testing strategy to achieve quality deliverables and ROI.

Strategy to implement test automation solutions

#1 Automating the right test cases: Since it is difficult to automate each and every component of the application, choose the ones that give the best test outcomes. Thus, it is better to run keyword based tests, for they are easy and quick to write. Moreover, they are best understood by the software as well. The challenge is to maintain these tests and keep them updated over time. The test cases to be automated should be of types such as large datasets, high risk, repetitive, and those covering a number of browsers, platforms and networks. The automation of such choicest test cases would help in identifying the glitches better, provide relevant dashboard reports for analysis and improving the processes, and free the test automation experts to explore other features thoroughly.

#2 Cover the development sprint: The Agile-DevOps ecosystem is about building a development sprint where writing and testing the codes are done concurrently. The goal of building a development sprint is to accomplish both the tasks of development and QA together and in the shortest possible time. This is not always feasible to achieve except with proper planning. For example, when new functionality is added, it should be subjected to unit testing. In other words, the functionality should be broken down into smaller units and tested based on keywords that correspond to your business requirements. This way, the business can derive the maximum benefit out of unit testing. The QA team can write automated scripts for various units while being a part of the sprint.

The test automation solutions can help the QA team to keep pace with the development team by quickly performing various unit tests. When the development and QA teams work in the same sprint, it significantly improves the quality of the code by a large measure.

#3 Making the automated scripts to last long: Among the many challenges of writing robust test cases is the fact that they can become irrelevant after a while. This can be pre-empted by writing smaller test cases in a sequence of steps to test a functionality. The advantage of writing smaller individual test cases is that they need not be rewritten once a new functionality is introduced. On the other hand, instead of rewriting the whole script from scratch, it will be easier to focus on a few test cases pertaining to the changes.

#4 Independent of UI: The flexibility of an automated test script can be ensured if the same is UI independent. The test cases should be written based on keywords that are duly supported by the backend. At the same time, should the test cases use the UI elements, it can spell trouble as the latter can undergo changes.  


The only way QA automation testing can succeed is by properly designing the QA build and devising a well thought out strategy. The focus should be to write test cases that remain relevant across processes and changes.

Author Bio

Komal Lopez

Komal Lopez works with Cigniti Technologies and is instrumental in helping enterprises make better decisions related to Quality Assurance products, tools, and services by leveraging research and content. She specializes in writing about technology trends, testing trends and has been in the Marketing and Communications industry for over a decade.



January 2017 edition of Women Testers is brought to you by the team of Women Testers in association with Testing Circus.

In this edition, we have bought to you the following articles: “Not Invented Here” by Claire Moss

A guest post  “Are Test Cases Dead?” authored by Sandeep Garg

A frequent speaker at international conferences writes about the “Benefits of public speaking” – Karen N Johnson

Check out this reading list by Parimala Hariprasad and her review of the books she read in “Bibliography – Books I read in 2016″

Amanda Perkins has authored a write-up about “ISO Multilingual Testers”

We look forward to you reading and sharing this edition which also has a special editorial written by Keith Klain titled “Nevertheless, she persisted…”

From around the testing world: These software testing conferences are lined up for March 2017 Agile Testing Days, Asia and Agile India. Do write to us at [email protected] and share your article for the upcoming edition of Women Testers by March 20th 2017.

Upcoming conferences:

Test Master’s Academy brings to you 2ND ANNUAL TEST LEADERSHIP CONGRESS – MAY 1ST-3RD, NewYork City.  Watch this video to learn more: https://youtu.be/ldKUOGOK2BM 

Thank you for your awesomeness authors and for the timely submissions. And Testing Circus for your continued inspiration and patronage.

Co-ordinator and Editor
Jyothi Rangaiah




October 2015 edition of Women Testers is brought to you by the team of Women Testers and in association with Testing Circus.

In this edition we have bought to you the following articles:
A guest post on Mindmapping essentials written by Dhanasekar Subramaniam

A co-authored post on Stress And Work written by Annie Rydholm and Paul Seaman

Parimala Hariprasad writes about her research on Competitor Analysis
Rikke Simonsen shares her experiences about Making The Transition From Manual To Automated Testing
A first timer at Women Testers Alexandra Schladebeck has shared a write-up on Some Words About Words
If you have liked reading about Testing in the Cloud – Part II written by Nitika Katiyar in our previous edition, this edition has the final part of this article.

Yes, Call For Papers is now open for NULLCON 2016.

We look forward to reading and sharing your new learning experiences while you are at these software testing conferences lined up for November 2015 EclipseCon, TestBashNY, EuroSTAR Conference 2015, Better Software Conference, LET’S TEST [South Africa]. Do write to us at [email protected] and share your article for the upcoming edition of Women Testers by December 20th 2015.

Thank you for your awesomeness authors and for the timely submissions. And Testing Circus for your continued inspiration and patronage.

Co-ordinator and Editor
Jyothi Rangaiah

WomenTesters – Frequently Asked Questions

FAQs – Frequently Asked Questions

1) The article is published elsewhere / on my blog. Can I re-share?
A) The decision to publish such articles will dwell with the editor and the author of the article.

2) I am not a woman, can I submit an article to women testers?
A) Yes. We have a guest post section. We will publish your articles there.

3) This is a quarterly e-magazine. What if a submission to the guest post is already made, will you publish the write-up?
A) Yes. We will accomodate based on the article length, relevance and based on the status-quo.

*We have changed the format of the publication of pdf to directly publishing the articles as we request or as we receive contributions. This is no longer a quarterly edition since January 2018.

4) I am a naive software tester, can I submit an article to women testers?
A) Yes. Please ensure that the article is reviewed prior to submission.

5) I am new to software testing, can I submit my questions to women testers?
A) Yes. Do write to [email protected] We will get the mentors in the industry to answer your question.

6) Where can I find the launch edition of this e-magazine?
A) Download your free copy of the launch edition here –> https://www.womentesters.com/women-testers-july-2014-edition/

7) What are the guidelines for submitting an article to women testers?
A) Follow this link https://www.womentesters.com/womentesters-guidelines-article-submission/

8) I wish to volunteer for the magazine, whom do I contact?
A) Like / Follow us on twitter and on Facebook and leave a message.

9) Why women testers?
A) All questions related to this question is answered in the editorial of the launch edition. A link to the launch edition is provided as an answer to question #6.

10) I am not a woman, can I volunteer / contribute / be a part of the review panel?
A) Yes.

11) I have questions, that are not answered here, whom do I get in touch with?
A) Please write to: [email protected]

12) What is the format in which images / photos needs to be shared in?
A) jpg and png are the formats in which images used in the articles and photos can be shared in. Do provide the credits / courtesy when sharing the images used in the article.