Category Archives: Awesome Women Testers

Getting the whole team engaged in testing for continuous delivery success

By – Lisa Crispin

More and more teams today are trying to move towards delivering changes to production once or twice a week, daily, or even multiple times per day. Given that we never seem to have enough time to complete all the necessary testing for any new feature, how can we possibly feel confident that these frequent deploys won’t cause pain for our customers? 

In their CAST 2019 tutorial, Ashley Hunsberger and I will lead participants through hands-on simulations to practice a whole-team approach to building confidence for continuous delivery and deployment (both known as CD). We’ll start by having participants think about what is currently getting in their team’s way of succeeding with continuous delivery.

We’ll explain common terminology used in DevOps activities, such as continuous integration, continuous delivery versus continuous deployment, deployment pipeline, and cycle time. Learning the language of DevOps helps testers communicate and collaborate with everyone on the delivery team.

Participants will work in table groups in a fun simulation to design a deployment pipeline and learn ways to shorten feedback loops. They’ll look at ways to solve puzzles like how to complete manual testing activities like exploratory testing and still succeed with continuous delivery. These are exercises they can take back and do with their team to have valuable conversations around understanding and improving their pipelines.

They’ll practice using Ashley’s Test Suite Canvas to evaluate their existing automated test suites and think about what gaps they want to fill. Other areas to be explored include value stream mapping, operability areas such as monitoring and observability, and other aspects of fitting testing into continuous delivery. Engaging the whole team will be the main focus. We’ll use templates and conversation starters that participants can later try with their own teams.

The tutorial wraps up with each table group designing an experiment to help with one of the obstacles to CD identified earlier and sharing with the large group. Participants can take these ideas to experiment with their own teams after the conference. 

About the author

Lisa Crispin

Lisa Crispin is the co-author, with Janet Gregory, of More Agile Testing: Learning Journeys for the Whole Team (2014), Agile Testing: A Practical Guide for Testers and Agile Teams (2009), the LiveLessons Agile Testing Essentials video course, and “The Whole Team Approach to Agile Testing” 3-day training course. Lisa was voted by her peers as the Most Influential Agile Testing Professional Person at Agile Testing Days in 2012. She’s a testing practitioner who enjoys helping people find ways to build more quality into their software products. Please, and for more.

Register here to attend this tutorial –

Q and A with Maryam Umar

Q) What is your educational background and how did you come to be in the testing profession?

I did two degrees in Software Engineering.

Since school, I was always inclined towards computers and really enjoyed ‘thinking in pseudo-code’. During my Master’s degree, I had a module in software testing and I found my calling. My professor, Mark Harman, was really inspiring and I found my love for ensuring that the products created are always of top quality. I found my first job as a software tester and have never looked back since.

Q) What is it about this testing community that helps you and others thrive?

It’s always great to vent and discuss challenges with a wider community. Sadly, QA is still seen as an afterthought, and it’s good to learn how various people have tackled the same challenge differently around their teams.

Q) Why is it necessary to be a part of a community around us? Please state any pros and cons?

As I discussed above, the stronger the community the more you will learn about how to tackle the same problems. It’s about coming together for the common good of how we help deliver software for customers like ourselves.

Q) Why volunteering is more than necessary in today’s tech world?

The tech world today is very busy. New technologies and programming languages keep cropping up every day and it becomes very easy to lose sight of individuals who are entering the industry and how we can help them.

In recent years, I have found that being mentored and volunteering have played a huge part in my professional growth. When I have been stuck for ideas or direction, my mentors (Daniel Terhorst-North, Lisa Crispin, Isabel Evans, Jez Humble) have helped me think differently and in a more positive light. Volunteering for organizations like has given me a greater sense of purpose and a strong sense of needing to give back to the industry which has so much to offer for young talent. It has also helped to go the extra mile to help provide options and opportunities to those who wouldn’t have a network otherwise.

Q) Imagine that there was no testing community, how would it be different for professionals then?

I don’t think society today has come to exist without communities. 

If there was none for software testing, most of us would continue to reinvent the wheel without having the opportunity to use multiple open source frameworks which make the lives of our engineers easier. Because of these, they can then concentrate more on the real value products provide to the end-users and how we can better improve them.

Q) You are in a flourishing conferring phase; what factors help you to continue to confer?

I don’t think I am in a flourishing conferring phase yet 😉 It’s a long road and a bit of a bumpy one too.

For the places I do speak at, I think these factors contribute hugely to delivering successful talks:

  • Originality
  • Authenticity
  • Real-world examples

Q) How do you make time for work, travel, confer, volunteer and be active in the community?

It’s not easy at all. A friend of mine said I have 3 jobs in total. My day job, public speaking, and volunteering.

It has taken some discipline, but I try not to let the other 2 interfere during my day job. I make a list of conferences every few months and make sure I keep track of them. I add volunteering days and meetings in my personal calendar and make sure I get reminders in time.

A few tips:

  • I have merged all my calendars in one so I can see my week in one view. Use color coding!
  • I check my twitter feed regularly every morning during my commute as well as non-work slack channels. I revisit these during lunch hour and just before bed as well.
  • I try to read. Books, articles or write. I try to do this at least twice a week.
  • Inform your manager in time for conferences and volunteering days.
  • Talk about your passion with work peers as well. You may accidentally inspire someone else as well 🙂
  • Every 3-4 months, I try to go on holiday.

Q) How do you unwind on a non-working day, any tips to our readers on activities that they can do to rejuvenate and be prepared for the next working day?

Listen to Music that makes you happy. Gardening. Read. Watch a couple of series (I don’t binge-watch). Laugh with my best friends. Make sure you make time for personal care. I also like to shop 🙂

On a Sunday, I try to plan my week. 

  • Sort out what I am going to wear during the week. 
  • Review my calendar
  • Decide on my dinner menu for the week
  • Sort out any lunches I need to arrange
  • If I am hosting in the upcoming weeks, I try to organize everything at least 10 days in advance. It helps me plan my weeks up to the event better.

Q) What are some of the ways in which professionals who are working from home get active and contribute to this testing community?

Use Twitter. Follow the people who have written some of the best books in our industry. Seek to have a conversation with them.

Watch conference talks. Most of the talks are now recorded and posted.

Write. A lot of people now write blogs on LinkedIn, Medium, etc. Ministry of Testing is also another great place to contribute.

Initially, you may feel that no one is reading my stuff. Don’t let that stop you. Ask people to review your content and share it with their network. Trust me. It does work and you’ll be surprised by the power of writing.

Q) What is/was the biggest challenge in your professional work and how did you seek help from the community around you to solve it?

It’s still very difficult to get buy-in for automated testing and quality becoming everyone’s responsibility in engineering teams. It frustrates me incredibly that QA professionals are brought in when the customer starts complaining. And not at the inception of the SDLC.

Every time I face these issues, I reach out to my mentors. I also reach out to the Women in Testing group and fortunately, I get a few options that I can explore to help solve the above issues.

All I will say is, you must not be afraid to ask. Don’t let hesitation stop you from reaching out for help.

Q) What is your message to those who have the belief that changing jobs is not easy or isn’t a viable option for their career and mostly feel stuck at the same job/workplace?

I always say, if you wake up in the morning and don’t feel like going to the office, it’s time for a change. Workplaces should be a source of joy and drive as that’s where you spend at least 40 hours every week. It’s the same choice we should make like we did when going to school and seeking higher education.

When changing jobs, again, reach out to your network to understand if you are stepping into yet another workplace which may cause you the same frustration. Use Glassdoor reviews. Do your homework for salaries. Reach out to recruiters who you have been successful with before. Always negotiate. And don’t make it look like you are running away from a problem.

You are valuable and you must always work in an organization which values you.

About the author

My entire thirteen-year career has been about assuring quality in products ranging from mobile to online restaurant to travel services. Currently, I work as a QA manager with a special focus on sustainable delivery practices. I have found that creating teams which work well together is more challenging than the projects to be delivered by them. I pay special attention to team dynamics and ensure engineers are in roles which give them a sense of purpose. I also work with them to put systems in place which help teams optimize flow and deliver high-quality products.

The Magic of Notes.

Authored by – Arlene Andrews.

Far from being unneeded, taking effective notes, and having them trigger memories of an event, is a skill that is still very relevant. From sketch notes, doodles, and outlines – the process of manually taking notes is coming back into the fore. Finding your own personal style of note-taking will take practice. However,  you will find that many times, the notes you take are more effective than the ones you used to type on the computer, tablet, or mobile device.

In the rush of getting business done, especially during changes or in a quick meeting, the process of getting information stored is becoming vital. Not only is this a tracking and documentation issue, but makes very sure that the correct people know what is happening, who is in charge of specific areas. This also gives an accurate evaluation of the skills, contributions, and abilities of every team member – from the newest intern to the end customer themselves. But we are hurried, and this information is becoming more likely to be missed.

Unless there is someone consulting, there likely isn’t a recording made of meetings that everyone can refer to in the future. It is now almost expected that when you go to a meeting, your laptop is open and ready, and your phone is nearby. Neither of these encourages you to observe what’s happening during the meeting, nor pay attention to what’s being said. Being present and engaged is a problem when any words being spoken are funneled directly from the ears into the keyboard.

I know this is something I have a problem with: I will concentrate so much on typing out notes – or recording every word that said perfectly – that I don’t pay attention to what’s actually going on until I read it. One businessman I know has recently banned all laptops and telephones from any meeting – even a short stand-up. The theory is that people will pay more attention to discussions, participate more, and be more aware of potential issues. And it has created a noticeable effect.1

Taking notes is an important skill that can add value to your business and to your career. After much less practice than you may believe, you can quickly learn to sort out the relevant information you need and write it down with just enough detail to trigger memories of who was speaking and the subjects. There is even proof of this: Using a written form, on paper, engages both your focus and your motor memory to allow a better grasp of concepts, both with an instant review and over a short time period2. I can attest to this: my handwritten notes are valuable, and usually trigger thoughts and connections later. So not only do I have the original material, but, in a more relaxed environment, I can add other information, and explore questions and possibilities.

Your notes also provide proof of work. The quick notes that you make can be expanded on (and, if your handwriting is anything like mine, re-written so they are readable once the meeting is over), which also gives you the chance to add more room for idea connections and additional questions to what you heard. Also, since it is fresh in the minds of the participants, if there is an area that you are unsure of your understanding, this is a good time to ask for a detailed explanation. Which may, of course, result in another batch of notes. But you will then be able to move forward with confidence and have the notes to refer to at a later time.

From the Moleskine notebook and the pack of 3 x 5 cards that have become associated with one business owner I am acquainted with, the well-designed rooms for testers where tables and seats are designed with handwriting in mind, and the special notebook the Ministry of Testing has designed for one of their events, the tools we use can become part of our personal look, as well as being a truth source. The key here is to try many different tools, and use the ones that you find are effective and comfortable.

Notebook from Ministry of Testing group.

I have been, personally, delighted in the rise of sketch notes that are coming from the skilled folks at conferences. Not only are they a delight visually, but they also allow a look at how this person feels the concepts overlap. I am not skilled with drawing, but even a quick set of boxes to show a workflow or a larger question mark over a section where I can reserve this area for topics I want to ask about has proven to be a help. A resource for ideas on how to start with sketchnotes is

Other techniques are out there: everything from a formal column system to doodles that separate and define areas of importance. And workshops on taking notes are starting to come up at many of the larger conferences – as well as being shared online. Every person has their own style – mnemonic devices that make sense to them – and this makes the notes you take even more powerful than a word-for-word transcript of what was said.

Let your personality and focus make the notes for you!

And this brings up the issue of “Who takes notes?” A clichè is that one person is chosen to make notes – every time – and anything that is missed, or not understood is their problem. Not only is this a potential downside if that person is not there that day, but they have to try and get notes for everyone’s point of view. The benefits of note-taking by more than one person means that you will have notes that focus on what is important to you – and a team may have several people that have overlapping notes. This helps with understanding and making sure that everyone is getting an understanding of the important concepts. Even Lynda, the online education site, offers Note-Taking for Business Professionals (

There is also the need for fail-safes. Recorded meetings are much less common than the used to be, so your notes now are the sources of truth. Yes, you can get a quick meeting with someone in the hallway to clarify a point – but write it down, in your own words, once you are done. This is never to be used as blame but to have a good working plan for this section of the project, or meeting, or activity. Once the larger meeting is done, the people working together have an understanding of what needs to be done, and they can take these notes, and make the first plan to get it accomplished.

Try this

In the age of the Internet, there are many possibilities to practice note-taking. Many conferences or businesses have either recorded talks or webinars, and these are great for practice. Choosing a familiar subject will allow you to try out different styles and methods, as well as increase your awareness of that subject. Later, a TED or TED-X talk is about 15 minutes, and the information is concentrated, as it may be in a meeting to solve a crisis. This is a great way to make sure that you are getting the important points. I also suggest choosing a topic that is of interest, but above your level of knowledge – you may be surprised to find out how much you do actually know about the subject, once you read over your notes.




About the author

Arlene Andrews

Arlene Andrews is a self-guided learner, moving into the Quality Advocate section of the tech world, and has been selected to guest write and review on podcasts, articles, classes, proposals for conferences. Having previously worked in a wide range of positions including both customer service and HR, she has a worldview of supporting customers, and is a fan of learning and applying new things. Moving into technology has been a huge leap, and she has a passion for connecting people to resources that will foster improvements.

Reviewed by – A volunteer at women testers.

Conference submissions – why and how?

Originally published here:

Recently someone asked me about how to become a conference speaker. I have spoken at conferences, and also served on programme committees, so I hope these thoughts are helpful to you in your quest to speak. Additionally, I been giving feedback to people whose submissions did not make it onto the EuroSTAR programme this year and who asked for feedback, and seen some common themes, including that with over 400 people applying for around 60 speaking places, and an excellent field of submission, many great submissions did not make it to the conference programme… not being selected doesn’t necessarily mean you did a bad submission.

Why speak at a conference?

My first question to you: Why would you want to speak at a conference? It is after all time consuming, stressful, and unlikely to be in the obvious mainstream of your job. Here are some reasons I speak:

  • to improve how I communicate about my subject – a skill for work.
  • to learn my subject: to give the talk, I’ll have to learn more, check facts, build my story.
  • to give back to my industry and educate others, by sharing challenges overcome.
  • for the fun of performing: it’s scary and fun, and a chance to play in public…

So, you want to speak at a conference… what to do? I’m assuming you have a story to tell, one you think is worth other people hearing? If you have not got a story to tell, there is no point speaking…

Don’t wait to be asked…

There are two ways to get a speaking place at a conference: you get invited, or you apply via a “call for submissions” (cfs) or “call for papers” (cfp). However famous you are, you might not get invited, so, if you want to speak at a conference, don’t wait to be invited. Instead, apply to speak. Your submission will be reviewed, and you will be accepted, or rejected. Don’t worry if you are rejected, it has happened to all of us – many times in my case. Conferences often have many more applications than they have speaking places. So review, and try again…

Choose your conference…

First job: decide which conferences you want to speak at, look at their websites to see what dates they run on and what style of submission they want. Look very carefully at any guidelines, themes, and style sheets they suggest. Also look at the websites for previous editions of the conference to see if there is a “house style” the conference favours. Also think about whether you can get to the conference if selected – travel visas, availability, dates and costs – can you go if you are selected, and how will you fund it? Some organisations will support you because they want representation at the conference. Some conferences provide funding towards travel and accommodation. When you are applying look at the balance of benefits and costs. Each of us will have a different view about what we want to do, what cost/benefit we need to make it worthwhile.

Investigate what information the cfp requires

Look at the session options offered carefully. Think about what they want for different types of session. Typically the minimum you will be asked for along with contact details is:

  • A title
  • An abstract
  • Your biography

You may be asked for a paper to explain your idea. You may be asked for key learning points, takeaways, what type of session this is, what type of audience it is aimed at, your speaking experience, evidence in the form of supporting documents, videos… it all depends on the conference.

What helps your submission succeed?

Factors that will help your submission for many industry conferences include:

  • telling a really good story – something compelling, coherent, concise, and which flows from the title, into the abstract and through to any takeaways.
  • focusing on your experiences of your projects – things where you are demonstrating your involvement, challenges you have faced and overcome, mistakes you have made and learnt from – rather than using your abstract to regurgitate theory.
  • having a new perspective to offer, something that has not been offered at this conference before.

If you don’t have speaking experience, think about getting mentoring – within your local/national industry communities, within your organisation, or via the conferences. You could look at SpeakingEasy for example ( ). Also look for opportunities to speak at local meet ups and national conferences before going for the larger international conferences. It’s likely that fewer people will apply and this increases your chances of being selected.

Conferences will often have themes that change year to year. Many conferences in addition are looking for speakers and sessions that increase the diversity of ideas and people, improve inclusiveness, are engaging, participative and interactive, allowing the audience to not just listen but also take part.


  • Have your own compelling story
  • About something unique, transformational
  • About overcoming challenges
  • Provide evidence!
  • Keep it coherent, well focused
  • Keep it clean…
  • Ask for help!
  • Get it reviewed
  • Get it proof read
  • Speak at smaller events first…
  • Ask for feedback

What to avoid

Don’t just send the same abstract to different conferences – they each want something different. Don’t send the same abstract several times to the same conference for different session types – it just annoys the programme committee. Don’t send an excessive number of submissions – it is better to have one really well thought out abstract.


  • Forget to spellcheck
  • Forget to tell your story
  • Present no evidence
  • Use bad language
  • Assume we know who you are
  • Ignore the conference style
  • Forget to ask for time off…
  • Expect to get in … necessarily

Useful links

Here are some useful other blogs and links…

Rob Lambert’s “Blazingly simple guide…”:

Steve Watkins’ “How to prepare…”


Good luck!

and give it a go – you won’t get in unless you try!

We also asked Isabel this question: Why should testers attend testing conferences?

..conferences are fun, but they are not just fun. Here are some reasons to go to conferences which benefit you and your employer/team:

  • To learn new ideas, new knowledge
  • To gain new skills at a tutorial or workshop
  • To meet industry experts
  • To meet colleagues
  • To share experiences
  • To present your own experiences
  • To showcase progress that your company has made
  • To meet vendors of tools and services so you can shortlist ones to approach after the conference
  • To try out consultants and trainers at their workshops and tutorials
  • To practice your skills and improve them
  • To practice presenting your own ideas to other people
  • To find out new directions for your profession, future predictions
  • To make friends with other practitioners – lifelong friends that you are also happy to work with and recommend

All of these benefit you personally in your professional growth. They also benefit your employer and your team, because you will return with ideas you can implement in your work, and contacts you can recommend for projects and teams.

Although it is possible to learn through reading, attending online events, taking virtual courses and in face to face training courses, in each of those you’ll have an experience limited by the medium for the interaction or by the number of people involved. When you meet people at conferences you’ll have a chance to hear many experiences and viewpoints, challenge your own and other people’s preconceptions of what your role does/could involve, make human connections, all in a way that is really only possible face to face.

I always find conferences really hard work, but really valuable – and they are as valuable as the effort I put in.

It is worth going with a plan – what sessions do you really want to hear? What subjects do you want to investigate? Who do you want to meet? And make that happen. Make sure you don’t just go to sessions to listen, but also take part in workshops, discussions, lab sessions, community activities. Give as well as take. Make sure you allow some “down time” when you are just chatting with friends you’ve made at this conference. Do attend a social event or two! The technical and managerial conversations continue there.

Above all, be thinking about the messages you are going to take back to your team and organization. Finish the conference with a plan of action, and an “in 1 minute” overview of the main benefits. That will help you think about the conference as an opportunity to learn and benefit rather than as time off…

Isabel Evans Bio

Independent quality and testing consultant Isabel Evans has more than thirty years of IT experience in the financial, communications, and software sectors. Her work focuses on quality management, software testing and user experience (UX), She encourages IT teams and customers to work together via flexible processes designed and tailored by the teams that use them. As well as her consultancy and project work, she is currently researching into the UX of testing tools for the tester, as a part-time, mature, postgraduate student at the University of Malta. Isabel authored Achieving Software Quality Through Teamwork and chapters in Agile Testing: How to Succeed in an eXtreme Testing Environment; The Testing Practitioner; and Foundations of Software Testing. A popular speaker and story-teller at software conferences worldwide, Isabel is a Chartered IT Professional and Fellow of the British Computer Society and has been a member of software industry improvement working groups for over 20 years. In November 2017, she was presented with the EuroSTAR Testing Excellence Award for services to the industry. She has a website: She has two blogs: and Her linked in page is at She tweets occasionally as @IsabelE_Test.


The tribe is a supportive online community that invites you to experience connection, nourishment, and joy. To create space to grow, expand and flourish in your body, mind, heart, and soul. As you reconnect with all parts of yourself, you will flourish at work, in relationships, and all parts of your life.

Is this for you? The tribe is ideal for anyone that feels or experiences:

  • Overwhelm
  • Stress
  • Lack of time
  • Lack of finances
  • People pleasing
  • Drained of energy
  • Lack of clarity + focus
  • Feeling depressed, or disconnected
  • Unhealthy relationships with self and/or others
  • Difficulty integrating into everyday life
come join the tribe!

The Radient Soul Tribe is a monthly membership community with a variety of weekly content filled with practices and tools to support you in each area of your life (and work). It’s a space to dive deeper into yourself, and to step into joy and peace!
Curious? Get more details here with a $1 one-month trial:

We welcome you to thrive with us!
Selena Delesie & Cindy Brown

Useful tips for API Tests with Postman

Postman is powerful tool to work with Rest APIs. It allows to test APIs manually. It allows creation of automated/randomised tests using request scripts and tests option. Postman collections can be executed outside of postman app using newman JS package, which is handy for automated test executions in Jenkins.

Below you will find helpful information and tips on How to setup and test API through postman. These information will be very helpful for the tester to set up, create, run and validate API tests.

A set of API’s can be grouped together is called collection, for easy access and management of API endpoints.


REST stands for Representational State Transfer. REST is web standards based architecture and uses HTTP Protocol.

Four HTTP methods are commonly used in REST based architecture:

GET – This is used to provide a read only access to a resource

PUT – This is used to create a new resource

DELETE – This is used to remove a resource

PATCH – This is used to update an existing resource or create a new resource

Test node.js APIs

Testing via Postman (similar steps for any other REST client)

  1. Open your postman and type endpoint in the enter request URL section and press enter
  2. Choose correct method
  3. If selected method is POST, go to body section, choose RAW and add request body
  4. Add authorization headers (if required)
  5. Click  “Send”
  6. This should give you a response 200 ok.

Note: ideally you can add tests that automatically validate response headers or data in the body.

Test Automation for node.js APIs

Test structure

Postman project can consists of following parts:

  1. Login scripts
  2. Test automation:
    1. Automated tests for GET methods
    2. Automated tests for POST methods


If API does not support anonymous access, meaning automation has to supply authorization header with valid authorization bearer. There are two options:

  • Provide valid authorization bearer manually each time (it expires after several hours)
  • Create scripts that will emulate login process and extract authorization header from server responses

Second approach was selected.

Login process consists of the following steps:

Step – 1 [fed – get cookie] – as a first step of automation validates presence of all environment variables. This step is required to finish first part of oAuth process: get Cookie PF and authorization token in path for the next call

Step – 2 [fed – authorize. set token env var]

This step is required to POST credentials and authorization token to receive Authorization Bearer

Step – 3 [Test request with retrieved token is successful]

This test is required to check validity of the received token automation

GET API automation can be created with an example for the following method:

  • GET /v1/xyz/:xyzid/T1
  • GET /v1/xyz/:xyzid/T2

POST API automation is created for the following method:

  • POST /v1/xyz/data for:
  • xyz1
  • xyz2
  • xyz3

Test asserts:

  • GET /v1/xyz/:xyzid/T1

tests[“Status code is 200”] = responseCode.code === 200;

tests[“Content-Type is present”] = postman.getResponseHeader(“Content-Type”);

tests[“Response time is less than 1000ms”] = responseTime < 1000;

tests[“string1 is present”] = responseBody.has(“string1”);

tests[“string2 is present”] = responseBody.has(“string2”);

tests[“string3 is present”] = responseBody.has(“string3”);

tests[“string4 is present”] = responseBody.has(“string4”);

tests[“string5 is present”] = responseBody.has(“string5”);’

tests[“string6 is present”] = responseBody.has(“string6”);

tests[“string7 is present”] = responseBody.has(“string7”);

tests[“string8 is present”] = responseBody.has(“string8”);

// M

tests[“value1_M should be “] =jsonData.items[5].value1 === “abc”;

tests[“value2_M should be “] =jsonData.items[5].value2 === “abc1”;

tests[“value3_M should be “] =jsonData.items[5].value3 === “abc2”;

tests[“value4_M should be “] =jsonData.items[5].value4 === abc3;

//tests[“value5_M should be “] =jsonData.items[5].value5 === abc4;

tests[“value6_M should be “] =jsonData.items[5].value6 === “abc5”;

tests[“value7_M should be “] =jsonData.items[5].value7 === “abc6”;

tests[“value8_M should be “] =jsonData.items[5].value8 === “abc7”;

// L

tests[“value1_L should be “] =jsonData.items[17].value1 === “abc”;

tests[“value2_L should be “] =jsonData.items[17].value2 === “abc”;

tests[“value3_L should be “] =jsonData.items[17].value3 === “abc_1 “;

tests[“value4_L should be “] =jsonData.items[17].value4 === “abc2”;

tests[“value5_L should be “] =jsonData.items[17].value5 === abc3;

//tests[“value6_L should be “] =jsonData.items[17].value6 === abc4;

tests[“value7_L should be “] =jsonData.items[17].value7 === “abc5”;

tests[“value8_L should be “] =jsonData.items[17].value8 === “abc6”;

tests[“value9_L should be “] =jsonData.items[17].value9 === “abc7”;

tests[“value10_L should be “] =jsonData.items[17].value10 === “abc8”;

tests[“value11_L should be “] =jsonData.items[17].value11 === “abc9”;

  • GET /v1/xyz/:xyzid/T2

tests[“Status code is 200”] = responseCode.code === 200;

tests[“Content-Type is present”] = postman.getResponseHeader(“Content-Type”);

tests[“Response time is less than 2000ms”] = responseTime < 2000;

tests[“stringa is present”] = responseBody.has(“stringa”);

tests[“stringb is present”] = responseBody.has(“stringb”);

tests[“stringc is present”] = responseBody.has(“stringc”);

tests[“stringd is present”] = responseBody.has(“stringd”);

tests[“stringe is present”] = responseBody.has(“stringe”);

tests[“stringf is present”] = responseBody.has(“stringf”);

tests[“stringg is present”] = responseBody.has(“stringg”);

tests[“stringh is present”] = responseBody.has(“stringh”);

tests[“stringi is present”] = responseBody.has(“stringi”);

tests[“valuea should be “] =jsonData.items[0].valueal === “abc”;

tests[“valueb should be “] =jsonData.items[0].valueb ===”abc1”;

tests[“rvaluec should be “] = (jsonData.items[0].valuec >= -abc2) && (jsonData.items[0].valued <= abc3);

tests[“valued should be “] =jsonData.items[0].valued === abc4;

tests[“valuee should be “] =jsonData.items[0].valuee === “abc5”;

POST commands tests: for following methods:

  • POST /v1/xyz/command for:
    • xyz1

tests[“Status code is 200”] = responseCode.code === 200;

tests[“Response time is less than 1200ms”] = responseTime < 1200;

tests[“Content-Type is present”] = postman.getResponseHeader(“Content-Type”);

tests[“value1 is present”] = responseBody.has(“value1”);

tests[“value2 is present”] = responseBody.has(“value2”);

tests[“value3 is present”] = responseBody.has(“value3”);

tests[“value4 should be “] = jsonData.items[0].value4 === val4;

tests[“value5 should be “] =jsonData.items[0].value5 ===”Test1”;

  • xyz2

tests[“Status code is 200”] = responseCode.code === 200;

tests[“Response time is less than 1200ms”] = responseTime < 1200;

tests[“Content-Type is present”] = postman.getResponseHeader(“Content-Type”);

tests[“value1 is present”] = responseBody.has(“value1”);

tests[“value2 is present”] = responseBody.has(“value2”);

tests[“value3 is present”] = responseBody.has(“value3”);

tests[“value4 should be “] = jsonData.items[0].value4 === val4;

tests[“value5 should be “] =jsonData.items[0].value5 ===”Test2”;

  • xyz3

tests[“Status code is 200”] = responseCode.code === 200;

tests[“Response time is less than 1200ms”] = responseTime < 1200;

tests[“Content-Type is present”] = postman.getResponseHeader(“Content-Type”);

tests[“value1 is present”] = responseBody.has(“value1”);

tests[“value2 is present”] = responseBody.has(“value2”);

tests[“value3 is present”] = responseBody.has(“value3”);

tests[“value4 should be “] = jsonData.items[0].value4 === val4;

tests[“value5 should be “] =jsonData.items[0].value5 ===”Test3”;

Create a repository for API automation

This should contain PostMan scripts for automating testing of the API’s.

Simple example below for creating test –

Test for node.js APIs (similar steps for any other REST client)

  1. Open your postman and type endpoint in the enter request URL section and press enter
  2. Choose correct method
  3. If selected method is POST, go to body section, choose RAW and add request body
  4. Add authorization headers (if required)
  5. Click  “Send”
  6. This should give you a response 200 ok.

You can add tests that automatically validate response headers or data in the body.

How do I get set up?

Someone else in the team can run and test API tests created above, they need to follow below steps

  1. Clone the repository that was created above
  2. npm install
  3. npm run build

The above steps are performed by the build system on every check in.

IMPORTANT: Make sure and run npm run build before pushing changes to the original. You can also run npm run lint to verify that your code passes the lint test as well as auto fix a number of linting issues.

How do I add new npm module?

If you want to add new test

npm install –save <name of module> npm shrinkwrap

Check in updated npm-shrinkwrap.json and package.json file.

How to run collection in Postman?

  1. Click on collection and click button “>” with mouse
  2. Press “Run”
  3. In Collection Runner window check if correct collection is chosen
  4. Choose Environment file and press “Run <Collection’s name>”

How to run collection in Newman?

  1. Open Command prompt
  2. In command prompt type this command if you use collection file:

newman run Path/CollectionName.json -e Path/EnvironmentFile.json

or this if you use collection link:

newman run <link to collection> -e Path/Environment.json

  1. Press “Enter”

How to import collection or environment file?

  1. In Postman press “Import” icon on top left side of window
  2. Choose “Import File” tab
  3. Press “Choose Files”
  4. Choose file you want import
  5. Press “Open” button

How to import collection or environment from link?

  1. In Postman press “Import” icon on top left side of window
  2. Choose “Import From Link” tab
  3. Paste link to collection or environment file
  4. Press “Import” button

How to export collection as link?

  1. In Postman choose collection you want to export
  2. Press button “>” with mouse
  3. Press the “Share” button
  4. Choose “Collection Link” tab
  5. Press the Get Link” button
  6. Copy link from field

How to export collection as Json file?

  1. In Postman choose collection you want to export
  2. Click on it with mouse right button
  3. Choose “Export”
  4. Choose version of collection
  5. Press “Export” button
  6. Choose path and name for collection file
  7. Press “Save” button

How to edit test in collection’s request?

  1. In Postman choose collection
  2. Choose request and double click on it to open
  3. Choose “Tests” tab
  4. Change test code as you wish
  5. Press button “Save” to save changes
  6. (Optional) Press button “Send” to send request and get test results as well
  7. Press button “>” with mouse
  8. Press “Share” button
  9. Choose “Collection Link” tab
  10. Press “Update Link” button

How to log test?

  1. In Postman choose collection
  2. Choose request and double click on it to open
  3. Choose “Tests” tab
  4. In test code use log command where you wish. Example of log command:

console.log(“so far”);

Benefits from API testing with Postman

Benefits from API testing with Postman are below

1. Easy to Use, manage & share

One can run any API endpoints individually without much steps involved in setting up.

Postman makes it very easy to create test suites and at the same time you can store and run it efficiently. You can store required information to run the tests effectively.

You can run test collections multiple times for testing in different environments.

You can easily share your test collections with other team members. You can also export test collections.

2. Store data for use in other tests Can run Tests as workflow

One API endpoint response can be passed to another API endpoint to run, so it allows you to run the flow from one api to another.

3. CI Integration

You can easily execute the API test collection with newman from CI tools, and you will have job results of pass/ fail on the individual test case run results.

Rama Anem’s Bio

Rama is a technical professional leading a team of high performing senior consultants, data quality analysts providing technology solutions. She has over 13 years of experience, has worked with companies like IBM, AMD in past, and led global teams. At AMD, she managed the enterprise-wide data platform quality practice and was responsible for building its first center of excellence. At IBM, Rama led multiple teams and served as one of the brand ambassadors for DB2. In addition to writing articles in a variety of technical journals, she also presents advanced topics at international software conferences and workshops.

How to use Design Thinking TO ENHANCE SOFTWARE Quality

Authored by Software tester Rozalia Baé – a first-time contributor to Women Testers e-magazine.

“The people who are crazy enough to think they can change the world are the ones who do.” – Steven Jobs

Design thinking for software testing can be used for the purpose of generating creative ideas and thereby discovering its own creativity. It offers the possibility to find new solutions by other means.

Design Thinking

“Innovation distinguishes between a leader and a follower.” -Steven Jobs

word cloud
A word cloud of words related to design thinking

Design Thinking cannot be described in just one word. Each individual feels addressed to a particular word and interprets for himself/herself what it means to him/her. Design Thinking has many definitions and each of them is unique, diverse and inspiring.

Design Thinking should help to localize problems, visualize them, and find an individual and creative solution for them. One of the goals is to present the complexity of the problems in small steps and to solve them. It also serves to show a change of perspective and to find solutions by some other means. In doing so, one’s own creativity is to be revealed. Design Thinking can be applied to discover one’s own creativity and develop it further.

Design Thinking for Software Testing

“It’s not a faith in technology. It’s faith in people.” – Steven Jobs

In the age of globalization, it is necessary to enter in the competition and to keep up with the competition. For software testing, it means developing innovative test methods, test types, and test strategies, just as much as testing. This can be achieved by a fundamental change in thought and action. All software testers, developers and managers should be involved in this process of change. Such an innovative practice requires open-minded employees, managers and the company itself.

Design Thinking Algorithms

“Sometimes when you innovate, you make mistakes. It is best to admit them quickly, and get on with improving your other innovations.” – Steven Jobs

Design Thinking is a systematic approach to complex problems. It is geared to the needs of users/customers and is intended to create innovations.

The process is based on three components:

The different perspectives for consideration of the problem are expressed within the following questions:

  • Human Values (desirability): what exactly is desired?
  • Business (viability): what is economically feasible?
  • Technology (feasibility): what is useful/realistic?

Only when all the above questions have been satisfactorily answered can one consider this as a path to innovation.

The core elements of Design Thinking are

  • Multidisciplinary Teams
  • Variable Space
  • Design Thinking Process

Tips on better Brainstorming:

  • Defer judgment
  • Encourage wild ideas
  • Build on the ideas of others
  • Stay focused on the topic
  • One conversation at a time
  • Be visual
  • Go for quantity

Design Thinking for Software Testing: Example

The application will be illustrated and clarified with the help of an example.

Answers to the questions:

  • Human Values (desirability): what exactly is desired?
    • Improvement of software quality through a better testing methodology
  • Business (viability): what is economically feasible?
    • It is economically viable
  • Technologie (feasibility): what is useful/realistic?
    • It is useful and realistic

Multidisciplinary Teams

A creative process requires people from various fields with their diversity, culture and different experiences. This is why multidisciplinary teams are a win for every company. Diversity is also an important success factor for software testing.

In order to generate innovative ideas for software testing, people from diverse areas might be involved. Other than for software testers, there also is a need for programmers, project managers, product managers, marketing managers and designers. Each person is an asset.

Variable Space

The aim of testing is to improve and enhance the quality of the software. Thus, testing is always closely tied to finding creative and innovative ideas.

Design Thinking offers the possibility to generate a creative space (work environment) where creativity can evolve, detached from purely analytical and linear thinking.

The goal of this creative space is to produce a feeling of freedom, thriving and serenity.


Discover new heuristics for Software Testing with Design Thinking

What are Heuristics?

Heuristic is a strategy that uses information to control problem solving in humans and machines. Heuristics are “rules of thumb”, they are experiential assumptions and also intuitive judgments.

The heuristic is a technique for experiential software testers that can help with problem-solving as well as discovering defects. A heuristic method is then used to quickly come up with a solution that is close to the best possible solution.

In software tests, heuristics provide a simple framework. It is based on past data and probabilities and helps to effectively test programs. By highlighting the location where bugs are found, a heuristic can find new important bugs, no matter how much you know about the program.

Design Thinking Process

Design Thinking process is an iterative process and offers feedback on the previous process steps.


  1. Understand
  2. Observe
  3. Point of View
  4. Ideate
  5. Prototype
  6. Test
5af04.png (1346×454)

References: HPI School of Design Thinking


The goal here is to understand the importance of software testing more precisely and emphasize it. Furthermore, here the resulting problems, both the past and the future ones, can be explained. Just as well as to how the risks have risen.

Consider: The vulnerabilities that have emerged in solving the problem of testing are explained and presented here


Here the insights from observation are gained. This step is essential because one directly deals with the topic of testing, problems, and risks. Talks are held and they are also heard. Each person explains their own approach and how they solve problems during testing. Everything is documented in a most precise manner.

Consider: The observations resulting from the understanding are analyzed and discussed here

Point of View

This step leads to new perspectives and is the foundation of the creative process. It shows that software testing requires more creative and innovative solutions with regard to the problems and risks. Furthermore, hereby the problems and risks from various perspectives are addressed and investigated.

Consider: The solution is discussed and presented here: finding new heuristics to increase finding bugs


In this step, ideas are collected without evaluating them. The collection of ideas takes place in the form of brainstorming, mind mapping, etc. Ideas that are initially were considered unrealistic can help find new ideas (see above “Tips on better Brainstorming”). In terms of software testing, ideas for solving problems are sought here in order to solve the problems and risks.

Consider: To collect new ideas for heuristics think of the below methods.

New Heuristic generation (examples)

→ Palindrome

→ Permutations

→ Play dice

→ Crosswords

→ Dichotomy

→ Ambiguity

→ Puzzles

→ Learning of test oracles

→ Transpose numbers

→ Use of data anomalies


A prototype is developed using the aforementioned steps. This prototype can be in the form of a model. The prototype is a possible approach to solving a problem, which is at first investigated and checked. The prototype is illustrated and visualized.

Consider: The Prototype Heuristic Model

New HeuristicsExample
Palindromefunctional testing

System Testing
Permutationssecurity testing
Play dicevisual testing
Crosswordsexploratory testing
Dichotomydata validation testing
Ambiguitystress testing
Puzzlesfunctional testing
Learning of test oraclessoftware migration
Transpose numbersintegration testing
Use of data anomaliesnon functional testing


The prototype is used and tested immediately. For software testing, this could mean that the solution is successful and could be integrated into testing as a fixed essential part. Or it can be seen as a way to further expand the idea.

Consider: These heuristics can be used in sprints (agile) to test how they work. In a positive application, these can later be included in the regression test


“Have the courage to follow your heart and intuition. They somehow know what you truly want to become.” – Steven Jobs

Design Thinking is DYNAMIC and can therefore also be adapted and applied to all disciplines.





[4] Book: “Design Thinking Live”

Christoph Meinel, Ulrich Weinberg, Timm Krohn

[5] Harvard Business Review

“The Evolution of Design Thinking”, September 2015

[6] wikipedia

[7] Test Heuristics Cheat Sheet

[8] Heuristics of Software Testability

Author bio:

Rozalia Baé works as a Software Test Engineer in scrum environment. As a student she worked for SAP Germany and started there her career as a Software Test Engineer. She has testing experience in industry and commerce software, medical software and mobile app testing. She is a Certified Tester of the ISTQB Foundation Level (CTFL) and Certified Tester Advanced Level – Test Manager (CTAL-TM).

Samanta Cicilia’s year as a tester 2018

2018 was a special year for me as a professional, I had an opportunity to go to my first international conference Test Bash San Francisco, participate in many meetups and talk a lot about the role of the Tester/QA in Software Engineering.

2018 was an important year for me as a tester because it made me think a lot about what software quality really means and how we can contribute to building the best possible product. At first, the role of the tester was basically to check if the software meets the requirements and this was done manually, the quality was seen merely as a check. With agile came a great need to optimize how to test and reduce the delivery cycles, then began a high investment in test automation, processes, choice of tools and paradigm shifts. I believe this was the second wave of the role of tester.

A little later, when we started talking about the DevOps Culture, the importance of ensuring quality throughout the cycle with different types of tests, Continuous Integration and Monitoring was reinforced, but I feel that at that moment the role of tester ended up focusing too much on tools and forgetting that Quality is much more linked to the user’s perception of our product than to the use of techniques and tools alone.

For me, 2018 has been a year back to the origins of what software quality really is and how we can create incredible digital products that exceed user expectations, and the answer to that is to define a strategy that focuses on the different people that interact with our product. I believe this is a third wave that will stay for a while and again modify the behavior and contribution of the tester within a team.

Thinking of examples, imagine an e-commerce site that has millions of hits every week, it’s not enough to simply automate flow tests, such as adding to and buying a cart, to guarantee quality. One, two, ten concurrent users may even be able to make a successful purchase, but what if it’s a thousand? Just ensuring functionality without thinking about performance and scale is not enough.

I really liked a phrase I saw in Alan Page post: “the role of testing is to accelerate the achievement of shippable quality”, so we have to think about what helps us to carry out this role and help our companies to have a market differential, delivering quality software in short cycles.

Another point that marked a lot of 2018 (here in Brazil) was that companies began to look more towards accessibility and how much this is an inclusion tool for people who have some impairment. The testers also had to learn more about regulations and how to test for accessibility to ensure that a blind person, for example, can make
a purchase.

Talking about communities, it was also a year that strengthened the creation of Meetups in different capitals of Brazil, where I shared a lot of knowledge and exchange of experiences, I am very happy to see Brazilian testers coming together and sharing ideas.

It was a year of considerable reflection and much learning, which culminated in a change of job, I left a Brazilian Consulting and started 2019 in an American Product Company, where I’ll put all this learning into practice.
Another nice thing was my return as a writer, I hadn’t done many posts for a long time and in 2018 I was able to resume this habit of writing and sharing knowledge. If you want to know more about my posts and materials, everything is available on my site
A great 2019 for all!


Author bio

Samanta Cicilia

QA Manager at PayCertify, helping our team to define Quality Strategy to create an incredible product.
I am passionate about agility. I believe in empiricism, continuous improvement and diversity.
Co-founder of the Carioca Testing Group that is now part of Ministry Of Testing ( and organizer in MoT, São Paulo.

Articles of QA published in

Specification By Example and Product Quality

Target audience: Product People, Testers, Developers

I’m sure that in your workday with software development you’ve heard words like BDDATDD, and Specification By Example. Most of us link these terms to just the test portion of an application but do not observe how powerful the practices are.

Rather than just contributing to the test of what your team is developing, they help ensure that it is actually doing the thing — what your customer needs!

To try to clarify a little better where these words came from and how we got here, I set up a timeline based on several different authors. These authors have written books covering the breadth of software testing and its principles. In the end they’re all basically saying the same thing, but they’re also presenting the theme according to their point of view.

A Little History*_-NRLWoFi2lkQYtGW88i5w.png

Figure 1

1999 — Kent Beck (Extreme Programming Explained: Embrace Change)

In this book, Kent Beck presents the values, practices, and principles of XP. Although he doesn’t talk about the term “Specification By Example”, he talks about continually specifying and refining during development, so that customer learning is reflected in the developed software.

In the cycle he proposes, the stories are specified immediately before they are implemented. After which, they become extracted tests of those specifications. In the cycle that follows —

  • The interface is designed to meet test needs
  • The code is written to match the tests and the interface
  • The design is refined to match the needs of the written code

This leads to a decision about which story to specify next. Meanwhile, the rest of the stories remain stationary, until they are chosen for implementation.*3E0JkiXfNSOwYjRvkRS7Ww.png

Figure 2

2004 — Martin Fowler — Specification by Example

In this article, Martin uses the phrase “Specification By Example” to refer to what Kent Beck had called specs in the XP part of testing. Instead of dealing only with behavioral specifications (thinking about preconditions and postconditions), the word example is added to make us think in a more humanized way in terms of actual use.

An important point that he raises is that Specification By Example alone does not solve all problems. You need to always have the support of more than one tool to achieve your goals and solve problems.

He also reinforces that this only works in collaborative environments — where all the people involved in software development are together building the best product.

“Always have the support of more than one tool to achieve your goals and solve problems”

2005 — Lisa Crispin — Customer Test-Driven Development

Communication and collaboration are two key words in this article by Lisa. For each story, she asked the client how he would know that the story was complete — in the form of tests and examples.

Just like in TDD, before writing any code, the tests are written first — and then if they pass, it means that the code meets the requirements. The tests should ideally be written in a way that can be understood by an automation tool. It should also be at a high level so that they can guide the exploratory tests.

“Write tests that can be understood by an automation tool”

2006 — Dan North — Introducing BDD

During his journey using TDD, Dan North noted some difficulties, such as — where to start, what to test and what not to test, how to call the tests, and how to understand why a test fails.

BDD is an agile practice that allows better communication between all technical and non-technical people involved during a software project. It describes a cycle of iterations with well-defined outputs and resulting in the delivery of tested and functioning software.

“BDD allows better communication between technical and non-technical people involved in a software project”

2008 — Elisabeth Hendrickson — Driving Development with Tests: ATDD and TDD

In this material, Elisabeth talks about executable requirements, where you create tests before coding. These tests represent the behavior that the software should have. It reminds you that it is very important that this work be done together with the stakeholders.

According to her, the specifications give feedback on how close the team is to the “done” of a story, giving even an idea of progress toward the goal.

2009 — Gojko Adzic — Bridging the Communication Gap: Specification by Example and Agile Acceptance Testing

The most relevant point here in this Gojko publication is — specification is not a programming technique, but a communication technique. It brings people involved in software development closer to the project.

“Specification is not a programming technique, but a communication technique”

2010 — Lisa Crispin — ATDD X BDD X Specification By Example

At Agile 2010, during a workshop on Functional Testing, about 20 people from the international software community were discussing what ATDD really meant.

As you can see in the summaries of each of the sessions above, several people were saying the same thing but with different names and some particularities — (examples, acceptance tests, scenarios, user tests, behaviour, functional tests, executable specification …).

As a result of this workshop they came up with a definition of Specification By Example that would be described in 2011 in Gojko’s book of the same name.*6UKh2aYAar6swiN5PTmGMg.png

Figure 3

2010 — Craig Larman — Specification By Example

When Craig talks about Less, he devotes a part to techniques that help achieve technical excellence. And guess what — the Specification For Example is there (as well as Integration and Continuous Delivery, Clean Code, TDD and others).

According to him, tests should be regarded as requirements and requirements as tests. The book reminds you that if you have a test that describes the behavior, and at the end of development this test passes — it means that you have reached the goal of that functionality.

In addition, he cites other elements we have seen in previous blocks, such as — doing workshops to collaborate and discovering the requirements, working on short development cycles, and focusing on problem prevention rather than finding.

“Tests should be regarded as requirements, and requirements as tests”*MDzNFZbAaQIjq69giVVggA.png

Figure 4

2011 — Gojko Adzic — Specification by example: How successful teams deliver the right software

It was through this book that I came to know the subject. Two years ago as I entered Concrete, it was the first book I received as a reference on QA’s work.

In this book, Gojko brings together a series of standards that have been observed in several successful projects. He defines Specification by Example as a set of these standards that help to develop the right software.

Since then I have been using the references in this book by Gojko when talking about Specification By Example. But in doing the research for this post, I saw that his book is a summary of what many other people were talking about the world.

Get Down to Business

Going back to the beginning of this text, many people associate Specification Examples with only the testing part of the application. But they do not use all of their tool-ing to effectively build quality products — both in terms of meeting functional specifications, and actually being what the customer needs.

I consider Specification By Example as a very focused approach to discovery. Here collaboration between those involved in the development process (product and engineering) is extremely important in order to discover the requirements needed to meet the customer’s need. It helps to exemplify them in a way that all of these people can understand what is being said (so it uses a domain language) and using engineering support (with automated testing and Continuous Delivery and Integration practices). This ensures the delivery of quality software.

Applying SBE in Practice

For each of the practices of the Specification By Example, I have examples of how we can apply it:

#1 Derive the scope from the objectives

Find out what the customer really needs. Then, give the development team the freedom to suggest the best way to do it. This is simpler to implement, faster and even cheaper.

Our focus, then, should be to understand what problem the customer wants to solve, or the goal he wants to achieve. And from there, the whole team collaborates to arrive at the solution design.

#2 Business Objectives -> Derive the Scope -> Create the Stories

For example, if we aim to solve a problem of customers who want to pay an account quickly, we can derive the following scope:

The bank account holder will pay a ticket through the barcode. He can type the barcode, which is slow, or can use the camera of the mobile phone to capture the image of the barcode. The application must interpret this image and fill in the ticket data to be paid without the need for user intervention. The user can review the data and confirm the payment.

The faster option should be offered as the first option to the user, while the slower option should be offered as an option.

Story 1: I, as a checking account holder, want to use my cell phone camera and identify a barcode, so that my ticket data is filled up quickly.

Story 2: I, as a current account holder, want to manually fill in the barcode number of a ticket, to pay it even if the bar code is unreadable.

Story 3: I, as a current account holder, want to review the completed data of a ticket informed via barcode, to confirm them and to effect payment.

#3 Specify Collaboratively

Here is where some people who try to implement Specification By Example end up getting lost. Often only one person within the team (usually the one who plays the role of QA) writes the specs, and the rest of the team just looks. If this is what happens, we totally lose the sense of collaboration.

The idea is that — all involved should participate, and this can be through workshops, or writing and review. The important thing is that the specifications are written by the team including the client. Language has to be part of the business domain and be understood by everyone.

#4 Illustrate Using Examples

Human beings better understand practical examples. This helps avoid ambiguity and communicates accurately. It is important to use concrete business examples and think about expected outputs.

In addition, the examples help to generate discussion and eliminate doubts. Here’s one —

Functionality: Free Delivery
1: Offered to VIP customers, once they buy a certain number of books;
2: It is not offered for ordinary customers or for VIP clients who buy anything other than books;
3: The minimum number of books for free delivery is 5

Figure 5

What do good examples look like?

  • Must be precise: it helps to avoid ambiguity, it should clearly inform the context and how the system should behave in each case
  • Must be complete: it is important to combine different inputs and think about the expected results, as well as to experiment with data
  • Must be realistic: always think of real user situations (caution with sensitive information)
  • Easy to understand: everyone involved should read the examples and understand without much difficulty. You can use abstractions — for example, use “car” as an example instead of describing exhaustively the characteristics of the car: four-wheel car, two-door etc., unless these characteristics influence the final result.

You can also use examples for non-functional requirements.

  • Performance: “The system needs to import X records into Y minutes by consuming CPUs.”
  • UI: Screen prototypes are good examples as well.

#5 Automate Specifications

Today we have several tools available that allow you to automate specifications. The best known is Cucumber. Here, all the scenarios you wrote using the Gherkin format are interpreted and implemented from any other automation framework. Some examples are — Appium or Selenium.

With each new feature or change to an existing one, you can add or modify its specifications and ensure that the regressive continues to work.

#6 Validate Frequently

It’s no good getting several automated tests that do not run continuously. To ensure that your specification is alive once you have implemented them, you need to use mechanisms such as Continuous Integration for these tests.

It helps to remember that they are tests built from the specifications — they ensure that what has been developed meets the business needs. They need to be run frequently with each new change. This ensures that if there is a problem, the tests give you quick feedback and indicate what needs to be fixed. Here you can use tools like JenkinsBambooCircleCI, among others.


The purpose of this text was to show that developing from specifications is not new. We have been talking about this for a long time in software engineering.

But the most important point of the specification as we know it today is the constant discovery of requirements related to the product. It’s also about the use of that knowledge to generate a business domain language, which will be used in the writing of live and executable documentation.

Image Credits:

Figure 2: Extreme Programming Explained: Embrace Change — Kent Beck

Figure 3: Results from Workshop in Agile 2010

Figure 4:

Figure 5:

References if any

Author bio:

Author's photo



QA Manager at Concrete (an Accenture Company), helping teams to define quality strategies to create incredible digital products. Today I lead 40 QAs in the Rio de Janeiro, São Paulo, Recife, Belo Horizonte (Brazil) and Colombia offices.

I am passionate about agility. I believe in empiricism, continuous improvement and diversity.

Co-founder of the Carioca Testing Group that is now part of Ministry Of Testing (

Articles of QA published in and

What is your upcoming strategy for Test Automation?

Implementing test automation is critical to build and deploy robust applications in quick time. However, the business should have a strategy in place to make test automation functional, cost effective and easy to maintain.

In an Agile and DevOps environment, the focus is on the quick detection of glitches in an application before its eventual deployment. This is due to the fact that a quality validated application stands a good chance in securing customer approval than the one devoid of it. It is better to have a robust and secure application with less number of features than having one with umpteen features but with glitches galore. To ensure this, the software must undergo a series of tests namely, unit, functionality, performance, usability, security, and regression test among others. However, should these battery of tests be conducted manually, the eventual outcome may not be to the satisfaction of everyone due to the inherent issues with manual testing.

Test automation provides the best way to validate an application in terms of quality and ensure its quick deployment. The automated tools can run scripts that test repetitive tasks to shorten the SDLC. This way, QA automation testing can achieve speed and reliability – important deliverables of the Agile-DevOps setup. Even though the test automation framework adopted by businesses has reduced the number of test cases, there are occasions where businesses have failed to implement their automated testing strategy like in regression testing. The reasons for the failure in implementing test automation can range from the difficulty in writing test scripts with a longer lifespan to maintaining them. Let us chart a robust automated testing strategy to achieve quality deliverables and ROI.

Strategy to implement test automation solutions

#1 Automating the right test cases: Since it is difficult to automate each and every component of the application, choose the ones that give the best test outcomes. Thus, it is better to run keyword based tests, for they are easy and quick to write. Moreover, they are best understood by the software as well. The challenge is to maintain these tests and keep them updated over time. The test cases to be automated should be of types such as large datasets, high risk, repetitive, and those covering a number of browsers, platforms and networks. The automation of such choicest test cases would help in identifying the glitches better, provide relevant dashboard reports for analysis and improving the processes, and free the test automation experts to explore other features thoroughly.

#2 Cover the development sprint: The Agile-DevOps ecosystem is about building a development sprint where writing and testing the codes are done concurrently. The goal of building a development sprint is to accomplish both the tasks of development and QA together and in the shortest possible time. This is not always feasible to achieve except with proper planning. For example, when new functionality is added, it should be subjected to unit testing. In other words, the functionality should be broken down into smaller units and tested based on keywords that correspond to your business requirements. This way, the business can derive the maximum benefit out of unit testing. The QA team can write automated scripts for various units while being a part of the sprint.

The test automation solutions can help the QA team to keep pace with the development team by quickly performing various unit tests. When the development and QA teams work in the same sprint, it significantly improves the quality of the code by a large measure.

#3 Making the automated scripts to last long: Among the many challenges of writing robust test cases is the fact that they can become irrelevant after a while. This can be pre-empted by writing smaller test cases in a sequence of steps to test a functionality. The advantage of writing smaller individual test cases is that they need not be rewritten once a new functionality is introduced. On the other hand, instead of rewriting the whole script from scratch, it will be easier to focus on a few test cases pertaining to the changes.

#4 Independent of UI: The flexibility of an automated test script can be ensured if the same is UI independent. The test cases should be written based on keywords that are duly supported by the backend. At the same time, should the test cases use the UI elements, it can spell trouble as the latter can undergo changes.  


The only way QA automation testing can succeed is by properly designing the QA build and devising a well thought out strategy. The focus should be to write test cases that remain relevant across processes and changes.

Author Bio

Komal Lopez

Komal Lopez works with Cigniti Technologies and is instrumental in helping enterprises make better decisions related to Quality Assurance products, tools, and services by leveraging research and content. She specializes in writing about technology trends, testing trends and has been in the Marketing and Communications industry for over a decade.