“It’s okay to make
mistakes,” they’ll tell you. “As long as you learn from your mistakes.” Not the
worst advice you could get, but a wiser person would tell you that it’s far
better to learn from someone
else’s mistakes. Wouldn’t you agree?
If so, you’re in luck, because organizations all over the world
continue to make the same software testing
mistakes over and over again –
all so that you don’t have to. While there are far too many to list here, we wanted to share a
few of the more common gaffes. Here they are, in no particular order:
Mistake #1: Testing too late
How do you know if you’ve waited too long to test your apps? One
sign would be if users are publically complaining via social media about
defects they found on their own. When it comes to quality, the sooner you can
involve the test team, the better off you are going to be. As you’ll see
throughout this post, many of the great testing blunders occur for this very
reason. Teams that see testing as the “last line of defense” not only
misunderstand the role of QA, but they put themselves and their business in
dangerous territory. Brian Marick explains the advantages of testing early:
Tests designed before coding begins can improve quality. They
inform the developer of the kinds of tests that will be run, including the
special cases that will be checked. The developer can use that information
while thinking about the design, during design inspections, and in his own
developer testing. Early test design can do more than prevent coding bugs.
Mistake #2: Testing with amateurs
In an ideal world, every testing project would have two types of
experts: a testing expert and a domain expert. The reasons for having the
testing expert on hand should be obvious (to readers of this blog anyway). The
great testers understand, regardless of the product, where flaws and
vulnerabilities are likely to be found. They are, as James Bach has said, professional skeptics. Domain experts on the other hand might not
know much about testing per se, but they will immediately pick up on
inconsistencies and shortcomings in the product from a feature/functionality
perspective. Unfortunately, many companies are making the mistake – the BIG
mistake – of having neither a testing expert nor a domain expert on their team.
Don’t be one of them.
Mistake #3: Testing without a scope
“Just look for bugs and tell me what you find.” Famous last
words! While this type of “improv” testing can often yield quality results, it
should never be standard operating procedure. Unfortunately it is inside many
companies. A note to testers: If this is the direction you’re asked to go
in, here is some good advice from Jon
Bach:
There are *always* requirements, but they aren’t always written.
They are both implicit and explicit. Some are hidden, some may be
obvious. Some take time to emerge, others hit you right away. You
find them and they find you. You’ll know you found one when you sense a user
has an unmet need.
Techniques for testing without requirements:
- Ask
stupid questions
- Read
the FAQ
- Look
at competing products
- Read
out-of-date documents
- Invite
a developer to the whiteboard, then diagram your assumptions of the
architecture or some states
Testing is an active search, not just for bugs, but for
requirements. Approach your requirements-gathering the same way you
approach testing. You have to go find the bugs – they often won’t reveal
themselves to you on their own.
Mistake #4: Testing “one and done”
A lot of time and energy goes into a software launch. So it’s
only natural to relax a bit when the application is released to users. It may
seem as though all is said and done from a testing perspective, but it ain’t.
As we’ve seen time and again, many of the more serious bugs and issues crop up
well after a product launches, often times through no fault of the engineering
team. Maybe the application isn’t synching with a recently updated third party
app. Maybe a browser setting changed. The list is endless. Whatever the
circumstance, companies would do well to avoid the mistake of failing to run
regression tests.
Mistake #5: Testing in a controlled
environment
These days, users are likely to consume your application under
all sorts of various conditions; an endless combination of devices, operating
systems, browsers, carriers and locations. Why then do so many companies only
test their applications within the confines of a highly controlled test lab?
Why are they not moving a portion of their testing into the real world? There
was a time when companies could get away with 100% on-site testing – and many
did. Unfortunately, many continue this practice (old habits die hard, after
all) resulting in production bugs that could have easily been found by testers in-the-wild.
Mistake #6: Testing too fast/slow
The Agile testers can’t seem to catch their breath, while the
waterfall testers look like they’re about to die of boredom. Our last testing
mistake has to do with the pace of testing – specifically, why many organizations
can’t seem to keep it consistent. Much of the problem (again) boils down to the
misperception of QA. When testing is seen as the last step in a long process
(usually with a looming deadline) it’s natural for testers to feel rushed. When
the role of QA is properly defined and agreed upon, testing will never be
rushed and it will never slow to a crawl. Instead, it will become an integral
part of the organization, not merely a department being waited on.
No comments:
Post a Comment