Test execution

I still really enjoy breaking stuff 🙂

I’m a hands on test professional who still enjoys being a part of the test execution process.  Hands on testing allows me to continue learning about the technology I work with, the processes and techniques that need to be updated or researched, the tools that are constantly evolving, and probably most importantly, how software development and delivery is constantly changing with every project and iteration.

I’m a technical tester; I need to know how the system I’m testing functions before I can start proving that it’s been implemented correctly  If I know how the system works, I can precisely target my testing to ensure that it’s effective.  If I understand how technology is being implemented into your project, I can build realistic estimates, I can advise where your test analysts may want to spend more or less of their time, and I can identify the areas that the test team may need to research and upscale.  Over the many years I’ve been closely involved with QA and software development I’ve found that being a technical tester has uncovered some potentially costly and nasty scenarios.  I’ve learned that developers usually have more confidence and respect in a tester who has a solid understanding and appreciation of the system under test (and more importantly how difficult the work of a developer usually is), and I’ve also identified that the most successful testing I’ve executed can be attributed to my technical knowledge of the systems I’m testing.

One area that has benefited from my technical knowledge of the systems under test, and how to test them, is where testing is a cross-discipline function; aside from system developers, I also work closely with Project Management, UX/UI teams, Account Management teams, Delivery Managers, and creative teams; these groups of people often do not actually need to know how something works until they suddenly need to, and I’ve found that a technical testing approach actually puts the test analyst in a position where they can explain project technology to suit the audience; not necessarily a case of ‘dumbing it down’, more a case of removing the geek’s hat.  Crucially, this also works very well when in a client facing role where members of the client project team may not be technically minded (nor do they actually want to be a lot of the time 🙂 )

Technical areas I’ve enjoyed working with

One of the benefits of working with multiple clients and on countless projects is that I’ve been exposed to so much technology over the years.

Back end systems -the stuff your clients won’t see:

  • Windows/Linux/Unix/Mac/iOS/Android Operating systems.  I’ve always gotten to grips with deeper functions of the OS. Knowing how they work and how to navigate and configure them has allowed me to understand what’s happening at lower level
  • Database:  SQL Server, MySQL, Oracle. I really enjoy database work, whether it’s simple verification checks, ETL projects, or full data migration.  After many years testing database driven projects I have found that if you know how the system database part works, you can test the system far more effectively. This is especially appropriate where a front-end function can write to a database function; I’ve uncovered some lovely critical defects by applying this knowledge; Denial of Service attack by your own software? I’ve seen it and found it 🙂
  • Content Management Systems:  “I really enjoy testing a CMS implementation”.  Not something that I ever expected to hear myself saying, and me from 20 years ago would look on somewhat puzzled.  I’ve worked with bespoke CMS, Off-the-shelf and Enterprise CMS including SiteCore, EpiServer, WordPress and Umbraco.  Having a deep level knowledge of how a CMS works has meant that testing their implementation, and especially the heavy customisation of a CMS that usually has to be achieved, results in effective CMS implementation testing.  Being able to connect the dots between a CMS, a database, web and application servers, complex user workflows and publishing routines allows me to formulate a testing plan and execute my tests with full coverage.
  • Web servers, application servers and their environment: One of my early QA roles was for a company that had a huge amount of infrastructure driving their public facing and internal systems.  The volumes of traffic across the public site required 16 web servers, 2 application servers, 2 database servers all served via load balancers.  The release management process was heavily controlled across six different environments from development through to production.  Having to understand and navigate these, how they communicated with each other, how every critical piece of the infrastructure puzzle slotted in was something I had to learn, and in doing so I gained an appreciation of the complexity that some systems need to have.  Being aware of all this back end tech meant that I could adapt some of my testing to ensure that testing hadn’t overlooked or omitted anything.  It also meant that I gained the skill of being able to quickly map and understand system architecture, something that I still use today for test planning and execution, as well as release management.
  • API’s: More and more websites and apps rely on API calls to manage and present data; knowing how the system under test works with API calls means I can produce effective tests.  A previous client site had a mixture of API calls directly via HTTP requests/responses and embedded within JavaScript calls.  We produced a detailed map of every API call across the system, every endpoint that the system touched, and built a successful suite of tests that harnessed manual and tool based testing.
  • Failover and Disaster recovery: Your clients demand 24/7 uptime, no matter what your product is; the connected world nowadays means that there’s always someone using your product, so you need to know that if part of your infrastructure fails, you’re not dropping out and leaving your users stranded.  I’ve been part of projects where complex load balancing systems have required full testing to ensure that user sessions remain unaffected, and I’ve also been responsible for managing disaster recover testing; if your main systems suddenly become unavailable, can you switch to a back up system?  Often these are areas that are overlooked during project delivery, but they should be a part of your project deliverables.

Front end systems – the shiny, public-facing bit that can make or break your brand:

  • Functional testing: If your app or web site functions incorrectly, your user base may abandon it at the first opportunity.  It can reflect so badly against your brand, in this age of instant feedback a poorly tested piece of functionality may soon be trending on social media for all the wrong reasons.  When I’m carrying out the functional testing phase, I’ll also appraise how the features actually function; does it feel natural? does it make sense? Is there some ambiguity or nuance that doesn’t quite make sense? Is the system consistent? There are times when what’s a great design on paper doesn’t always translate as well once it’s being used.  Experience in testing and using hundreds of applications and sites over the years gives a great insight into functional usability.
  • Browser compatibility and responsive design: Modern UI frameworks are much more robust than they used to be, but we still need to check that your website or app looks and functions correctly across different platforms and devices.  Desktop, laptop, phone, tablet: 4 different device platforms that we need to consider.  Windows or Mac; Android or iOS; landscape or portrait? More variables that should be factored into the test cycle.  Windows 7, 8.1, 10; Android Nougat, Marshmallow, Lollipop? iOS 10, 9, 8? Your target audience are not all using the same version of operating systems.  I’ve been testing web front ends since the days of Netscape, I’ve seen the successful evolution of browser and devices and how we present the content to the end user; I’ve lots of experience planning and testing static and responsive sites, I’ve developed browser support profiles for clients based upon historic and real time statistics; it can be a laborious process, but ensuring that the system under test functions against the supported browsers and devices is an essential part of any test phase.  Get your front end wrong and your end users will spot it immediately.  In a worse case scenario where insufficient front end testing is carried out, you may find yourself releasing a product that doesn’t even work on an entire set of browsers.  Until browsers interpret and render HTML, JavaScript and CSS uniformly, you’re going to need to factor in front end testing.
  • Accessibility testing: Is your site or application accessible to users who may have an impairment that results in them being unable to use your product? Can a user with a visual impairment see the light coloured text contrasting against the light background?  Are those animations that you’ve spent time on likely to affect users with epilepsy? Does the user who relies on a screenreader as their only means of navigating your site have to comprehend a barrage of meaningless web-speak when they engage with your product?  Your product should be accessible to all users, which is why over the years I’ve spent time working on a suite of tests that meet Web Content Accessibility Guidelines (WCAG 2.0).  Making your site accessible isn’t just about ensuring all users can benefit from your site or application, it also means that the underlying web technology is coded to certain recognised standards. Accessibility is sometimes overlooked until the last minute; plan for its inclusion and testing as early as possible, otherwise it might prove to be an extra project cost that you were not expecting.

Requirements and specification testing: testing that we’re building the right thing

All software has a set of requirements behind it; it may be half a page of notes hastily jotted down in a meeting, or conversely you may have complex library of functional and technical requirements documents.  It’s essential that your test phase accounts for these and incorporates full traceability for proof of coverage.  The test preparation phase will identify inconsistencies and anomalies that may exist within your documents, as test analysts prepare their test scripts and documentation.  Testing isn’t just about the look and feel of the delivered product, it’s very much about confirming that your developers are delivering the product as the client has agreed.  The scale of some projects, alongside other mitigating factors can result in software delivered to testers is missing agreed features, or that functionality and designs have not been correctly implemented.

I’ve learned over the years that a test team with an encyclopaedic knowledge of requirement documents, and a team that bake in full requirement traceability to their test cases and scripts, is a team that delivers quality test execution as close to time and financial budgets as possible.

I now really enjoy breaking into stuff (legitimately!) 🙂

Over the past few years I’ve dabbled with application security, penetration testing and Information Security.  It’s a subject that never stops being interesting, there is always something to learn, there are always numerous reports that provide essential education whilst being entertaining at the same time.

What initially started as a passing interest soon became a major focus of my testing, the result of which is that I’m now an EC Council Certified Ethical Hacker.  This certification and skill set essentially means that I can offer clients my services to hack their systems and applications prior to release, or to conduct a penetration test against their systems, with the intention that a penetration test identifies security vulnerabilities before the bad guys do.  Head over to the Information Security and Penetration Testing page for further information.