Introduction

Recently I’ve been working on more greenfield projects and have been setting my own scope on what to test and quality looks like. One of the areas of quality I’ve been exploring are the non functional areas (something I’ve not previously really gotten into) and have found some easy to get started with tools.

It’s easy to dismiss non functional testing as something “not in scope” especially early in the development of a product. But having to change a fully developed product because it’s not performant or secure is way harder than incrementally making changes early, meaning it’s something that’s really useful to provide insights on as soon as we can. The whole not in scope thing also means that we need to be pragmatic in what we share early, so that we can bring our teams along with us and help be a trusted advisor (we don’t want to be seen as holding the team to account for things nobody cares about for the sake of a “gold standard of quality”).

Here are the tools that I’ve been using and have found really simple to get started with.

Performance – K6

Website: https://k6.io/

K6 is a load testing tool that can schedule making multiple calls to an API endpoint within a set amount of time.

• Load: We can fire high traffic at an endpoint to see that these are successful and response time doesn’t get worse.
• Stress: We can fire more and more traffic at an endpoint to see when the service crashes and fails.
• Endurance: We can set the tool to make requests over a longer timeframe to see that this is successful and response time doesn’t get worse.

K6 is a stand alone tool that can be installed and run locally or in a Docker container and is run via some simple command line requests. You can also export your manual API requests from Postman meaning that you don’t have to be able to write scripts from scratch. Like with Postman requests, K6 can be pointed to any API via the http address meaning you can test against local, test and production environments.

If you can make a request in Postman then you’ll easily be able to get using K6 to start performance testing.

Security – OWASP ZAP

Website: https://owasp.org/www-project-zap/

OWASP ZAP (Zed Attack Proxy) is a vulnerability scanning tool that uses known vulnerabilities and pre-canned attacks to identify potential vulnerabilities for your product.

• Passive Scanning: It can monitor traffic whilst you manually use a site to build a map of the site and identify security issues in real time.
• Web Crawling: Using Spiders it can crawl through the website to identify API calls, identify web traffic and map out the product.
• Active Attacks: The tool can use pre canned attacks based on the identified site map to see what issues exist.

OWASP ZAP is a stand alone application that can be run locally or in Docker and can be pointed to any site (local or in an environment) via web traffic sniffing. Because it uses pre-canned security attacks and a repository of known vulnerabilities you can get started easily with little pre-existing knowledge. This tool is open source and under development so it’s also kept up to date with new vulnerabilities as they are uncovered.

This tool, as well as highlighting issues, provides a lot of information about the issue including cause and how to fix it which is all exportable. As such it’s great for a new starter getting into security testing because it explains things as you go.

Accessibility – Axe

Website: https://www.deque.com/axe/

Axe is a browser add on that uses WCAG 2.1 standards to review the UI code and highlight potential issues surrounding accessibility. This mostly revolves around those areas that can be codified like labelling, colour and fonts.

You can install axe to Chrome browsers and run the tool in browser across a whole page (or part of a page) to highlight issues. Like with OWASP ZAP this uses pre-canned tests and provides information about what issues has been found so you can start using this tool with little to no knowledge and learn as you go.

Usability – Flesch-Kincaid Calculator

Website: https://goodcalculators.com/flesch-kincaid-calculator/

The Flesch-Kincaid Calculator is a website that you can paste text from your product to get a readability score. This score can be mapped back to a reading level, education level or book difficulty to get an indication of who could understand what you’re saying. The higher the score, the more advanced the reader would have to be to understand your sentences.

This calculator provides the score as well as a series of meaningful insights in its results, as such it’s perfect for someone new to testing readability.

Sharing Low Hanging Fruit

So you’ve done some exploration and used the tools, then what? We share the information back to the team.

• Socialise your method: Talk about what you did to find this information, the tool you used, how long that you took and what you were looking to find. You can also talk generally about this area of non-functional testing and why it’s important to users.
• Talk through the issues: Just like in an exploratory testing debrief. Talk about what you found, what the issues could be and talk about any technical details you have.
• Say it’s not a benchmark or a fail: Stress that we’re being pragmatic and just sharing for interest; that what we’ve found are insights that we can make decisions from. Mention that we’re not holding the product to account to a specific benchmark and that any improvements (if there are any) we want to make will help to improve the product.
• Relate to the end user: Talk about the specifics around how this relates to your project and what this means for the end user. Could a performance issue knock out a demo? Could accessibility issues make the product less sellable? Would security insights help provide confidence to your organisation?
• Say these are only low hanging fruit: Stress that when we use a tool we’re just finding the easy issues and that there could be more that we can uncover with more testing. Use this testing to enter a discussion to show that some issues exist and suggest that more testing in this area could find more issues.
• Be prepared for them to be ignored: Sometimes these issues will just be out of scope or not of interest right now. Don’t be disheartened, you can always try again another day. Remember, we’ve now provided information and made a decision based on explicit information rather than nothing so that’s a win!

When I’ve shared information back to product owners about non-functional requirements I’ve sometimes won people over leading to non-functional fixes. Sometimes I’ve been thanked and told that that information isn’t needed now but I’ve gained the knowledge of how to start testing these areas.

Let’s use tools to find the low hanging fruit non-functional issues as part of setting our own scope. Then we can share these insights with our projects.