The perception of performance and speed from the user’s point of view is what drives the quest of performance testing. It’s as simple as this: we know how quickly we’ll leave an app or a site if it doesn’t work. Even with this knowledge though, it seems that in many quarters performance testing is still considered a reactive process, something that’s done after a system or website is built as a way to troubleshoot problems and make things ‘work better’.  

But I’d like to suggest that performance testing, in this very traditional definition of the term, also has the capability to be enhanced and bolstered, to boost your competitive advantage in a highly mobile, 24/7, 100 per cent digital world.  

While there is certainly a case to be made for the role of traditional performance testing, it can be a stop-start process. This doesn’t always make sense in an Agile environment, where new builds and features are added almost hourly, let alone daily. Ironically, Agile is also a reason performance testing sometimes takes a back seat:  it’s not easy to test something when the goalposts are continually shifting.  

Which is why we have to stop and rethink the entire concept if we’re going to use performance testing to our advantage – to build faster websites and e-commerce engines and transactional environments that not only perform optimally, but do so while serving tens of thousands of simultaneous users. 

The best way to do this is to focus less on the systems and more on the results. How do we want our systems and websites to function? What metrics can we use to measure a satisfactory user experience, rather than just pointing out potential pitfalls? It’s a subtle shift, but a shift nonetheless, and requires not only testers but also engineers and performance experts to get us there.  

What’s more, the evolution of traditional performance testing, and the tools we need to make that jump, are already here and waiting for us.  

I’m not talking about ‘new age’ next-generation technology either. Take NeoLoad, for example, which happens to be the continuous performance testing tool we already use extensively across all our client sites in the UK and around the world. The focus here is on ‘continuous’, because that’s how we must approach performance testing if we expect it to deliver performance-driven results.  

But even continuous performance testing is not enough on its own. We need to be able to deconstruct what it is that clients want their systems to do, from their users’ perspective. Load times, response times – the way information appears on a webpage and how soon it starts to appear – these are all intrinsic parts of the human experience that can’t be measured with traditional tools.  

We may be able to simulate how stable a server will be under extreme load, but can we say how quickly the first lines of text appear on a page when the user clicks it? Users don’t care how many other users are online at the same time, only about their own individual experience. The secret is not only to enhance speed in reality, but also to enhance the perception of speed. So, part of the performance testing evolution is engineering how websites are coded, getting text to appear before graphics, getting the first line of text to appear within a second, and making sure text and graphics are continually loaded as the user interacts with the page, keeping them engaged for longer. 

This very idea is what tools like Lighthouse are made for - open-source, diagnostic tools designed for the sole purpose of auditing the quality of web pages, with scores for performance, SEO, accessibility, best practices and its level of progressive. It’s not rocket science either; for example, you can run Lighthouse in Chrome DevTools, from the command line, or as a Node module.  

You can use any of the top web performance tools against any web page, like we did for globally renowned shoemaker Jimmy Choo. Working with a web performance expert, we utilised a selection of tools to create a highly-responsive web experience for the company, worthy of its premium brand.  

Web performance is an art on its own, where we look at the code within a website and how it can be optimised. We call this our Inspired Performance Testing methodology and it demonstrates, without doubt, that taking performance testing to the next level in our highly mobile-centric digital world is where the future lies. We have the means; we just need to shift our way of thinking and start using these very important and very useful tools as part of our existing processes. 

The evolving industry of digital banking is another perfect case in point for optimal user experience. Banks like Monzo and Revolut are built on the premise that the traditional rules of banking are being turned on their head. Now, the customer – the user - is in control of their own destiny – and if they don’t like the rules of engagement, they can go elsewhere at the click of another app. Three seconds is all it takes for impatience to set in. Perhaps the site is broken, they think, as they grumpily log off. Let that happen a couple of times and your customer will be out of there – permanently.   

If the above makes you think that new-age performance testing is far more complex than the cookie cutter approach of old, that’s because it is. There’s no one-size-fits-all when it comes to digital transformation, and the same goes for performance testing. As developers and testers, we are beginning to realise that continuous performance testing at the DevOps and API level is more important than signing off a system that can withstand short bursts of extreme traffic. We have to balance the needs of the client with the wants of the user; it’s no good to us or the client if we deliver a rock-solid system to a digital bank that hemorrhages users to a competitor because the user experience is less than ideal.     

Yes, the usual performance testing suspects are still relevant. Capacity testing, load testing, volume testing, stress testing, soak testing and spike testing all fall under the performance testing umbrella, as does identifying bottlenecks and breaking points - and they’re all important. But they’re not enough. We have to ‘shift left’, to start performance testing much earlier in the development cycle, and continue long after the system is delivered, to take user experience into account.  Performance testing should be part of our daily sprints, and with the right tools correctly implemented, it becomes easier and simpler to find user paths to track and optimise UX. Ultimately going beyond performance testing becomes a client retention strategy. If your site or app isn’t responsive, fast, usable and stable, your clients will switch to your competitors, and once they do, they will be gone forever.  

-- ENDS --

About Inspired Testing
Inspired Testing’s disruptive global testing model offers a revitalised alternative to outdated traditional offshore models. With a scalable pool of 260 SQA professionals in the UK and South Africa, the company’s strength lies in knowing how to structure, execute and automate testing. The company’s ISO 27001 certification and its alignment to the UK/EU GDPR (General Data Protection Regulation) ensures full protection of critical personal data. Inspired Testing uses a unique combination of experience, technique and blended onshore offshore delivery capabilities to provide expert software testing across most platforms, devices and environments. www.inspiredtesting.com

Inspired Testing is a wholly owned subsidiary of Dynamic Technologies, a software and technology group with 1 100+ staff and 14 group companies across the UK and South Africa, providing a diverse range of technology solutions, digital services and related core competencies. Dynamic Technologies’ group companies comprise Blue Pencil Consulting; Blue Pencil Creative; CloudSmiths; DotModus; DVT; DVT Academy: Dynamic DNA; Dynamic Talent; Emerald Consulting; EventSmiths; IndigoCube; Inspired Testing; SparkSmiths; and Swarm.  www.dynamic-tech.com

Testing should not be viewed as a ‘quick fix’, but an integral part of the DevOps process, says Inspired Testing.

The proof that considerable advances have been made in mobile testing is evident when one compares the user experience a decade ago to present time. From frustratingly slow and difficult to navigate, mobile interfaces are now coming into their own, providing millions of seamless and happy encounters.

Does this mean mobile testing has finally found its groove?

Well, yes and no. Yes, because the small screen platform is finally being understood as a discrete medium that requires specific testing skills and expertise. Mobile testing is now an area of specialisation. And no, because the diversity of content, continuous updates and the plethora of devices on the market at any given time – with their associated operating and application software – create a constant challenge to mobile testers, and their clients, to provide a fast, error-free and fluid user experience.

With what do we test it, dear Liza?

“One of the biggest tribulations for companies wishing to test their software on mobile devices is access. The number of makes and models available can make it a cumbersome, expensive and ongoing process,” says Jacques Fouché, CEO of software testing firm Inspired Testing.

Cloud-based public device farms provide a range of mobile devices and offer a useful rental service, but they have some drawbacks. Firstly, it can get very expensive, especially for larger corporates that have a wide range of services they need to cover. Larger organisations often require 24/7 access to the devices for a prolonged period of time. “Public device farms are more suitable for smaller companies that have a couple of apps they want to test, but this also depends on their risk profile,” explains Fouché.

Another option is cloud-based private device farms, where you strategically procure your own devices and either house them in your own data centre within the business, or with an outsourced provider. Either way, you have 100% access to the devices at any time, without having to pay a rental fee. ‘Strategically procure’ means you buy certain models within a brand, as testing on one device will give you similar screen real estate on four or five other devices. “This option is good for high-volume device users requiring stringent levels of security, such as companies in the financial technology space,” says Fouché. Companies that strive to achieve a high frequency of continuous integration and deployment would also do well to consider a private device farm, as this will give them unrestricted access to the devices.

Houston, we have lift-off… or not

Once the challenge of how to access devices for testing purposes has been solved, the process of testing begins. This is where companies can really come unstuck, says Lloyd Dignan, Director: New Business, Inspired Testing. “Whether it’s a new release or an update, a complete retest is the only solution. There is a perception that once you have released your app or launched your Web site, any subsequent updates will not be as critical. Many big players have discovered the hard way that this is not the case. No company is immune to crashes and errors. To avoid this, testing should be viewed not as a ‘quick fix’, but an integral part of the DevOps process,” he explains.

Understanding which devices your market uses is also critical to the testing process. Tapping into the information streams available, such as market research, Google Analytics and your marketing engine will provide insights into the make and model of devices accessing your site or app, from which you can extrapolate technical data. Also bear in mind that device usage differs significantly from country to country. For example, in the UK, the majority of 18-35-year-olds have the latest iPhone. In Romania, the Samsung Galaxy J5 is the most popular phone, while Chinese brands Tecno and Infinix are big brands in Nigeria. “Keeping current with devices and operating system updates is one of the main challenges facing companies,” says Dignan.

One thing mobile users across markets, countries and continents do have in common is intolerance to poor app and Web site performance. “There is absolutely no loyalty,” says Dignan. “If an app crashes or takes too long to load (between four and eight seconds on average), the user will move on – and even delete the app before they go.” Because there is such fierce competition, there is significant pressure on companies to go live as soon as possible. “The opportunity for mobile is massive, but organisations have to make sure their systems and applications are fully tested – and the errors fixed – before they release. Errors become compounded with each new update and it gets harder to fix each time,” explains Dignan.

A changing landscape

There are clear statistics that show a massive transition from user access via desktop and laptop to mobile devices (including tablet), while the consumption of data has moved significantly from laptop to mobile. Mobile devices are not only used to access apps; they are becoming the platform of choice to access Web sites. This makes Web site testing as important as app testing, as an estimated 63% of users worldwide now use their mobile devices to access Web sites. There is much more likelihood of someone completing their cart on a phone than on a laptop or desktop. In addition, says Dignan, the emergence of responsive Web sites – Web sites that shrink down and respond to the size of the screen – made mobile browsing much easier, while many companies are now releasing apps to augment their Web sites, to provide an interactive user service. Progressive Web applications that look and load like ‘normal’ Web pages, but offer the functionality of an app, are also becoming standard practice.

These shifts are creating massive opportunities in the mobile space, and consequently, a significant explosion in mobile testing, says Fouché, particularly in industries such as fintech, logistics and retail. Automated mobile testing has become an area of specialisation in itself, and mobile testing is no longer a sub-set of testing, but a category in its own right.

-- END --

About Inspired Testing

Inspired Testing’s disruptive onshore offshore model offers a revitalised alternative to outdated offshore models. With a scalable pool of 250+ expert SQA professionals in the UK and South Africa, the company’s strength lies in knowing how to structure, execute and automate testing. Importantly, our ISO 27001 certification and its alignment to the UK/EU GDPR (General Data Protection Regulation) ensures full protection of critical personal data. Inspired Testing uses a unique combination of experience, technique and blended onshore offshore delivery capabilities to provide expert software testing across most platforms, devices and environments. www.inspiredtesting.com

Inspired Testing is a wholly owned subsidiary of Dynamic Technologies, a software and technology group with 1 000+ staff and thirteen group companies across the UK and South Africa providing a diverse range of technology solutions, digital services and related core competencies. Our group companies comprise DVT (which includes the DVT Academy), Inspired Testing, DotModus, Cloudsmiths, IndigoCube, Blue Pencil Consulting, Dynamic DNA, Blue Pencil Creative, Emerald Consulting, Dynamic Talent, EventSmiths and Swarm. www.dynamic-tech.com