I come across this question quite often. Most of those who ask this question are after the same thing — quick automation with no development and/or maintenance overhead. Don’t we all crave an effortless piece of software with no bugs and maintenance? Yeh well, eventually we all realize that this is a dream we don’t really want to pursue as it will probably result in a dysfunctional piece of software.

So what are the perceived benefits of recording automated tests rather than developing them?

1. No need for developers – automated test development could be done by anyone and doesn’t require development skills
2. No false-negative results – recording the test holds less room for errors, thus providing a lesser chance for failures that are not application bugs (automation issues).
3. No maintenance is needed – when recording automated tests, the code is generated automatically and hardly requires maintenance
4. No complicated object-model and constant adjustments – the code is automatically generated when recording automated tests, thus no need for an object-model or hardly any code adjustments.
5. Easy to use and quick to develop – recording an automated test is super quick, you just hit the record button, do whatever you want to be included in your test, and then hit the stop button. A piece of cake.

Real benefit or only a perceived one?

Let’s examine these perceived benefits and see if there are any real and actual benefits there.

1 – No need for developers?

Using unskilled personnel for developing automated tests is always a bad decision. The point is simple, you need a software engineer for developing code. Thus, by doing so, we are actively using non-professional employees for an important task that might affect the quality of our product.

For example, let’s say we’re using .NET and decided to use Microsoft’s Coded-UI recording capabilities (I must point out that they do a very good job detecting elements on screen). The basis for test recording is to “read” user actions and translate them into re-runnable code. In order to do so, Microsoft Coded-UI generates the code inside a map file. one file (default) with all generated code from all recordings. Now think about 100 recorded tests… The result is hundreds if not thousands of generated code lines (!)
What happens if something changes in your product?
How will those tests be adjusted?
Who will be able to go through all the generated code and change it?
This approach is bound to fail as it is 100% unmaintainable and is even more dangerous when assigning such a task to an employee with an incompatible skill set.

Well-written automation includes a solid object model which is constantly updated and evolves alongside the product it is intended to test. Moreover, such automation must comply with common code development conventions and patterns to make it as generic and maintainable as possible.

2 – No False-Negative Results?

The common claim will be that recording a test holds less room for errors and thus is less likely to have failures that are not application bugs (automation issues). This is completely wrong.

Every piece of software has bugs, and most of the time, automation test issues occur as a result of changes in the Application Under Test (AUT), which actually requires the code to be adjusted rather than fixed…

False-negative results are a natural part of automated tests since the AUT is constantly evolving and changing, which requires adjustments to the automatic tests, and the automatic tests are a piece of software and as such, they might have bugs.

The real question here is, what is the best way to handle those false-negative results.

Read more here

3 – No maintenance needed?

This is a mistake at best. No piece of software requires zero maintenance and we’re discussing an automatically-generated piece of software… Maintenance will always be required and should be treated as such. Don’t forget that this piece of software depends on another piece of software (your AUT) which is constantly changing.

4 – No complicated object-model and constant adjustments?

A complicated object-model is always better than not having an object-model at all. However, as with every piece of software, the object-model for your automated tests should be carefully designed and should be constantly reviewed and adjusted according to the requirements and needs. And regarding the adjustments, they will always be there, and they should as the AUT is constantly evolving.

Read more here

5 – Easy to use and quick to develop?

Finally, something we can consider as true (or real). Recording tests is amazingly easy to use and quick to develop. No doubt about it. However, when considering all the downsides, is it really easy to use and quick to develop? I don’t think so…

So What Should It Be Used For?

As you may have figured out by now, I think that the recording strategy is not only inefficient but also dangerous. However, it might be quite useful when used for the following:

1. Iterative simple test (that doesn’t change so often) to make a manual QA’s daily work easier. This is a great solution for recurring testing although the code still has to be maintained and adjusted along the way (which requires some coding knowledge).
2. When working on a test that involves a complicated business flow in the application, the recording might be useful for understanding how controls/elements should be interacted with.

Having said all that, there are many companies that are actively working on smart solutions for code-free test development utilizing Machine Learning and Artificial Intelligence. I’ll keep you posted… 🙂