How to Test the Usability of Technical Documents

Technical documentation is one of the main components of the software product in any respectable company. But not all organizations devote enough time to develop effective documentation. Oftentimes, there are situations when a software product is smart and ingenious, but the documentation is feeble and helpless.

Since a technical document serves many purposes and can affect the quality of a product, technical documentation must be accurate as it might lead to gaps between stakeholders.

Producing reliable technical documentation is one of the main headachy processes for software development teams.

As a result, development teams include a technical writer who ensures that the technical document is correct with the help of the documentation testing process.

What Is Documentation Testing

Documentation testing is the process for testing the documented items developed throughout the whole process of product testing. It is a non-functional type of software testing that allows you to make sure your documentation is in a constant state. In addition, such a testing process guarantees your users a better experience and decreases tension around common issues as a writer.

In case a document does not work, there may be some problems, such as:

• reader’s confusion;
• people missing out on a benefit because the document is too elusive;
• someone may get hurt, and there could be lawsuits or penalties because a message about the hazard wasn’t clear enough or absent.

Documentation testing reveals problems in a document and gives you insights into how to fix them before presenting the document to a broader audience.

Documentation Testing Checklist

In general, documentation is created to find a solution for any problematic situation competently and without panic. This is a fundamental principle when thinking over the content and structure of any technical document. Here is the list of qualities to pay attention to when testing the documentation:

• Ease of understanding. If a product is designed for ordinary users, then the documentation for it should describe the user’s actions in simple, understandable terms. Server and management software manuals are often geared to system administrators. Use the terminology that is most appropriate for the objects under test. If you use a specific term, then it is worth describing it separately. In case of a double interpretation of the term, clarify which one is used.
• Grammar. Sure enough, the content must be composed correctly. To check the syntax and punctuation, you need to read the text, understand its meaning, know the meaning of each of the terms used. You can check the spelling with special tools like Grammarly, LanguageTool, WebSpellChecker, or the like.
• Scenarios efficiency. One of the essential qualities that documentation should have. The scenarios should be described accurately; their execution should lead to achieving the goals for which the product was created. If there are alternative scenarios, they should also be mentioned.
• Completeness. Describe every function element, whether an interface element such as a button, checkbox, etc., or the command. Any mandatory and important user actions shouldn’t be mentioned in passing. Pay attention to describing all the necessary actions. Such documentation is not a loan agreement where all additional conditions are described in small print. For example, it is worth explicitly registering the information that the application will not run if the configuration file contains a comma instead of a period.
• Correctness. If the user is in a hurry, they need to find the answer to the question or follow the instructions without additional references or wasting time.
• Structure, easy document navigation. The documentation should have a clear structure, and the user should be able to find information on the table of contents quickly. If it takes a lot of time to search for some information, the user can quit the search. Not everyone has the patience to carry out actions. All file trees, bookmarks, etc., should be visible the moment when a user opens the document. Don’t forget about the alphabetical index or search. Everything that will help a user find a solution to the problem. The text should also have a clear structure – so that at any time, readers can remember where they left off or understand which paragraph contains precisely the information they need.
• The sequence of actions. In some scenarios, the sequence of steps is important. For example, when making soup, we first pour water and then add other ingredients such as potatoes. If you first put the potatoes, and pour the water much later, then instead of soup, you will get something inedible;
• Accuracy of links. Any online document contains links. Check the links to find out if some of them are broken. With ClickHelp, you can do it with the Link Viewer feature. The ClickHelp engine checks: Cross-topics links inside topic content, resource links in style files, links in scripts files.
• Relevance. If you test the documentation for a software product with many versions, you should pay attention to the relevance of the description. It may happen that the functionality has been changed in the current version, but this wasn’t mentioned in the documentation. Or it was decided not to include the feature in the recent release on short notice, but the feature has already been described in the documentation. The relevance of the date, contact information, system requirements, license agreement, screenshots, and the like, is also worth paying attention to.

Documentation Testing Techniques

There are three ways to test the usability of documents:

• paraphrase testing;
• plus-minus testing;
• task-based testing.

Paraphrase Testing

Such testing is a great way to see if people understand the messages in a document. In a paraphrase test, each participant goes through a fragment of a document at a time. After reading each fragment, the contributors tell you, in their own words, what was said in that fragment. This testing is helpful to learn whether:

• the document’s organization makes sense to your readers;
• your paragraphs are short enough for people to get all the parts;
• your contributors understand the words you use;
• the contributors use different terms from those in the document.

Before starting the test, you should decide where the breaks between fragments will be. For example, a fragment might be a sentence, a short paragraph, a list, or one provision of a contract. Each fragment should be meaningful and small enough for your contributors to grasp.

Plus-Minus Testing

The authors of this technique are Dutch researchers Menno de Jong and Peter Jan Schellens. This is what they say about the plus-minus testing: “Participants are asked to read a document and put pluses and minuses in the margin for positive and negative reading experiences. After that, the reasons for the pluses and minuses are explored in an individual interview.”

This testing is a fine way to get people’s reactions to a document. Choose what plus and minus are in your document, subject to its purpose and what you’d like to learn.

You can use plus-minus to:

• gather opinions of your participants about what is clear and what is not clear to them;
• get people’s emotional reactions to a document;
• research some other information. For example, find out how well the document transmits a brand’s values.

When we ask for positive and negative reactions in general, we often get comments about wider topics, such as the tone of a document with comments about whether the document makes sense.

Task-Based Testing

Not every document is read from beginning to end. There are a lot of documents, such as handbooks, insurance policies, manuals, and many other types of documents that people don’t read through. They refer to them only to solve their problem, check a specific fact, or follow instructions.

With task-based testing of a document, you watch as your contributors try to use the document to find and understand the information. The reader here is a user who wants to read only what is necessary. Therefore, finding the right place is critical. The reader sees the information; then scans to read just enough to get the answer, check the fact, or follow the instructions.

Take notes on the suggestions and comments you get during the test. This will help you make your document perfect. If you’re not sure which testing to apply for your document, the purpose of the document can help you decide how to test it. If the purpose of your document is to explain something in detail – whether to read now, refer to later, or act on after reading, use paraphrase testing. Plus-minus testing fits a document that gives a general understanding of a topic or creates an emotion, for example, builds trust. In case a document provides answers or instructions – use task-based testing.

Conclusion

The availability of high-quality documentation has its advantages:

• Users do not seek to abandon the product, the load on technical support decreases;
• There are fewer questions about how the product should function from the development team;
• It is easier to sell the product.

Usability testing of documentation helps you find out how well your document works for the people who use it. When we create documents, we can become too familiar with them. The results from testing cut through that familiarity and help us make better decisions about what to change and what to keep.

Good luck with your technical writing!
ClickHelp Team
Author, host and deliver documentation across platforms and devices