How To Measure The Effectiveness Of The Documentation

Technical writing and communication involves communicating complex information to specific audiences with clarity and accuracy. The key word here is communication, which is a term for two-way interactions. But in case of technical writing, the addressee side is shrouded in mystery. Technical writers often have a very vague idea of the end-users, their level of competence and their needs, because they lack knowledge and feedback from their audience. But a clear understanding of the target audience knowledge is essential for adjusting the writing for the needs of the particular audience and communicating the message in the proper manner.

In this article, we’ll focus on ways to gather metrics that may further be used for improving the quality of the documentation and level of user satisfaction.

Use Web Analytics

Web analytics is a powerful instrument that allows measuring the user experience. You may use Google Analytics, Adobe Analytics, or some other web analytics service, to collect the most basic but still useful information about your target audience. It’ll provide you with the following metrics:

• Time spent on a particular page.
• The number of viewed pages per visit.
• The number of page visits, and more.

For example, ClickHelp technical writing tool has built-in statistics feature that allows monitoring the viewing statistics of a particular topic or the whole project.

Using metrics of web analytics you can make certain conclusions on how to adjust the documentation and make it suit your audience’s need. For example, you may learn which topics are the most viewed and how much time users spend on these pages. So in case a topic is popular but users spend too much time there, enough to read it several times, this may mean that this topic needs to be improved: you may want to rewrite it using plain language, add screenshots and gifs, use less jargon, make it more clear and concise and so on.

Use Surveys and Interviews

Another way to learn more about the shortcomings of the user documentation is to directly ask the end-users if they had a positive experience with a particular piece of documentation. In order to do that, you need a built-in feedback mechanism on every help topic page of the documentation.

We advise you to use a simple response mechanism like thumb-up and thumb-down buttons. For example, ClickHelp contains ratings report feature that gathers statistics according to the data of the following widget.

More detailed insights can be gathered through fill-in forms that pop up when users punch the thumb-down button. This way you may learn more about whether a particular topic met users’ needs. You may also use a pop-up with a short interview that appears when users are attempting to leave the page.

Do Usability Tests

Unfortunately, only a small percentage of users usually provide feedback and even less leave their comments in case of negative experience. It means that conclusions based on the surveys are not always representative. So option #3 is to do a usability testing.

According to Jakob Nielsen, you only need 5 random participants to conduct a usability study and receive reliable results. In order to gather even more representative statistics, you need to find five participants representing different types of the documentation users: developers, new users, novice users, expert users and so on depending on the type of the documented product.

Instead of gathering lots of participants for one study, try doing several usability tests with a small number of participants. A variety of tests conducted with a small group of people proves much more useful than doing one study with a large number of people. This way you’re going to gather more data and feedback and your company is going to achieve more for the same amount of money invested.

Analyse Support Tickets and User Forums

Another way to measure whether the documentation is effective is to research support tickets and forum threads. Unfortunately, neither of these sources is informative and reliable enough to make it the primary source of user feedback.

Users contacting support usually don’t describe their experience with the documentation: whether they consulted the docs, what they were looking for and whether they’ve found the solution. But when they do, it allows you to find out if a particular topic contains all the necessary information about the subject matter or it should be expanded. If the topic contains the necessary solution, but the user is unable to find it, then you need to consider improving the particular topic or the whole structure of the documentation.

When analyzing the information from the support tickets and user forums, you need to keep in mind that these sources of user feedback provide a one-sided picture. When users are satisfied with their experience with the docs they do not contact the support team or share their experience on the forums.

Conclusion

Unlike other types of writing, technical writing communicates with different types of audience that depends on the documented product, its functionality, the peculiarities of the niche it belongs to. Web metrics, surveys, interviews, usability tests, support tickets, and user forums together provide a set of metrics, that allows measuring the effectiveness of technical documentation and its particular topics and sections. Quantitative and qualitative analysis of the mentioned sources allows you to find opportunities to improve the quality of the docs and the level of the user satisfaction.

Good Luck with your technical writing!
ClickHelp Team
Author, host and deliver documentation across platforms and devices