Online Documentation and SEO. Part 2 - Crawler-Friendly Pages

Posted by
Anastasia in SEO Lifehacks on 8/9/20132 min read

Search Engines

This is a continuation of a blog post series on the SEO aspect of technical writing. You can find the first post here: Part 1 – Human-Readable URLs.

Web crawlers used by search engines request web pages by URL. At that, most of the search spiders do not run any scripts when loading pages...

It's like when you disable scripts in your web browser, and browse pages. If some pages heavily rely on scripts to render their contents, the web crawlers may get incomplete or incorrect page contents. This makes your SEO efforts of very little value to the end result.

This is what the architecture of a standard Web crawler looks like:

Architecture of a standard Web crawler

This does not mean that you can't use scripts in your online documentation. Scripts may give some nice functions and visual effects to the help topic contents. At that, it certainly makes sense to have a script-free version of those pages for the search engines to properly index the contents. In ClickHelp, we have put the needed attention to these specifics, and made sure the web crawlers get the topic contents properly. So, when using ClickHelp to create online documentation, you don't need to bother about web crawlers not running scripts.

Another important note – don’t forget about metadata with the "robots" name. The value of this meta tag can significantly change the web crawlers behavior. For example, <meta name="googlebot" content="noindex" /> prevents the page from being indexed by Google crawlers (read more at

One more important note is related to the robots.txt file. But we will write about it in one of the next blog posts. Stay tuned!

Happy Technical Writing!
ClickHelp Team


Give it a Try!

Sign up for a 14-day free trial!
Start Free Trial
Post Categories
Want to become a better professional?
Get monthly digest on technical writing, UX and web design, overviews of useful free resources and much more.

Mind if we email you once a month?

Professionals never stop learning. Join thousands of experts subscribed to our blog to get tips, ideas and stories from us once per month.
Learn more on ClickHelp
By using this site, you agree to our Privacy Policy and Terms of Use.

Learn more