I came across this service called BeTheBot. This is basically for you, a human, to look at websites the way googlebot or Yahoo! slurp does.

Why would I want that?

Have you ever searched for something on Google or Yahoo! search? Obviously, you have, many a time. Now, in the results that appear, your search keyword may be present in the site descriptions of the sites that appear in the search result pages.

But after visiting one such website link, have you ever wondered why the keyword is not present in the whole webpage? The very keyword, which had earlier appeared in the preview of the website in the search result is completely missing from the webpage you have visited.

This is not because of a mistake on the search engine’s part or yours. It is simply a tactic used by webmasters to bring visitors to their site, or sign them up for something etc.

Whenever google’s spider accesses a site, it identifies itself as googlebot. Now webmasters may program their sites to show specific content for googlebot or Yahoo! slurp which may not be the content you, a human, are seeing.

So, in order to know what exactly is being hidden from you or shown to search engine bots, this service comes in handy. You can simply enter the site’s URL into the search box and find out how the site looks from googlebot or Yahoo! slurp’s point of view.

Keep up to date

Want to know the latest? Subscribe to our RSS or get Regular Email Updates.

Share the Post

Related Posts