Googlebot simulator?

Is there a Googlebot simulator for testing sites?

[vieth]

A while back I heard about a tool (software or web-based) that will check a website and simulate Googlebot, Google's spider/crawlwer, to report how much of the website gets crawled and why Googlebot might not be spidering/crawling all of the pages. Anyone familiar with this?

[rogerd]

I don't know about a specific Googlebot emulator, but Xenu will simulate a rather dumb spider (not as smart as Googlebot, but better to be safe when considering other SEs, too) and show what can be spidered. On this site's sibling, searchengineworld, there's a spider simulator that will look at individual pages and report what's found. There's also a server header checker.

I use all of these tools when checking a site for spiderability.

[raptorix]

Try to navigate your site with a text only browser, i.e. Lynx, if your site is browsable with a text only browser, google will normally be able to crawl it.

[Brett_Tabke]

[searchengineworld.com...]

[Sandro]

Hey!

I do not get access to this link:
[searchengineworld.com...]

What am I doing wrong?
I have a paid membership.

Joerg

[Dreamquick]

Rather strangely it wont work via a direct link from here, but the same URL will work if you go via [searchengineworld.com...] and then click the "Sim Spider" link towards the bottom of the game.

- Tony

[indigojo]

I like this one

"This service allows web authors to see what their pages will look like (sort of) when viewed with Lynx, a text-mode web browser. "

[delorie.com...]