Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

Site Not in Index But Gets Spidered Heavily Now

Is this daily spiderable a good sign?



5:34 am on May 14, 2004 (gmt 0)

10+ Year Member

I had a domain in google with great rankings but in the last update when a few sites copped some sort of penality (which is still don't know why) my site was also banned and totally out of the index.

However is still gets spidered daily and has even spidered new pages ive since added since this ban.

Is this a good sign my site will be straight back into the index once this ban lifts?


9:28 pm on May 15, 2004 (gmt 0)

10+ Year Member

I guess you're right. It's really a good sign if the bot is still hungry for your site.


9:38 pm on May 15, 2004 (gmt 0)

10+ Year Member

potentially. One can only hope.


3:32 am on May 16, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

Talking about spidering heavily, I believe new sites are not getting crawled as agressively as before. One of my new large content site sees the bot hitting only the main page and a few sub-pages daily (only crawled about 0.001% so far), but no deep crawl at all and its been almost one month since the site is online and its got around 100 of quality backlinks including many 7 PRs as well. With this rate, many webmaster may become insane.

Featured Threads

Hot Threads This Week

Hot Threads This Month