Forum Moderators: DixonJones

Message Too Old, No Replies

Spiders Hit and Go

Is there something wrong with my html?

         

SPYGUY

11:40 am on May 6, 2002 (gmt 0)



I've been checking my logs and it seems googlebot 2.1 has been to my site. It is the first time it has been in the logs for awhile which makes me happy because after last index I was completed dropped from google and I couldn't figure out why, maybe this new visit is a sign of things to come?

But, now in my logs, it seems that I'm getting hit by certain spiders i.e. googlebot, fast, etc., but then there is either 0 or 1 pageviews and thats the end of it.

Is there a problem with my pages that is making them 'uncrawlable'?

Thanks for your help!

wilreynolds

2:00 pm on May 6, 2002 (gmt 0)



Here's some things to think about:

1> Are you links image maps, or dynamically generated?

2> Do you have a frames site?

3> Do you have text links throughout the site for the bot to follow?

4> Do you have any bots excluded in your robots.txt?

SPYGUY

4:08 pm on May 6, 2002 (gmt 0)



Sorry, I wasn't very descriptive in my first post...

The answer to all of those is no, except for question 3. I have lots of text links for the bots to follow.
I don't have robots.txt file.
I had a listing at 3 for my most important keyword at one time without one, on google- so I can't really say how important the robots.txt file is.

My html seems really good, Is it typical for spiders to hit and then not keep going ???