Forum Moderators: open

Message Too Old, No Replies

Will an endless loop affect my crawl?

         

eaden

2:23 am on Jan 6, 2003 (gmt 0)

10+ Year Member



Hi there,
first post here, I have been reading this forum for about a month, and got a few nice tips, and in my first update, straight to PR5 and #8 for main keyword, and #1 for other keyword combinitaions :) so thanks :)

But, googlebot is deep crawling my site now, and has been for a day or so, but it's in an endless loop.

i.e. I have a calender, and it will keep on going up to 3000AD+!

Now I understand that google is reasonably intelligent about these sort of things, but would it affect the deep crawl at all? Will it eventually bailout and not bother with the rest of the site? I would rather it deep crawled my forum. The endless loop is in a program that I use, and there are a few other ones, modifiying them would be a lot of work, but if it would be quite benificial i would consider doing it.

Eaden

mat_bastian

2:26 am on Jan 6, 2003 (gmt 0)

10+ Year Member



<miss read your question>

warmasol

2:32 am on Jan 6, 2003 (gmt 0)

10+ Year Member



Can you exclude the pages with the endless loops by robots.txt?

eaden

2:36 am on Jan 6, 2003 (gmt 0)

10+ Year Member



>> Can you exclude the pages with the endless loops by robots.txt?

No, it it like

www. .com/page/page?action=edit
www .com/page/page?action=diff

etc. I could use modrewrite, or hack the php code to some extent. I know how to stop it, but it is a lot of work. My question is "Is it bad that googlebot goes in -nearly- endless loop?"

jdMorgan

3:13 am on Jan 6, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



eaden,

Welcome to WebmasterWorld [webmasterworld.com]!

Put yourself in a search engine company's shoes. They spend hours crawling your site and find very little useful content... What "rating" would you expect?

Fix it.

Jim