Google crawl

It has started On the 4th?

[zuko105]

Started on the 4th this month? Or am I just getting a late crawl?

Just looked at my weblogs and they are crawling my raw directory.

Threw a dummy index.htm file in all of my directories to try to prevent this from happening again.

Hope it works....or maybe I should just keep my server clean.....

Zuko

[dukeblue219]

There isn't any special crawl going on. The Googlebot can and will spider your files all month long, so don't be surprised when you see her tracks in your logs.

[edited by: dukeblue219 at 11:43 pm (utc) on Oct. 4, 2002]

[Marcia]

It's not unusual for sites to still be getting crawled.

Always keep sites clean, I understand a directory index that shows the file structure only is a hole that can be exploited. You can do pages with links to other parts of the site or the index page and use <meta name="robots" content="noindex,nofollow">

The October Everflux thread [webmasterworld.com] already started. Let us know if you see any immediate changes as a result of this, we like to keep track.