Forum Moderators: open

Message Too Old, No Replies

How far in advance should a site be spidered?

What is the consensus on the 'cutoff date' for new indexes?

         

stuntdubl

9:25 pm on Sep 16, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I know this has been discussed before, but I am wondering with Google's fairly new emphasis placed on 'fresh' pages if the "cutoff date" for pages being indexed in the new database has changed. I have heard that if you want a page to make it into the monthly index you should have it on your site a month (sometimes even two months) ahead of time. Has this changed at all? and does pagerank have any impact on how long it takes for a page to be included in the monthly index?

Nick_W

9:28 pm on Sep 16, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



There's generally a crawl a few days after an Update.

If you get crawled then, you're in the next Update...

Nick

nancyb

10:41 pm on Sep 16, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



What I think I've discovered this month is that a file that is crawled after the the cut off date can come into the index in the "everflux" mode.

Since I was (hopefully it is now over!) penalized until this last update it was easy to see because the "new" file showed in similar serps on the first page to what I enjoyed before the site was penalized.

When "everflux" was not in mode, the old file was listed in the horrible old positions. Now, I believe there is some settleing down of the "everflux", at least in my categories, because the "old" file is now showing in the "new this month" serps.

So, what I really said, is that a new file can show up in the index, but it may not stay in there and may/may not show in the next update.