Forum Moderators: open

Message Too Old, No Replies

Does Freshbot obey "no index, no follow"?

         

dvduval

11:30 pm on Jul 14, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Freshbot seems to keep reindexing pages that have moved. I have "no index, follow", but it continues to ignore this and refreshes the page every day.

I thought this was a new super freshdeep bot. An unfortunate side effect is Google is NOT indexing my new material, but indexing lots of old moved pages.

What would be the best strategy to help Freshbot see the good stuff and ignore the old moved pages?

[Sorry, I'm an idiot - this should have been posted in Google News]

g1smd

12:13 am on Jul 15, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Umm, I had a site that moved to a new domain in May. I put noindex,follow on every page of the old site in mid-May and pointed all "internal" links on the old site to point to the equivalent page at the new site. The new site had only gone online at the beginning of May, and it dropped cleanly into the index in mid-June, the very same day Google removed all of the old pages from the index at the same time. The old site still shows PR4, but the new site is still PR0. The old site was at #80, and the new site is #1 with about 10 incoming links. I didn't see any problem with the noindex,follow method. The old site has only one incoming link, but is still listed in Altavista.