homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

Should I Panic
after changes Only index page is visited

10+ Year Member

Msg#: 8579 posted 9:07 pm on Jan 19, 2003 (gmt 0)

I recently moved my site to another host. The reason to be able use the use the custom error and change the format of the urls from
[mysite.com...] to

Every thing is working fine with regards to the 200 status.

I was told in another thread to block google from the old format by adding
Disallow /items to my robot.txt, and I thought that was the right thing to do.

I had freshbot on my site and grabbed the index page with all the links on it, but all the links in the index page are the category links (ford, BMW, Nissan, etc.), they got indexed by google and that what they show if you do a search for my keyword.

in the meanwhile, google dropped all 400+ pages they had in the old format since I asked it to not crawl them. now I have the lowest in traffic from google for some time.

My concern is, did I do the correct thing to block the crawler this early or I should have waited until the new format of urls got indexed?

Also, since I have been visited by the freshbot crawlerXX, would my site get deep crawled this month?

Please help.




Msg#: 8579 posted 9:15 pm on Jan 19, 2003 (gmt 0)

Personally, I would have waited until the new format was indexed, before I blocked Googlebot from the old format. I have done this before and it worked fine for me.

About the only thing you can do now is wait and be patient. If freshbot is coming around to your new format itís a good bet that you will deep crawled next time Googlebot is on the prowl.


10+ Year Member

Msg#: 8579 posted 1:01 am on Jan 20, 2003 (gmt 0)

Thanx lots0 for the reply.

Google was and still is my main source of traffic, I hope there is a dance this month and i hope it will start soon.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved