Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

Should I Panic

after changes Only index page is visited



9:07 pm on Jan 19, 2003 (gmt 0)

10+ Year Member

I recently moved my site to another host. The reason to be able use the use the custom error and change the format of the urls from
[mysite.com...] to

Every thing is working fine with regards to the 200 status.

I was told in another thread to block google from the old format by adding
Disallow /items to my robot.txt, and I thought that was the right thing to do.

I had freshbot on my site and grabbed the index page with all the links on it, but all the links in the index page are the category links (ford, BMW, Nissan, etc.), they got indexed by google and that what they show if you do a search for my keyword.

in the meanwhile, google dropped all 400+ pages they had in the old format since I asked it to not crawl them. now I have the lowest in traffic from google for some time.

My concern is, did I do the correct thing to block the crawler this early or I should have waited until the new format of urls got indexed?

Also, since I have been visited by the freshbot crawlerXX, would my site get deep crawled this month?

Please help.



9:15 pm on Jan 19, 2003 (gmt 0)

Personally, I would have waited until the new format was indexed, before I blocked Googlebot from the old format. I have done this before and it worked fine for me.

About the only thing you can do now is wait and be patient. If freshbot is coming around to your new format itís a good bet that you will deep crawled next time Googlebot is on the prowl.


1:01 am on Jan 20, 2003 (gmt 0)

10+ Year Member

Thanx lots0 for the reply.

Google was and still is my main source of traffic, I hope there is a dance this month and i hope it will start soon.


Featured Threads

Hot Threads This Week

Hot Threads This Month