Welcome to WebmasterWorld Guest from 54.166.191.159

Forum Moderators: goodroi

Message Too Old, No Replies

Effect of Robots.txt

     
11:21 am on Nov 9, 2004 (gmt 0)

10+ Year Member



hi
in my live site robot.txt was wrongly added for disallowing all spiders not to crawl my site. can you please tell me what problem it can create for my site.
as this file was there from last seven days on my site.

please reply....

11:39 am on Nov 9, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



It would block all spider for the last 7 days from accessing your site, now the block has been removed next time you have a robot visit it should spider your site again

Have you noticed any pages removed from any of the search engines?

A good guide is the Robots.txt Tutorial
[searchengineworld.com...]

12:28 pm on Nov 9, 2004 (gmt 0)

10+ Year Member



hi

yes when i search for my domain, it syays no information available.

12:38 pm on Nov 9, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



That means it's been removed, there was a big update last week and the new database has gone live over the last couple of days.

keep checking you log file for any spider activity and hopefully googlebot will revisit your site soon and rectify the problem

5:31 am on Nov 15, 2004 (gmt 0)

10+ Year Member



hi
thanks for your reply but can you show me some good ways how to get my site back in googl as wehn i do URL search our site doesnt appear.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month