homepage Welcome to WebmasterWorld Guest from 174.129.76.87
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Effect of Robots.txt
seolancer

10+ Year Member



 
Msg#: 488 posted 11:21 am on Nov 9, 2004 (gmt 0)

hi
in my live site robot.txt was wrongly added for disallowing all spiders not to crawl my site. can you please tell me what problem it can create for my site.
as this file was there from last seven days on my site.

please reply....

 

ncw164x

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 488 posted 11:39 am on Nov 9, 2004 (gmt 0)

It would block all spider for the last 7 days from accessing your site, now the block has been removed next time you have a robot visit it should spider your site again

Have you noticed any pages removed from any of the search engines?

A good guide is the Robots.txt Tutorial
[searchengineworld.com...]

seolancer

10+ Year Member



 
Msg#: 488 posted 12:28 pm on Nov 9, 2004 (gmt 0)

hi

yes when i search for my domain, it syays no information available.

ncw164x

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 488 posted 12:38 pm on Nov 9, 2004 (gmt 0)

That means it's been removed, there was a big update last week and the new database has gone live over the last couple of days.

keep checking you log file for any spider activity and hopefully googlebot will revisit your site soon and rectify the problem

seolancer

10+ Year Member



 
Msg#: 488 posted 5:31 am on Nov 15, 2004 (gmt 0)

hi
thanks for your reply but can you show me some good ways how to get my site back in googl as wehn i do URL search our site doesnt appear.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved