homepage Welcome to WebmasterWorld Guest from 54.211.113.223
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
How Can I Make Bots Visit My Web Site?
emadido



 
Msg#: 3282726 posted 2:07 pm on Mar 15, 2007 (gmt 0)

Hi all

i want to know how can i make search bots visit my site?

<url removed>

[edited by: encyclo at 3:02 pm (utc) on Mar. 15, 2007]
[edit reason] no URLs please, see TOS [webmasterworld.com] [/edit]

 

Vishal

10+ Year Member



 
Msg#: 3282726 posted 2:30 pm on Mar 15, 2007 (gmt 0)

One way would be to read more and focus less on posting URLs.

Check out Library posts here & technically speaking, you will receive your first diploma in SEO.

jdMorgan

WebmasterWorld Senior Member jdmorgan us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3282726 posted 2:34 pm on Mar 15, 2007 (gmt 0)

Robots.txt cannot be used to "make" robots do anything. It is used to ask robots not to fetch certain parts of your Web site such as membership signup pages, scripts that take an action you would not want a robot to trigger, etc.

Compliant robots will honor your requests, while e-mail address harvesters and site scrapers won't. For the latter, you can take stronger steps by using server-side access controls (e.g. .htaccess) and/or server-side scripting.

If you want robots to visit your site, then you need to get other Web sites to link to your site. Robots follow the links that they find on sites that they already know about, and find more sites. "Search engine submission" as practiced in the 90s is dead, and that's why the similarly-titled forum here at WebmasterWorld is deprecated.

To get more love from the search engines, get more links from other topically-related sites with link text that is relevant to their site and to yours.

Jim

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved