homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

I don't have a robots.txt file..error 404
Can't get good Google or Yahoo placement

 1:28 am on Feb 11, 2004 (gmt 0)

If I search using the term Jaguar parts, my site comes up in the top 5 on most search engines EXCEPT Yahoo and Google, I get 20 Error 404 reports showing robots.txt not found and I thought I needed this file to get in these search engine listings. What's strane also is I can go to Overture.com and type in Jaguar parts and I am #5 in the free listings. Overture says it's powered by Yahoo. Shouldn't I be #5 in Yahoo. After reading some info about this robots.txt I think it is only to stop crawlers. Anyone know what I need to do to get found?



 1:42 am on Feb 11, 2004 (gmt 0)

Robots.txt isn't required. Good backlinks are. Work on getting "google" listed sites to link back to you and that should solve your problem with Google.


 1:50 am on Feb 11, 2004 (gmt 0)

I know that I have many out there, I sell Jaguar parts and I linked up to Jaguar clubs and different directories, but I have a feeling that something is stopping Google and Yahoo from listing me but I can't figure it out. Any suggestions or should I find someone more experienced look over my site? I don't mind spending the cash, pay per clicks are already taking it!


 1:53 am on Feb 11, 2004 (gmt 0)

How old is the domain? How long have you been linked with other Google listed site? About a year ago, Google would get all our pages every 30 days. Now that only get a % of them each month and get them all every few months.


 2:04 am on Feb 11, 2004 (gmt 0)

Well I just realized that I have to pages [url.com...] and www.url.com would this affect Google and Yahoo? I'm not sure which page is linked. I read somewhere that I should link one to the other so the crawlers get to it. What do you think?


 2:17 am on Feb 11, 2004 (gmt 0)


Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved