homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

Anyone seen this?

 12:50 pm on Feb 17, 2007 (gmt 0)

I have a travel guide website and have just noticed this make its first appearance.

I seem to remember reading somewhere that a TripAdvisor bot crawls
the web for content which is then re-written by their editors. Don't know if this is accurate ..?



 8:31 pm on Feb 26, 2007 (gmt 0)

This bot has visited one of my sites a few times over the last month, asking for robots.txt and one other page.

As it's highly unlikely to bring any traffic to the site, I've now banned it.


 10:28 pm on Mar 16, 2007 (gmt 0)

i don't suppose you know the name of it? i wouldn't mind banning it too

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved