homepage Welcome to WebmasterWorld Guest from 107.20.91.81
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
How to remove your site from the Alexa index
Alexa contact details?
johnd




msg:403465
 1:55 am on Dec 27, 2002 (gmt 0)

how do I remove my site from the alexa db? I have managed to remove it from www.archive.org by blocking the archiver spider through the robots.txt file but what about alexa?

 

jimbeetle




msg:403466
 3:35 am on Dec 27, 2002 (gmt 0)

Hi johnd,

They don't give any details on removal except for robots.txt,

User-agent: ia_archiver
Disallow: /

Not sure how often Alexa crawls (don't think it's that frequent) so it might take some time to have any effect.

Jim

wilderness




msg:403467
 4:39 am on Dec 27, 2002 (gmt 0)

The Alexa site under help and robots offers the following
[pages.alexa.com...]
This page offers a variety of options.

The Alexa search is "Powered by Google"

carfac




msg:403468
 12:50 am on Jan 13, 2003 (gmt 0)

They sure are not very responsive. I used their form to remove personal data about me from the website profile two weeks ago... and it is still there!

dave

johnd




msg:403469
 1:27 am on Jan 13, 2003 (gmt 0)

as we all know, their robot spiders what they want, regardless of our robots.txt files..

carfac




msg:403470
 5:44 pm on Jan 13, 2003 (gmt 0)

johnd:

Not on MY site.

In fact, now that Amazon is using alexa for their little spy, it is for sure denied via UA banning (and IP banning!) Of course, this is in violation of Amazon Associates TOS, but their robots violayted my TOS first!

dave

Brett_Tabke




msg:403471
 5:48 pm on Jan 13, 2003 (gmt 0)

Alexa uses the Google db. Alexa and archive.org are completely different entities when it comes to search.

duncan12




msg:403472
 10:03 pm on Jan 13, 2003 (gmt 0)

Lots of different issues getting mixed up here. Let's separate the strands...

Alexa shows some info about sites on the "Detail Page"... I believe the original post wanted to remove his contact info from these pages. Alexa has a form, listed at the bottom of all detail pages for this purpose. Response is slow though...

Alexa crawls sites... this is a different issue. Alexa does obey robots.txt, but because Alexa crawls aggressively and frequently, changes to your robots.txt files are not noticed right away. You wouldn't want the crawler to check the robots.txt file each and every time it crawled a page on your site... so it picks it up periodically. If you ever have a problem with the crawl, crawl@alexa.com usually gets a same day response.

Alexa shows Google results. Changes to Alexa's site info or Alexa's crawl won't affect the search results. Search is separate from Detail Pages and from the crawl.

Cheers.

Brett_Tabke




msg:403473
 2:01 am on Jan 14, 2003 (gmt 0)

Thank you for the clarifications Duncan.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved