homepage Welcome to WebmasterWorld Guest from 54.205.144.54
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / WebmasterWorld / Webmaster General
Forum Library, Charter, Moderators: phranque

Webmaster General Forum

    
Archive.org :: What's the deal?
Why hasn't it visited me in sooo long?
Vidman




msg:359162
 9:34 am on Jan 12, 2003 (gmt 0)

My site was archived a number of times throughout 2001, but it hasn't touched me since.

Anyone know why? .. and how to invite it back?

-=vidmån=-

 

vitaplease




msg:359163
 10:40 am on Jan 12, 2003 (gmt 0)

Vidman, welcome to WebmasterWorld,

Check this thread:

[webmasterworld.com...]

It seems they are always 6 months behind at least.

duncan12




msg:359164
 11:12 pm on Jan 13, 2003 (gmt 0)

The Internet Archive gets its data from Alexa. Check to see if Alexa has crawled you by checking your logs for "ia_archiver".

Alexa donates its crawl to the Internet archive after 6 months, so the Wayback Machine will never have the most recent stuff...

Mardi_Gras




msg:359165
 11:21 pm on Jan 13, 2003 (gmt 0)

You may want to check your robots.txt file and make sure you have not disallowed the bot.

Vidman




msg:359166
 6:07 am on Jan 14, 2003 (gmt 0)

Thanks for not reaming me out for not searching previous threads :)

My robots.txt contains:

User-agent: *
Disallow: /cgi-bin/

Thanks for the log tip... I'll write a script to check it on a daily basis.

Vidman




msg:359167
 2:14 am on Jan 18, 2003 (gmt 0)

*woohoo*

I see it crawling my site :)

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / WebmasterWorld / Webmaster General
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved