homepage Welcome to WebmasterWorld Guest from 54.198.140.148
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / WebmasterWorld / Webmaster General
Forum Library, Charter, Moderators: phranque & physics

Webmaster General Forum

    
Getting on archive.org
markbiz




msg:4001779
 11:07 pm on Oct 5, 2009 (gmt 0)

I have a site that's been active since 2007, and it's still not on archive.org. I've seen ia_archiver in awstats a lot of times, so i don't know what's wrong. Any idea how i can get my pages on there?

 

Leosghost




msg:4001791
 11:26 pm on Oct 5, 2009 (gmt 0)

curiosity :)..why would you want on be on there ?

ChipD




msg:4001796
 11:41 pm on Oct 5, 2009 (gmt 0)

Just to check the obvious, you don't have a robots noarchive instruction on those pages, right?

phranque




msg:4001867
 2:47 am on Oct 6, 2009 (gmt 0)

have you checked your robots.txt file for relevant exclusions?

you should check the Internet Archive Frequently Asked Questions [archive.org] or you can try filling out the form to be crawled by Alexa [alexa.com].

maximillianos




msg:4001891
 4:08 am on Oct 6, 2009 (gmt 0)

Getting your pages listed there is more trouble than it is worth. Once they get them, they will always be public... Even if you delete them.

Be careful what you wish for... =)

markbiz




msg:4002696
 3:45 am on Oct 7, 2009 (gmt 0)

curiosity :)..why would you want on be on there ?

In case someone steals my articles, or if someone accuses me of stealing.




you don't have a robots noarchive instruction on those pages, right?

I don't use that on any of my pages.




have you checked your robots.txt file for relevant exclusions?

I have 17 bots blocked in there, but none of them are ia_archiver.




you can try filling out the form to be crawled by Alexa

I did that a long time ago.

tangor




msg:4002710
 5:02 am on Oct 7, 2009 (gmt 0)

Make sure your server is not sending a x-robots header.... (probably not). Me and Wayback Machine parted company many years back.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / WebmasterWorld / Webmaster General
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved