homepage Welcome to WebmasterWorld Guest from 54.237.122.241
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Aim-pro Newsletter: robots.txt and too many urls
Brett_Tabke




msg:1528150
 8:32 am on Jun 6, 2000 (gmt 0)

Checkout this months Aim-Pro newsletter. Very interesting reading on how to detect if your domain is still banned by Altavista. Thanks BK, Seth.

 

jilly




msg:1528151
 12:19 pm on Jun 6, 2000 (gmt 0)

Okay it's official -- I'm brain dead. Sorry Brett, it was right in front of my eyes!

seth_wilde




msg:1528152
 3:22 pm on Jun 6, 2000 (gmt 0)

That was BK's and Rick's brain child, I was definitely pleased with the results, I think it helps alot to explain why AV's been handling robot.txt wierd for some domains.

Brett_Tabke




msg:1528153
 7:41 pm on Jun 6, 2000 (gmt 0)

Oh ok, thanks Rick. I thought you were in on that Seth, my mistake. That is great trick. They didn't sign it other than mention BK.

I don't have access to any domains that Alta has banned. Those that they did, we got lifted in April and it has been smooth sailing ever since.

jilly been there - done that

Brett_Tabke




msg:1528154
 5:42 am on Jun 29, 2000 (gmt 0)

BK, you done any more work along these lines, or do we have to wait for the next newsletter?

BK




msg:1528155
 6:22 pm on Jun 29, 2000 (gmt 0)

Well, I never take my eyes off AV for too long - they always have something new in the works to make life interesting. :)

The thing I find most interesting at the moment is the banning or blocking of the WPG domains. Brent's explaination regarding AV claiming that his distributor's pages were too similar to his own is interesting.

Naturally the claim doesn't hold water if we look at www.mirc.com , www.mirc.co.uk , and a dozen other mirrors of the exact same content pages. What I'm doing is seeing whether or not there is any bias or detection of identical pages across multiple domains.

Results will take a little tme, of course, but the way I see it, AV have "targeted" Brent Winters' business and are using bluff and bull to hide behind. If it can happpen to him it can happen to us, so I'm going to try to burst this bubble before it lands on the rest of us.

BK

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved