homepage Welcome to WebmasterWorld Guest from 54.204.58.87
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Local / Foo
Forum Library, Charter, Moderators: incrediBILL & lawman

Foo Forum

    
Brett's new Bot Blog
Check it out...
Brian_M




msg:291290
 6:09 pm on Dec 20, 2005 (gmt 0)

I was curious to see how Brett's "bot war" was progressing, so I took a quick look at his robots.txt file.

Interesting reading...

Thanks, Brett!

 

jimbeetle




msg:291291
 6:28 pm on Dec 20, 2005 (gmt 0)

he, he, he.

That's great. Only an old BBS hand would come up with something like that.

bcolflesh




msg:291292
 7:12 pm on Dec 20, 2005 (gmt 0)

That's interesting - I had no idea Brett coined the term "SERP".

pageoneresults




msg:291293
 7:17 pm on Dec 20, 2005 (gmt 0)

That's a great idea!

I didn't know he coined that phrase either.

Has Brett started a new craze? Are we going to see robots.txt files become the next marketing medium? ;)

Frequent




msg:291294
 7:24 pm on Dec 20, 2005 (gmt 0)

That's the best robots.txt I've ever read.

Freq---

Leosghost




msg:291295
 10:10 pm on Dec 20, 2005 (gmt 0)

pageone' ..do not give people ideas ..all we need is keyword spamming of robots text ..this is just Mr Tabke showing his underworld credentials ..next he'll get into graphics in the robots text and regfiles ..;)

anax




msg:291296
 12:13 am on Dec 21, 2005 (gmt 0)

We need some good ASCII art in there, Brett: maybe Santa and his sleigh, just to complete the retro feel.

(Wonders how to send a trackback from his own robots.txt...)

vincevincevince




msg:291297
 12:17 am on Dec 21, 2005 (gmt 0)

That's a really imaginative idea. I'm wondering if there is a market for a cgi to manage the posting / edits / etc. for the robots.txt - may even manage comments via external links?

I'm almost inspired to write a whole article and put it in an html comment tag now ;-)

I'd be interested to see what would happen if someone put a link to the robots.txt file. Whilst we know robots.txt is indexed - would Google dutifully ignore the commented lines, or would it happy serve it within actual SERPS? If only I knew someone with a website :-(

Leosghost




msg:291298
 12:30 am on Dec 21, 2005 (gmt 0)

I can just see it ..Brett adds "greetz fly out" and the "we may be on IRC but maybe not" to the robots text ..

retro with chrome fenders and old school flames ..

so who's first to get a robots.txt to top of serps in G then ..? ;)

TFMC_Mark




msg:291299
 12:53 am on Dec 21, 2005 (gmt 0)

For some reason I have found myself blogging about webmasterworld twice in the past couple of months and both times it has been because I have been amazed by the robots.txt.

When all bots were banned I raved, and now there is a robots.txt bot blog. Brett is trying to be old_school cool and it is working for me. (so much so that I come out of lurking mode for a brief post.)

Brett_Tabke




msg:291300
 10:13 pm on Dec 21, 2005 (gmt 0)

Thanks Mark.

PhraSEOlogy




msg:291301
 4:21 am on Dec 22, 2005 (gmt 0)

WebmasterWorld - Expect the unexpected!

Some ascii art would give it a little boost.

Talikng about the retro feel - I got shudders thinking about the LARGE letters you used to get on the first page of green bar report printouts like:

VM/CMS or IBM/3270

But I digress...

JAB Creations




msg:291302
 8:02 am on Dec 22, 2005 (gmt 0)

IoI I *love* finding things on my own that I would only be able to find being a web designer (though I didn't see that until I read the thread). A good hidden read. It's an area of interest and I'd like to learn from Brett's experiences and his observations as well. I'd have to actually stop and think if I was offered a choice between a room full of gold or an access log from WW, Wiki, Google, or Yahoo!

I would like to add that the AdWords trick people most likely (in my kind of humble opinion heh) that most people's screens are not optimized. Many people EEK when I max out (hopefully 100% unless there is tearing then typically between 84-92% on cheapo-screens) the RGB setting without realizing that it, brightness, and contrast are three very separate things. Honestly the only reason I know the difference is because I tweak everything I can that I understand including my hardware. So long as monitors are sent unoptimized to non-tech-savvy computer users this screen contrast trick should continue to work I suspect.

vincevincevince




msg:291303
 11:13 am on Dec 22, 2005 (gmt 0)

AdWords trick people most likely (in my kind of humble opinion heh) that most people's screens are not optimized

Not sure I follow?

pmkpmk




msg:291304
 11:22 am on Dec 22, 2005 (gmt 0)

@pageoneresults:
Are we going to see robots.txt files become the next marketing medium?

Reading the robots.txt of competitive sites is a must for me!

@Leosghost:
Have you tried [google.com...] - seems Brett is in a good neighbourhood.

vincevincevince




msg:291305
 11:33 am on Dec 22, 2005 (gmt 0)

Are we going to see robots.txt files become the next marketing medium?

A firefox plugin to show the robots.txt in a window / popup anyone?

JAB Creations




msg:291306
 3:56 pm on Dec 22, 2005 (gmt 0)

Vince, most people's screens have low RGB settings, and various levels of brightness and contrast (the physical controls/options besides the on/off button for clarification). So if you put my screen next to someone else's you definitely see things on my screen (optimized) that a non-savvy user would not be able to see. The same if you see a rack of screens all with various settings in a store and again are unoptimized at least in some cases I'd imagine in that sort of scenario.

Leosghost




msg:291307
 10:30 am on Dec 23, 2005 (gmt 0)

@Brett.. I just noticed something , should n't
# [webmasterworld.com...]
# Actual robots can always be found here for: [webmasterworld.com...]
# Old full robots.txt can be found here: [webmasterworld.com...]

read
# [webmasterworld.com...]
# Actual robots .txt can always be found here for: [webmasterworld.com...]
# Old full robots.txt can be found here: [webmasterworld.com...]

or are you running an aff deal via the backdoor with 20th century fox for the movie toys? ;)

pmkpmk




msg:291308
 2:29 pm on Dec 28, 2005 (gmt 0)

Getting a 404 right now.

Key_Master




msg:291309
 10:59 pm on Dec 27, 2005 (gmt 0)

What happened to it? Hopefully, it's coming back. I really enjoy reading it. :)

iamlost




msg:291310
 1:33 am on Dec 28, 2005 (gmt 0)

Unlike 99.99% of all bloggers who publish for a week and then get on with their lives leaving a tiny bit of their life littering the web Brett has picked up his comments.

I also thought it great fun...

Save the WebmasterWorld robots.txt Blog

New WWW Life Form Missing

I Scream, You Scream, We all Scream for robots.txt Blog

Brett_Tabke




msg:291311
 2:40 pm on Dec 28, 2005 (gmt 0)

ahhh, the dangers of a scripted bots.txt. ;-)

...should work now (tm)

whoisgregg




msg:291312
 10:02 pm on Jan 2, 2006 (gmt 0)

So, most blogs I just subscribe to the RSS feed. I've already checked for a /robots.rss or /robots.xml with no luck. :( So, this blog I have to remember to check every day.

Is it ironic that this is the first time I've felt like programming a robot for my personal use? :)

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Local / Foo
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved