homepage Welcome to WebmasterWorld Guest from 23.22.173.58
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Local / Foo
Forum Library, Charter, Moderators: incrediBILL & lawman

Foo Forum

    
Brett's new Bot Blog
Check it out...
Brian_M

5+ Year Member



 
Msg#: 9709 posted 6:09 pm on Dec 20, 2005 (gmt 0)

I was curious to see how Brett's "bot war" was progressing, so I took a quick look at his robots.txt file.

Interesting reading...

Thanks, Brett!

 

jimbeetle

WebmasterWorld Senior Member jimbeetle us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 9709 posted 6:28 pm on Dec 20, 2005 (gmt 0)

he, he, he.

That's great. Only an old BBS hand would come up with something like that.

bcolflesh

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 9709 posted 7:12 pm on Dec 20, 2005 (gmt 0)

That's interesting - I had no idea Brett coined the term "SERP".

pageoneresults

WebmasterWorld Senior Member pageoneresults us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 9709 posted 7:17 pm on Dec 20, 2005 (gmt 0)

That's a great idea!

I didn't know he coined that phrase either.

Has Brett started a new craze? Are we going to see robots.txt files become the next marketing medium? ;)

Frequent

5+ Year Member



 
Msg#: 9709 posted 7:24 pm on Dec 20, 2005 (gmt 0)

That's the best robots.txt I've ever read.

Freq---

Leosghost

WebmasterWorld Senior Member leosghost us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 9709 posted 10:10 pm on Dec 20, 2005 (gmt 0)

pageone' ..do not give people ideas ..all we need is keyword spamming of robots text ..this is just Mr Tabke showing his underworld credentials ..next he'll get into graphics in the robots text and regfiles ..;)

anax

5+ Year Member



 
Msg#: 9709 posted 12:13 am on Dec 21, 2005 (gmt 0)

We need some good ASCII art in there, Brett: maybe Santa and his sleigh, just to complete the retro feel.

(Wonders how to send a trackback from his own robots.txt...)

vincevincevince

WebmasterWorld Senior Member vincevincevince us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 9709 posted 12:17 am on Dec 21, 2005 (gmt 0)

That's a really imaginative idea. I'm wondering if there is a market for a cgi to manage the posting / edits / etc. for the robots.txt - may even manage comments via external links?

I'm almost inspired to write a whole article and put it in an html comment tag now ;-)

I'd be interested to see what would happen if someone put a link to the robots.txt file. Whilst we know robots.txt is indexed - would Google dutifully ignore the commented lines, or would it happy serve it within actual SERPS? If only I knew someone with a website :-(

Leosghost

WebmasterWorld Senior Member leosghost us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 9709 posted 12:30 am on Dec 21, 2005 (gmt 0)

I can just see it ..Brett adds "greetz fly out" and the "we may be on IRC but maybe not" to the robots text ..

retro with chrome fenders and old school flames ..

so who's first to get a robots.txt to top of serps in G then ..? ;)

TFMC_Mark

10+ Year Member



 
Msg#: 9709 posted 12:53 am on Dec 21, 2005 (gmt 0)

For some reason I have found myself blogging about webmasterworld twice in the past couple of months and both times it has been because I have been amazed by the robots.txt.

When all bots were banned I raved, and now there is a robots.txt bot blog. Brett is trying to be old_school cool and it is working for me. (so much so that I come out of lurking mode for a brief post.)

Brett_Tabke

WebmasterWorld Administrator brett_tabke us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 9709 posted 10:13 pm on Dec 21, 2005 (gmt 0)

Thanks Mark.

PhraSEOlogy

10+ Year Member



 
Msg#: 9709 posted 4:21 am on Dec 22, 2005 (gmt 0)

WebmasterWorld - Expect the unexpected!

Some ascii art would give it a little boost.

Talikng about the retro feel - I got shudders thinking about the LARGE letters you used to get on the first page of green bar report printouts like:

VM/CMS or IBM/3270

But I digress...

JAB Creations

WebmasterWorld Senior Member jab_creations us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 9709 posted 8:02 am on Dec 22, 2005 (gmt 0)

IoI I *love* finding things on my own that I would only be able to find being a web designer (though I didn't see that until I read the thread). A good hidden read. It's an area of interest and I'd like to learn from Brett's experiences and his observations as well. I'd have to actually stop and think if I was offered a choice between a room full of gold or an access log from WW, Wiki, Google, or Yahoo!

I would like to add that the AdWords trick people most likely (in my kind of humble opinion heh) that most people's screens are not optimized. Many people EEK when I max out (hopefully 100% unless there is tearing then typically between 84-92% on cheapo-screens) the RGB setting without realizing that it, brightness, and contrast are three very separate things. Honestly the only reason I know the difference is because I tweak everything I can that I understand including my hardware. So long as monitors are sent unoptimized to non-tech-savvy computer users this screen contrast trick should continue to work I suspect.

vincevincevince

WebmasterWorld Senior Member vincevincevince us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 9709 posted 11:13 am on Dec 22, 2005 (gmt 0)

AdWords trick people most likely (in my kind of humble opinion heh) that most people's screens are not optimized

Not sure I follow?

pmkpmk

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 9709 posted 11:22 am on Dec 22, 2005 (gmt 0)

@pageoneresults:
Are we going to see robots.txt files become the next marketing medium?

Reading the robots.txt of competitive sites is a must for me!

@Leosghost:
Have you tried [google.com...] - seems Brett is in a good neighbourhood.

vincevincevince

WebmasterWorld Senior Member vincevincevince us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 9709 posted 11:33 am on Dec 22, 2005 (gmt 0)

Are we going to see robots.txt files become the next marketing medium?

A firefox plugin to show the robots.txt in a window / popup anyone?

JAB Creations

WebmasterWorld Senior Member jab_creations us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 9709 posted 3:56 pm on Dec 22, 2005 (gmt 0)

Vince, most people's screens have low RGB settings, and various levels of brightness and contrast (the physical controls/options besides the on/off button for clarification). So if you put my screen next to someone else's you definitely see things on my screen (optimized) that a non-savvy user would not be able to see. The same if you see a rack of screens all with various settings in a store and again are unoptimized at least in some cases I'd imagine in that sort of scenario.

Leosghost

WebmasterWorld Senior Member leosghost us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 9709 posted 10:30 am on Dec 23, 2005 (gmt 0)

@Brett.. I just noticed something , should n't
# [webmasterworld.com...]
# Actual robots can always be found here for: [webmasterworld.com...]
# Old full robots.txt can be found here: [webmasterworld.com...]

read
# [webmasterworld.com...]
# Actual robots .txt can always be found here for: [webmasterworld.com...]
# Old full robots.txt can be found here: [webmasterworld.com...]

or are you running an aff deal via the backdoor with 20th century fox for the movie toys? ;)

pmkpmk

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 9709 posted 2:29 pm on Dec 28, 2005 (gmt 0)

Getting a 404 right now.

Key_Master

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 9709 posted 10:59 pm on Dec 27, 2005 (gmt 0)

What happened to it? Hopefully, it's coming back. I really enjoy reading it. :)

iamlost

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 9709 posted 1:33 am on Dec 28, 2005 (gmt 0)

Unlike 99.99% of all bloggers who publish for a week and then get on with their lives leaving a tiny bit of their life littering the web Brett has picked up his comments.

I also thought it great fun...

Save the WebmasterWorld robots.txt Blog

New WWW Life Form Missing

I Scream, You Scream, We all Scream for robots.txt Blog

Brett_Tabke

WebmasterWorld Administrator brett_tabke us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 9709 posted 2:40 pm on Dec 28, 2005 (gmt 0)

ahhh, the dangers of a scripted bots.txt. ;-)

...should work now (tm)

whoisgregg

WebmasterWorld Senior Member whoisgregg us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 9709 posted 10:02 pm on Jan 2, 2006 (gmt 0)

So, most blogs I just subscribe to the RSS feed. I've already checked for a /robots.rss or /robots.xml with no luck. :( So, this blog I have to remember to check every day.

Is it ironic that this is the first time I've felt like programming a robot for my personal use? :)

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Local / Foo
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved