Welcome to WebmasterWorld Guest from 23.22.46.195

Forum Moderators: incrediBILL & lawman

Brett's new Bot Blog

Check it out...

   
6:09 pm on Dec 20, 2005 (gmt 0)

5+ Year Member



I was curious to see how Brett's "bot war" was progressing, so I took a quick look at his robots.txt file.

Interesting reading...

Thanks, Brett!

6:28 pm on Dec 20, 2005 (gmt 0)

WebmasterWorld Senior Member jimbeetle is a WebmasterWorld Top Contributor of All Time 10+ Year Member



he, he, he.

That's great. Only an old BBS hand would come up with something like that.

7:12 pm on Dec 20, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



That's interesting - I had no idea Brett coined the term "SERP".
7:17 pm on Dec 20, 2005 (gmt 0)

WebmasterWorld Senior Member pageoneresults is a WebmasterWorld Top Contributor of All Time 10+ Year Member



That's a great idea!

I didn't know he coined that phrase either.

Has Brett started a new craze? Are we going to see robots.txt files become the next marketing medium? ;)

7:24 pm on Dec 20, 2005 (gmt 0)

10+ Year Member



That's the best robots.txt I've ever read.

Freq---

10:10 pm on Dec 20, 2005 (gmt 0)

WebmasterWorld Senior Member leosghost is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



pageone' ..do not give people ideas ..all we need is keyword spamming of robots text ..this is just Mr Tabke showing his underworld credentials ..next he'll get into graphics in the robots text and regfiles ..;)
12:13 am on Dec 21, 2005 (gmt 0)

5+ Year Member



We need some good ASCII art in there, Brett: maybe Santa and his sleigh, just to complete the retro feel.

(Wonders how to send a trackback from his own robots.txt...)

12:17 am on Dec 21, 2005 (gmt 0)

WebmasterWorld Senior Member vincevincevince is a WebmasterWorld Top Contributor of All Time 10+ Year Member



That's a really imaginative idea. I'm wondering if there is a market for a cgi to manage the posting / edits / etc. for the robots.txt - may even manage comments via external links?

I'm almost inspired to write a whole article and put it in an html comment tag now ;-)

I'd be interested to see what would happen if someone put a link to the robots.txt file. Whilst we know robots.txt is indexed - would Google dutifully ignore the commented lines, or would it happy serve it within actual SERPS? If only I knew someone with a website :-(

12:30 am on Dec 21, 2005 (gmt 0)

WebmasterWorld Senior Member leosghost is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



I can just see it ..Brett adds "greetz fly out" and the "we may be on IRC but maybe not" to the robots text ..

retro with chrome fenders and old school flames ..

so who's first to get a robots.txt to top of serps in G then ..? ;)

12:53 am on Dec 21, 2005 (gmt 0)

10+ Year Member



For some reason I have found myself blogging about webmasterworld twice in the past couple of months and both times it has been because I have been amazed by the robots.txt.

When all bots were banned I raved, and now there is a robots.txt bot blog. Brett is trying to be old_school cool and it is working for me. (so much so that I come out of lurking mode for a brief post.)

10:13 pm on Dec 21, 2005 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Thanks Mark.
4:21 am on Dec 22, 2005 (gmt 0)

10+ Year Member



WebmasterWorld - Expect the unexpected!

Some ascii art would give it a little boost.

Talikng about the retro feel - I got shudders thinking about the LARGE letters you used to get on the first page of green bar report printouts like:

VM/CMS or IBM/3270

But I digress...

8:02 am on Dec 22, 2005 (gmt 0)

WebmasterWorld Senior Member jab_creations is a WebmasterWorld Top Contributor of All Time 10+ Year Member



IoI I *love* finding things on my own that I would only be able to find being a web designer (though I didn't see that until I read the thread). A good hidden read. It's an area of interest and I'd like to learn from Brett's experiences and his observations as well. I'd have to actually stop and think if I was offered a choice between a room full of gold or an access log from WW, Wiki, Google, or Yahoo!

I would like to add that the AdWords trick people most likely (in my kind of humble opinion heh) that most people's screens are not optimized. Many people EEK when I max out (hopefully 100% unless there is tearing then typically between 84-92% on cheapo-screens) the RGB setting without realizing that it, brightness, and contrast are three very separate things. Honestly the only reason I know the difference is because I tweak everything I can that I understand including my hardware. So long as monitors are sent unoptimized to non-tech-savvy computer users this screen contrast trick should continue to work I suspect.

11:13 am on Dec 22, 2005 (gmt 0)

WebmasterWorld Senior Member vincevincevince is a WebmasterWorld Top Contributor of All Time 10+ Year Member



AdWords trick people most likely (in my kind of humble opinion heh) that most people's screens are not optimized

Not sure I follow?

11:22 am on Dec 22, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



@pageoneresults:
Are we going to see robots.txt files become the next marketing medium?

Reading the

robots.txt
of competitive sites is a must for me!

@Leosghost:
Have you tried [google.com...] - seems Brett is in a good neighbourhood.

11:33 am on Dec 22, 2005 (gmt 0)

WebmasterWorld Senior Member vincevincevince is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Are we going to see robots.txt files become the next marketing medium?

A firefox plugin to show the robots.txt in a window / popup anyone?

3:56 pm on Dec 22, 2005 (gmt 0)

WebmasterWorld Senior Member jab_creations is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Vince, most people's screens have low RGB settings, and various levels of brightness and contrast (the physical controls/options besides the on/off button for clarification). So if you put my screen next to someone else's you definitely see things on my screen (optimized) that a non-savvy user would not be able to see. The same if you see a rack of screens all with various settings in a store and again are unoptimized at least in some cases I'd imagine in that sort of scenario.
10:30 am on Dec 23, 2005 (gmt 0)

WebmasterWorld Senior Member leosghost is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



@Brett.. I just noticed something , should n't
# [webmasterworld.com...]
# Actual robots can always be found here for: [webmasterworld.com...]
# Old full robots.txt can be found here: [webmasterworld.com...]
read
# [webmasterworld.com...]
# Actual robots .txt can always be found here for: [webmasterworld.com...]
# Old full robots.txt can be found here: [webmasterworld.com...]

or are you running an aff deal via the backdoor with 20th century fox for the movie toys? ;)

2:29 pm on Dec 28, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Getting a 404 right now.
10:59 pm on Dec 27, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



What happened to it? Hopefully, it's coming back. I really enjoy reading it. :)
1:33 am on Dec 28, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Unlike 99.99% of all bloggers who publish for a week and then get on with their lives leaving a tiny bit of their life littering the web Brett has picked up his comments.

I also thought it great fun...

Save the WebmasterWorld robots.txt Blog

New WWW Life Form Missing

I Scream, You Scream, We all Scream for robots.txt Blog

2:40 pm on Dec 28, 2005 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



ahhh, the dangers of a scripted bots.txt. ;-)

...should work now (tm)

10:02 pm on Jan 2, 2006 (gmt 0)

WebmasterWorld Senior Member whoisgregg is a WebmasterWorld Top Contributor of All Time 10+ Year Member



So, most blogs I just subscribe to the RSS feed. I've already checked for a /robots.rss or /robots.xml with no luck. :( So, this blog I have to remember to check every day.

Is it ironic that this is the first time I've felt like programming a robot for my personal use? :)

 

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month