Welcome to WebmasterWorld Guest from 54.198.221.13

Forum Moderators: lawman

Message Too Old, No Replies

WebmasterWorld Back In Google Index?

     
10:13 pm on Dec 17, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Feb 18, 2004
posts:298
votes: 0


Not sure who else is seeing this or if its already been posted but I'm seeing WebmasterWorld with a homepage PR7 and 239,000 pages indexed in Google.
10:16 pm on Dec 17, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 9, 2003
posts:815
votes: 0


Maybe Brett was somehow able to manually let in googlebot or googlebot can log in now.
10:20 pm on Dec 17, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Dec 9, 2005
posts:60
votes: 0


Interesting.

Good find.

10:31 pm on Dec 17, 2005 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 22, 2003
posts:1483
votes: 0


Will the real robots.txt please stand up? Never a dull moment around here. I hope it's back for good.
10:56 pm on Dec 17, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Feb 18, 2004
posts:298
votes: 0


I hope it's back for good.

Me to! I've been missing the search capabilities.

HOWEVER, I'm not going to hold my breath.

11:59 pm on Dec 17, 2005 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38079
votes: 28


lol skip factor...

We are working on stuff - everything is in flux. Things will change...

bug reports to brett at ....you know the address. lol

...thanks GoogleGuy, YahooGuy, MSNDude, JeevesGuy, his Taco'ness [slashdot.org], and OJR [ojr.org]...

12:32 am on Dec 18, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Aug 8, 2004
posts:1679
votes: 0


Brett - do you cloak your robots.txt depending on IP address that requests it?
1:52 am on Dec 18, 2005 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38079
votes: 28


only for hot/honey pot purposes.

eg: if you have been banned for agent spoofing - send a reinclusion request via e or sticky...

3:42 pm on Dec 18, 2005 (gmt 0)

Senior Member from KZ 

WebmasterWorld Senior Member lammert is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 10, 2005
posts:2936
votes: 24


I first saw that the Alexa ranking for WW was returning to the level some weeks ago. Only after that I realized that WW was re-indexed again by Google. I hope we now get the best of both worlds, i.e. WW searchable by the main search engines again, but still blocking the unwanted bots to maintain performance.
7:25 pm on Dec 17, 2005 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38079
votes: 28


working on site a bit this afternoon...ignore any problems for the next few hrs...
8:09 pm on Dec 17, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 20, 2002
posts:1716
votes: 0


I find it highly problematic that you're working on a Saturday afternoon. ;)
8:13 pm on Dec 17, 2005 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator lawman is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:May 28, 2001
posts:3608
votes: 63


I read both posts. The boss has spoken. :)
12:18 am on Dec 19, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 21, 2003
posts:427
votes: 0


I loved to see WebmasterWorld is back, but checking robots.txt I am very curious whether it's google that no longer care for the disallow / thing or it's a manual enquiry or what :)
4:50 am on Dec 19, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 4, 2004
posts:801
votes: 0


google search is working great, msn is lagging though, still only at 80k.

Good to see you letting search back in brett, too many resources here to cut off, jd morgan has already answered so many questions, too much information to not have searchable, same for other coding stuff especially.

oh, and don't try surfing the site with a spoofed search bot user agent, as brett says.

moftary, unless you have a google ip address and are googlebot, it's unlikely you'll be seeing what you want to see, enough snooping for me, LOL...

9:45 am on Dec 19, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 21, 2003
posts:427
votes: 0


moftary, unless you have a google ip address and are googlebot, it's unlikely you'll be seeing what you want to see, enough snooping for me, LOL...

hehe you should not have been snoopy.
Looks like I missed some posts here.
after Brett's sticky mail I got the whole story.

welcome back, WebmasterWorld :)

12:54 pm on Dec 19, 2005 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38079
votes: 28


published a response in the robots.txt..

[webmasterworld.com...]

3:45 pm on Dec 19, 2005 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 22, 2003
posts:1483
votes: 0


>>published a response in the robots.txt..

Only Shak could make it into a robots.txt clear from China. I vote for:

User-agent: Flaming Shak-Attack Bot
Disallow: /

Great post, thanks. First time I've bookmarked a robots.txt. Where can I find the one cloaked for humans with the subliminal marketing stuff?

2:38 am on Dec 20, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 25, 2005
posts:95
votes: 0


I love the robots.txt blog

Lemme guess, you were looking for bots in the bottom of that bottle, and the blog "just happened"?

Cool...

K

6:55 am on Dec 20, 2005 (gmt 0)

Administrator from JP 

WebmasterWorld Administrator bill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Oct 12, 2000
posts:15023
votes: 136


LOL. Can't wait till you enable comments in that blog Brett. ;)
1:39 pm on Dec 20, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:June 21, 2000
posts:626
votes: 0


published a response in the robots.txt..

[webmasterworld.com...]

I took a read, had some time to kill. Why would you slot this under the Yahoo! portion?

6 Degrade/filter/penalize/whatever any URL with a question mark in it.

I can deal with that no problem BUT most mom and pop sites, on a tight budget, would not be able to most likely (in a low budget scenario) have the funding to make this happen. Wouldn't that hurt the "little people"/small startups which all of us start out like (well most of us ;)). Not ranting here just curious.

Take care,

Brian

10:08 pm on Dec 21, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member sem4u is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Dec 18, 2002
posts:3072
votes: 1


I'm loving the blog too :)
6:05 am on Dec 22, 2005 (gmt 0)

Senior Member from HK 

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 14, 2002
posts:2297
votes: 18


Would like to see if Google chokes on a large robots.txt. I know they choked on a 1.2MB robots which I had created for the sake of the removal tool. :(
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members