homepage Welcome to WebmasterWorld Guest from 54.234.141.47
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Local / WebmasterWorld Community Center
Forum Library, Charter, Moderators: lawman

WebmasterWorld Community Center Forum

    
WebmasterWorld Back In Google Index?
ebound

10+ Year Member



 
Msg#: 3086 posted 10:13 pm on Dec 17, 2005 (gmt 0)

Not sure who else is seeing this or if its already been posted but I'm seeing WebmasterWorld with a homepage PR7 and 239,000 pages indexed in Google.

 

ganderla

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3086 posted 10:16 pm on Dec 17, 2005 (gmt 0)

Maybe Brett was somehow able to manually let in googlebot or googlebot can log in now.

BadSense

5+ Year Member



 
Msg#: 3086 posted 10:20 pm on Dec 17, 2005 (gmt 0)

Interesting.

Good find.

skipfactor

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3086 posted 10:31 pm on Dec 17, 2005 (gmt 0)

Will the real robots.txt please stand up? Never a dull moment around here. I hope it's back for good.

ebound

10+ Year Member



 
Msg#: 3086 posted 10:56 pm on Dec 17, 2005 (gmt 0)

I hope it's back for good.

Me to! I've been missing the search capabilities.

HOWEVER, I'm not going to hold my breath.

Brett_Tabke

WebmasterWorld Administrator brett_tabke us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3086 posted 11:59 pm on Dec 17, 2005 (gmt 0)

lol skip factor...

We are working on stuff - everything is in flux. Things will change...

bug reports to brett at ....you know the address. lol

...thanks GoogleGuy, YahooGuy, MSNDude, JeevesGuy, his Taco'ness [slashdot.org], and OJR [ojr.org]...

Lord Majestic

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3086 posted 12:32 am on Dec 18, 2005 (gmt 0)

Brett - do you cloak your robots.txt depending on IP address that requests it?

Brett_Tabke

WebmasterWorld Administrator brett_tabke us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3086 posted 1:52 am on Dec 18, 2005 (gmt 0)

only for hot/honey pot purposes.

eg: if you have been banned for agent spoofing - send a reinclusion request via e or sticky...

lammert

WebmasterWorld Senior Member lammert us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 3086 posted 3:42 pm on Dec 18, 2005 (gmt 0)

I first saw that the Alexa ranking for WW was returning to the level some weeks ago. Only after that I realized that WW was re-indexed again by Google. I hope we now get the best of both worlds, i.e. WW searchable by the main search engines again, but still blocking the unwanted bots to maintain performance.

Brett_Tabke

WebmasterWorld Administrator brett_tabke us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3086 posted 7:25 pm on Dec 17, 2005 (gmt 0)

working on site a bit this afternoon...ignore any problems for the next few hrs...

Hawkgirl

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3086 posted 8:09 pm on Dec 17, 2005 (gmt 0)

I find it highly problematic that you're working on a Saturday afternoon. ;)

lawman

WebmasterWorld Administrator lawman us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3086 posted 8:13 pm on Dec 17, 2005 (gmt 0)

I read both posts. The boss has spoken. :)

moftary

10+ Year Member



 
Msg#: 3086 posted 12:18 am on Dec 19, 2005 (gmt 0)

I loved to see WebmasterWorld is back, but checking robots.txt I am very curious whether it's google that no longer care for the disallow / thing or it's a manual enquiry or what :)

2by4

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3086 posted 4:50 am on Dec 19, 2005 (gmt 0)

google search is working great, msn is lagging though, still only at 80k.

Good to see you letting search back in brett, too many resources here to cut off, jd morgan has already answered so many questions, too much information to not have searchable, same for other coding stuff especially.

oh, and don't try surfing the site with a spoofed search bot user agent, as brett says.

moftary, unless you have a google ip address and are googlebot, it's unlikely you'll be seeing what you want to see, enough snooping for me, LOL...

moftary

10+ Year Member



 
Msg#: 3086 posted 9:45 am on Dec 19, 2005 (gmt 0)

moftary, unless you have a google ip address and are googlebot, it's unlikely you'll be seeing what you want to see, enough snooping for me, LOL...

hehe you should not have been snoopy.
Looks like I missed some posts here.
after Brett's sticky mail I got the whole story.

welcome back, WebmasterWorld :)

Brett_Tabke

WebmasterWorld Administrator brett_tabke us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3086 posted 12:54 pm on Dec 19, 2005 (gmt 0)

published a response in the robots.txt..

[webmasterworld.com...]

skipfactor

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3086 posted 3:45 pm on Dec 19, 2005 (gmt 0)

>>published a response in the robots.txt..

Only Shak could make it into a robots.txt clear from China. I vote for:

User-agent: Flaming Shak-Attack Bot
Disallow: /

Great post, thanks. First time I've bookmarked a robots.txt. Where can I find the one cloaked for humans with the subliminal marketing stuff?

kokaroach

5+ Year Member



 
Msg#: 3086 posted 2:38 am on Dec 20, 2005 (gmt 0)

I love the robots.txt blog

Lemme guess, you were looking for bots in the bottom of that bottle, and the blog "just happened"?

Cool...

K

bill

WebmasterWorld Administrator bill us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3086 posted 6:55 am on Dec 20, 2005 (gmt 0)

LOL. Can't wait till you enable comments in that blog Brett. ;)

Reflect

10+ Year Member



 
Msg#: 3086 posted 1:39 pm on Dec 20, 2005 (gmt 0)

published a response in the robots.txt..

[webmasterworld.com...]

I took a read, had some time to kill. Why would you slot this under the Yahoo! portion?

6 Degrade/filter/penalize/whatever any URL with a question mark in it.

I can deal with that no problem BUT most mom and pop sites, on a tight budget, would not be able to most likely (in a low budget scenario) have the funding to make this happen. Wouldn't that hurt the "little people"/small startups which all of us start out like (well most of us ;)). Not ranting here just curious.

Take care,

Brian

sem4u

WebmasterWorld Senior Member sem4u us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3086 posted 10:08 pm on Dec 21, 2005 (gmt 0)

I'm loving the blog too :)

shri

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3086 posted 6:05 am on Dec 22, 2005 (gmt 0)

Would like to see if Google chokes on a large robots.txt. I know they choked on a 1.2MB robots which I had created for the sake of the removal tool. :(

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Local / WebmasterWorld Community Center
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved