Welcome to WebmasterWorld Guest from 54.159.50.111

Message Too Old, No Replies

How does Google know about my hidden forum?

     
4:23 am on Jul 30, 2011 (gmt 0)

Preferred Member

10+ Year Member

joined:Apr 24, 2003
posts:544
votes: 0


I saw today in my webmaster tools under site performance an example link to a topic in my discussion forums premium member section. There is no link to it publicly and you not only have to be logged on to see it, but also in a special member group.

So how does Google find out about those links and why does Google bother to sniff there around and give it as an example of site speed when its an error message anyway?
5:45 am on July 30, 2011 (gmt 0)

Senior Member from LK 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 16, 2005
posts:2417
votes: 17


You may not have linked to it, but someone else may have - could one of your members have linked to it from somewhere?
5:56 am on July 30, 2011 (gmt 0)

Preferred Member

10+ Year Member

joined:Apr 24, 2003
posts:544
votes: 0


@graeme, sure i cant control that and this might be a possibility althouth i doubt it, they are not very internet savvy users on my forum and its no use to link as you have to be special member group to see it anyway, and the question remains, why would Google make this page part of the site speed measurement. It doesnt make sense...
6:30 am on July 30, 2011 (gmt 0)

Senior Member from CA 

WebmasterWorld Senior Member 10+ Year Member

joined:June 18, 2005
posts:1692
votes: 3


Anyone using the Google toolbar or Chrome would help get your pages detected.
6:35 am on July 30, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:Mar 9, 2010
posts:1806
votes: 9


viggen, google has now gone beyond links to crawl content. Not sure how they are doing it and I am still trying to trace this. But I can confirm that from what I see on a few sites and some people are already aware of it.
6:50 am on July 30, 2011 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month

joined:Apr 9, 2011
posts:12708
votes: 244


Analogy: Consider how many search results point to something interesting on page 17 of an article which turns out to live in a members-only database, so you have to pay to read beyond the abstract if you're not affiliated with a major university. I don't think it's because g### are in cahoots with the database owners. They've just never met a door they couldn't open.
9:22 am on July 30, 2011 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 12, 2004
posts:608
votes: 1


Definitely toolbar or Chrome.. I noticed my own internal text editing script among site speed samples.. It wasn't optimized for speed and it (probably) was dragging average speed down..
12:20 pm on July 30, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Sept 25, 2005
posts:966
votes: 68


GWT Site Performance statistics have nothing to do with links, crawling or indexing. They're drawn from usage data, pulled from various sources such as the Google Toolbar. Google isn't 'sniffing around' those private pages, but your users obviously are, and they might have the toolbar enabled, thereby sending (anonymous) statistics, such as page load time, to Google and consequently to your Webmaster Tools account.
12:30 pm on July 30, 2011 (gmt 0)

Preferred Member

10+ Year Member

joined:Apr 24, 2003
posts:544
votes: 0


@robzilla, thanks for that,

i just wonder why Google would show that particular link as an example in site speed, i have about 50 people that have access to that, so it is not even close to be relevant (i get about 10.000 unique visitors a day) so is that site speed data and the links that are shown in GWT just a random lucky draw of no relevance?
12:44 pm on July 30, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 26, 2000
posts:37301
votes: 0


...when its an error message anyway?

What http status is served in the server header along with this error message?
2:50 pm on July 30, 2011 (gmt 0)

Preferred Member

10+ Year Member

joined:Apr 24, 2003
posts:544
votes: 0


@tedster
Result code: 404 (NotFound / Not Found)
I believe it is the standard error message from invision forum when a page is called that doesnt exist or one hasnt access to it...
3:09 pm on July 30, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 26, 2000
posts:37301
votes: 0


If there's a 404 status in the http header, rather than just a text message in the page, then Google does not normally index the page. If they do, it's usually a short term error on their part.
4:05 pm on July 30, 2011 (gmt 0)

New User

10+ Year Member

joined:Jan 6, 2006
posts:14
votes: 0


I was doing a search on a product on Google and interetingly the link to the download page was indexed by Google. If this was a newbie marketer I would understand the error but that was a seasoned marketer. Seems strange.
4:36 pm on July 30, 2011 (gmt 0)

Full Member

10+ Year Member

joined:Oct 12, 2005
posts: 262
votes: 0


Let me guess, you have analytics on these pages as well ;)
Maybe even AdSense?

Then there are referals, Google Toolbar, Chrome.

So many ways that tell google about a webpage
7:58 pm on July 30, 2011 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:May 14, 2008
posts:3092
votes: 2


You missed out gmail, which I would not rule out.

On my Control Panel I block browsers that include the GTB string in their UA. It's easy enough to block others, including chrome itself, assuming the visitor is not savvy enough or corrupt enough to forge the UA.
8:07 pm on July 30, 2011 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member netmeg is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Mar 30, 2005
posts:12677
votes: 143


They show admin URLs for my various WordPress sites too, and there's no point in measuring those for speed. Nor can they even get into those pages as they are password protected. That's just Google.
3:33 am on July 31, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member sgt_kickaxe is a WebmasterWorld Top Contributor of All Time 5+ Year Member

joined:Apr 14, 2010
posts:3169
votes: 0


Instead of writing out the list, which is extensive, on ways google gathers data let me give you an example of how NOT private the internet it.

Lets take this very post I'm writing as an example.

The instant I started writing this post Google knew my IP and the exact time I started writing it because of the javascript file that loaded with the little +1 button at the bottom of the page. Google will know the INSTANT I hit publish that mine was the first ever IP address on this page AFTER this is posted because of the +1 button javascript file that I will see on the next page after posting. Google will know that the IP they recorded with this forum posting is me the next time I log into my gmail or adsense since the connection will be made with my IP upon logging in. It's the same on all forums/blogs.

Even if there wasn't a +1 button down there most sites have SOMETHING Google. A search box, analytics, adsense, a 3rd party tracking system which shares data... the list is long.
3:39 am on July 31, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:Mar 9, 2010
posts:1806
votes: 9


Not all users have a fixed IP...In fact, majority of web users have dynamic IPs. If you say that they can use something like IP + machine identifiable info through those cookies, it might be valid to some extent.

But remember that google never gives away IP of visitors in its analytics tool and I think they even say that personally identifiable information are not gathered. This could be applicable to any of their products.

[edited by: indyank at 4:03 am (utc) on Jul 31, 2011]

3:45 am on July 31, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:Mar 9, 2010
posts:1806
votes: 9


netmeg, I am sure that you are one who wouldn't use a google toolbar or any of their products while accessing your wordpress sites or do you?
7:12 pm on July 31, 2011 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:May 14, 2008
posts:3092
votes: 2


Sgt - I have JS blocked in this forum so from that viewpoint they do not know who I am - I see no active JS on this page at all. I almost always block google JS nowadays anyway - froogle/base is the exception.

On the other hand, since I have had the same fixed IP for several years and have registered my domain with them for various things in the past (eg WMT), I am sure google knows exactly who I am IF they can grab my IP. They can probably make a shrewd guess just from my nickname here.
11:11 pm on Aug 1, 2011 (gmt 0)

Preferred Member

10+ Year Member Top Contributors Of The Month

joined:Jan 20, 2005
posts:489
votes: 0


This is probably because of toolbar data. My browser homepage was mysite.com/myadminpage, without analytics and without adsense. Randomly it seemed, it was one of the top pages in G webmaster tools. The Google toolbar is the only way that Google could have known this page existed.

Since uninstalling the toolbar and basically ridding my system of anything Google and only using Chrome for testing purposes, the myadminpage that is my homepage has completely disappeared from my WMT account.
1:45 am on Aug 2, 2011 (gmt 0)

New User

joined:July 13, 2011
posts:36
votes: 0


Your forum member unknowingly have created an action that has created a certain. As you said, they are not internet savvy but we do not know what applications and programs have been used by these users.

Just a theory though...
3:44 am on Aug 2, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:July 26, 2006
posts: 1619
votes: 0


I'm sure that just having a Google toolbar gives away alot more information about your surfing habits than they'll ever admit.