Welcome to WebmasterWorld Guest from 54.160.163.163

Message Too Old, No Replies

Google Sitemaps - Updated

With new information about crawl, index, pagerank

     

skyhawk133

7:58 pm on Nov 16, 2005 (gmt 0)

10+ Year Member



I just logged in to my Google Sitemaps ([google.com ]) and noticed they made some updates in the past hour or 2.

Noticeably, the interface is a little different. Sitemaps are now listed as "See stats for: yourdomain.com" Clicking this link shows several new stats (used to only show the crawl stats)

Now you are presented with this:

Query stats ¶ Crawl stats ¶ Page analysis ¶ Index stats

Here are the descriptions:


Query Stats:
Top search queries are the top queries to Google that return pages from your site. Top search query clicks are the top queries to Google that directed traffic to your site (based on the number of clicks to your pages in our search results).

Crawl Stats:
These statistics provide distribution information for pages we have crawled. [This page also shows page rank distribution across all pages crawled.]


Page Analysis:
These statistics show you how the Googlebot sees your site.


Index Stats:
You can use our Advanced Search page to find out how your site is indexed. Below, we've done some of these searches for you. [Just quick links to site:domain.com, etc.]

I like the information being provided on query stats the most, it's only a little glimpse, but lets you know what keyword you are showing up most for.

There is also a new "Errors" tab that has the following info:

HTTP errors ¶ Unreachable URLs ¶ URLs restricted by robots.txt ¶ URLs not followed ¶ URLs timed out

In order to use any of the advanced statistics, you must verify your domain. Information on stats and verifying can be found here [google.com]

aeiouy

8:18 pm on Nov 16, 2005 (gmt 0)

10+ Year Member



Thanks for pointing that out. Had not been to sitemaps for a bit.. Some interesting information there.

skyhawk133

8:30 pm on Nov 16, 2005 (gmt 0)

10+ Year Member



I'm curious why they are including this information in sitemaps. Did G just decide to display this info and figured sitemaps was the best place to put it? It doesn't seem to have much to do with the actual sitemaps as the crawl errors are all pages that aren't in my sitemap.

Part of a bigger plan?!

bird

8:34 pm on Nov 16, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I like the information being provided on query stats the most, it's only a little glimpse, but lets you know what keyword you are showing up most for.

Sounds like a lure to attract more people to using sitemaps?
Certainly makes it tempting!

ssjxxx

8:39 pm on Nov 16, 2005 (gmt 0)

10+ Year Member



Based on the appearance of the updated site, it seems to provide a lot more info. I'm getting metrics for average pagerank of my sites' pages and metrics about the type of content. Very nice.

The stylesheet is different, so it required a hard refresh when I visited the page. But other than that, well done. :)

ssjxxx

8:52 pm on Nov 16, 2005 (gmt 0)

10+ Year Member



Here's the details on the new stats:
[google.com...]

-S

webdude

9:03 pm on Nov 16, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I like it!

garyr_h

12:00 am on Nov 17, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Both the Top Search Queries and the Top Search Query Clicks include the top 5 of each for your site.

I love the feature, definately nice to see what people are clicking on compared to our actual ranking.

engine

12:04 am on Nov 17, 2005 (gmt 0)

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Following on from the announcement earlier in the year [webmasterworld.com], Google have now updated the tools. This latest announcement was made at the WebmasterWorld Pubcon in Las Vegas today.

What's new:
There are new "query stats" that show top Google search queries that return pages from a site. It also shows the top search queries that drove users to click on a site.
Crawl errors: Will now show, for example, specific HTTP errors Googlebot runs into when crawling a page.

Looks useful to me.

Log in here [google.com...]

pontifex

12:30 am on Nov 17, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



that seems to become very interessting, but:

"Data is not available at this time. Please check back later for statistics about your site."

especially the

"The PageRank of your pages in Google ¦ Distribution"

With me: the majority is "Low" ... #*$!IGO?

Nevertheless: thanks for the heads up, not everybody is drinking as it seems :-) ...

p!

selomelo

1:03 am on Nov 17, 2005 (gmt 0)

10+ Year Member



With the implementation of new Sitemap features, we can be sure that it will be another weighing factor in SERP algo. I can see how Google would make use of this new feature. And I suspect that they implemented another parameter not visible to webmasters: Time spent on a page accessed through keywords.

If visitors reaching your page through a query string spend considerable time at your site, then google would regard it as an indication of relevancy, and would increase weighing factor of the string in question for your site. Conversely, if the visitors spend little or no time, then google would suspect that the association between the query string and your page is weak, that the relevancy is low, and accordingly will decrease its weigiht.

IMHO, this new feature is a real breakthrough in Google's fight against spam. It can easily turn out to be the spammers' nightmare if my assumption about the use of time parameter is correct.

iblaine

2:08 am on Nov 17, 2005 (gmt 0)

10+ Year Member



Nice upgrade. I'm still waiting on the sitemaps feature to let me know which pages were crawled organically vs naturally. My particular problem is that G says I have 30,000 pages when my sitemap only has a few thousand.

Hanu

4:25 am on Nov 17, 2005 (gmt 0)

10+ Year Member



It's great! I like the "Top search queries" vs "Top search query clicks" feature. That's data that hasn't been available before.

shri

5:35 am on Nov 17, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>> With me: the majority is "Low" ... #*$!IGO?

Pontifex : Perhaps they revealed the *real* PR for a second or two through a buggy sql query? ;)

Essex_boy

6:45 am on Nov 17, 2005 (gmt 0)

WebmasterWorld Senior Member essex_boy is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Looks good but not to sure how to use it yet...

reseller

8:54 am on Nov 17, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Great Info!

I like it ;-)

RockyB

9:48 am on Nov 17, 2005 (gmt 0)

10+ Year Member



Fantastic, I'm really going to have to look at this in more detail :D

As far as I can tell 'low' pagerank counts as either a 3 or a two. Not quite sure which though.

A shame it doesn't explicitly state which pages googlebot has hit though, I've never been able to figure that out except for a 15 day trial of analyse spider.

isorg

9:50 am on Nov 17, 2005 (gmt 0)

10+ Year Member



Data is not currently available. Please check back later for statistics about your site.

I submitted the sitemap a few days ago. Is this delay normal?

cornwall

10:18 am on Nov 17, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Matt Cutts has a bit about it on his blog

I signed up today and itís pretty sweet. For example, you can now see crawl errors, timeouts on pages, robots.txt errors, unreachable urls, etc. Just really useful hard data that tells you if you have crawl problems and what they are. And you do not need a sitemap to use this functionality. You just create an empty file to verify that you own the domain.

vincevincevince

10:32 am on Nov 17, 2005 (gmt 0)

WebmasterWorld Senior Member vincevincevince is a WebmasterWorld Top Contributor of All Time 10+ Year Member



And you do not need a sitemap to use this functionality. You just create an empty file to verify that you own the domain.

That is absolute gold-dust...

Questions:

Are all stats shown if you don't have an empty file (currently I only see query stats - but will that change?)

What's the difference between " Top search queries" and " Top search query clicks"... I'm seeing terms which my site doesn't even come up for under the latter column.

Problem with the system:

It mixes image and search click results (at least) but has a link only to the main search results. This makes isolating the two very difficult.

[edited by: vincevincevince at 10:44 am (utc) on Nov. 17, 2005]

isorg

10:39 am on Nov 17, 2005 (gmt 0)

10+ Year Member



That is absolute gold-dust

... because it tells google who owns what domain (better than WHOIS data ever could) and therefore who is cross-linking all their sites, so that they can take this into account when assigning values to links...

vincevincevince

10:46 am on Nov 17, 2005 (gmt 0)

WebmasterWorld Senior Member vincevincevince is a WebmasterWorld Top Contributor of All Time 10+ Year Member



... because it tells google who owns what domain (better than WHOIS data ever could)

That's a good point. Remembering the smart pricing approach, penalties could now easily be reflected across all your sites. A good case for multiple sitemap accounts?

I think I'll be creating one account per website unless Google can make a statement to the effect that they will not use the data we verify here to relate one site to another, and I'd suggest others do the same.

I'm getting this for most sites (no sitemap, but are verified):

Data is not available at this time. Please check back later for statistics about your site.

isorg

11:27 am on Nov 17, 2005 (gmt 0)

10+ Year Member



Data is not available at this time

Sandbox...?

ProTalker

11:49 am on Nov 17, 2005 (gmt 0)

10+ Year Member



It's great! I like the "Top search queries" vs "Top search query clicks" feature. That's data that hasn't been available before.

Too right it's great! So what I'm assuming it's saying is "here's a list of the queries you're turning up in the strongest" (presumably a combination of position and frequency of the search?) and "here's a list of the queries where people are actually clicking on your result". Obviously we know the second one from our logs, but the first bit of information is fantastic. It shows one of my sites to be appearing strongly for the query "widgets" (and indeed, on checking, it does). But I never knew about this, because nobody ever clicked on my Google result. If I just go away and play with improving the presentation of that result, perhaps I can change all that!

b0rdslide

12:48 pm on Nov 17, 2005 (gmt 0)

10+ Year Member



If visitors reaching your page through a query string spend considerable time at your site, then google would regard it as an indication of relevancy, and would increase weighing factor of the string in question for your site. Conversely, if the visitors spend little or no time, then google would suspect that the association between the query string and your page is weak, that the relevancy is low, and accordingly will decrease its weigiht.

I don't think that would be the case as there is no way for google to know the amount of time that a user spends on any particular site. Unless of course that site is running the new google analytics code or using adsense.

It would be pretty foolish for google to add a weighting factor to their algorithm that they could only determine for a fraction of their index.

stinkfoot

1:28 pm on Nov 17, 2005 (gmt 0)

10+ Year Member



>It would be pretty foolish for google to add a weighting factor to their algorithm that they could only determine for a fraction of their index

Really? Are you talking about what used to be the old well reputed search facility some university guys set up or are you talking about the exceptionally large business that seeks only one thing as all business entities do ... profits?

BradleyT

4:31 pm on Nov 17, 2005 (gmt 0)

10+ Year Member



... because it tells google who owns what domain (better than WHOIS data ever could) and therefore who is cross-linking all their sites, so that they can take this into account when assigning values to links...

I disagree.

I have 12 domains in my sitemap account. My company owns 3 of them, the rest are our clients.

vincevincevince

5:04 pm on Nov 17, 2005 (gmt 0)

WebmasterWorld Senior Member vincevincevince is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Data which was available earlier today has been removed again. Either something's wrong or the server has taken a severe battering (it is sluggish now).

BradleyT

5:20 pm on Nov 17, 2005 (gmt 0)

10+ Year Member



It seems to be hit or miss on whether data displays for me. More often I get "Data Not Available" than any actual data.

LJCoolB

8:40 pm on Nov 17, 2005 (gmt 0)

10+ Year Member



Our huge site is part of a much larger corporate site, so I uploaded the blank html file for verification to:

www.domain.com/site/

The sitemap statistics tool will not let me access query stats, crawl stats, or page analysis. The error message reads:
"If you verify at [domain.com...] we will add it to the Site Overview page and show you errors with URLs for the entire site, as well as a greater variety of site statistics "

But I don't want site statistics for [domain.com...] I want them for [domain.com...]

Google treats www.domain.com and www.domain.com/site/ differently in the index and I actually do not have access to the main corporate directory. Very frustrating. And I was so hopeful.

We haven't submitted a sitemap because we get crawled just fine. Does anyone know if the results are better if you do have a Google sitemap?

Is anyone experiencing this same problem? Is there a workaround?

I sent Google a note asking how to get my site stats, but I have little hope I will get more than an autoresponse.

This 40 message thread spans 2 pages: 40