How does Google recognize a site map / crawler page?

Looking for suitable site map file names and text links

[HarryM]

I understand that Google not only makes use of site maps, but can also recognise a site map from the link to it. Does anyone know what it looks for in the link?

In the text? In the file name? Can it also use the title attribute?

I've searched WW, and there's plenty of stuff here, but nothing I can find specific to my query.

[rogerd]

Probably the link text "Site Map" would work if you want it to be identified as a site map. I'm not sure that doing so is desirable, or if Google really looks for site maps as a positive thing. Google will probably interpret the page as a page of links one way or the other, and depending on a variety of factors may spider all or some of the pages.

[jimbeetle]

Hi HarryM,

I haven't heard of Google specifically recognizing links to site maps, though this should be fairly simple if it wanted to since most links to site maps us "Site Map" as the link text.

Depending on your site navigation and interal linking it does help to have a site map to make sure spiders can find and crawl all of your pages.

Jim

[HarryM]

Thanks for the feedback.

Due to an increase in number of pages, I am currently splitting up my existing site map into three. The original was linked from my index page by <a href="crawler.php">site map</a>.

I was hoping to get something definitive so I could create three links in an intelligent way. I want to avoid the term "site map" in each of the three links as it gets a bit clunky, but would use it if that is what Google (or other bot) looks for.

[rogerd]

HarryM, I wouldn't be so sure that Google (or other SE) goes looking for site maps as a great way to index a site. If pages can be found only through the site map and not other means of navigation and linkage, they may not be indexed or rank well. Logically, a page like that can't be very important, right? The whole PR structure is geared to measuring page worth by linkage. A site map link isn't going to carry the weight of linkage from multiple "important" pages, for example.

In short, what I'm saying is that you should be sure not to use the site map as a substitute for good, logical, and spiderable linkage on the entire site.

[HarryM]

RogerD

In short, what I'm saying is that you should be sure not to use the site map as a substitute for good, logical, and spiderable linkage on the entire site.

Thanks for that advice. However what I am hoping for is that someone knows the answer to my specific question.

Does Google recognise site maps? And if so how?

I can't find the thread now but GG seemed to consider site maps important and gave advice on how big they should be - which is why I am splitting mine into three.

[jomaxx]

I think you are referring to Google's advice to webmasters here:
[google.com...]

A site map is generally useful but not mandatory, and it's very doubtful Google would try to "recognize" it or handle it differently from any other page.

[HarryM]

To clarify my preceeding post. The following is taken from Google Information for Webmasters, Design and Content Guidelines.

Offer a site map to your users with links that point to the important parts of your site. If the site map is larger than 100 or so links, you may want to break the site map into separate pages.

<edit>Thanks Jomaax. Our posts crossed.</edit>

[vitaplease]

If Google would have to recognise site maps due to a link with "site map" it would have to do quite some translation work on all the other 200 languages, not to mention all the synonyms.

I think Google mentions the fact that a site map is important so that webmasters do not forget that a page needs a regular link to be spidered and indexed by Google.

Also I would say they mention the limit of 100 links so that a Pagerank 2 sitemap will still allow all of its "off-spring" to be indexed.

[Flippi]

You don't have necessarily to declare a page as 'site map' (title e.g.). Google will recognize the page as 'important' cause the crawlers find there a lot of inbound links that leed into the deep of your website. And because the links are the food for a crawler it's important to have a 'downstairs link page' or a so called 'site map'. That's all.