Welcome to WebmasterWorld Guest from 54.162.240.235

Message Too Old, No Replies

GWT Sitemap and Google Bot Crawl Rate

     
12:03 pm on Dec 12, 2013 (gmt 0)

10+ Year Member



I haven't added a sitemap to Google Webmaster Tools before because I was concerned since not all of my new material gets added to the sitemap of my site. I was afraid Google would concentrate more on the sitemap than crawling the rest of my site and end up missing this new material from my site.

If you add one or more sitemaps, does Google Bot crawl rate go down or do they still try to find new/updated material the old fashion way?

[edited by: aakk9999 at 12:06 pm (utc) on Dec 12, 2013]
[edit reason] replaced "site index" with "sitemap" to make a question clearer [/edit]

1:40 pm on Dec 12, 2013 (gmt 0)

WebmasterWorld Senior Member netmeg is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



If there's a link to it somewhere, Google will find it. They may or may not index or rank it, but they'll find it.
9:45 pm on Dec 12, 2013 (gmt 0)

10+ Year Member



So you recommend the sitemap then?

All of my articles are included in the sitemap but it is the user generated comments and when I update pages on topics etc.. (which are not articles) that don't get included in the sitemap.

Google still crawls the site as per norm looking for other new content beyond what is included in the sitemap?

Thanks
3:29 am on Dec 13, 2013 (gmt 0)

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



google is pretty good with link discovery.
make sure googlebot doesn't discover non-canonical urls in your sitemap and you should be okay.
perhaps more important for link discovery is a good internal link structure and external links to your site from relevant and authoritative content.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month