Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Site maps uploaded much more frequently

Is this normal?

         

dataguy

2:14 pm on Oct 10, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I'm using Google site maps on just one of my web sites. My site maps are dynamically generated from the database that powers the web site and because they are not static XML files I decided to put in a function that emails me each time Google grabs the XML files.

Since the beginning Google would grab the files once a week, usually on Fridays. For the past 10 days they have been grabbing it about 7-10 times per day. These files are substantial in size (split across 3 separate XML files) and require some pretty hefty server resources to generate.

Is this happening across the board or is this unique to me? Any possible explanations?

netmeg

3:36 pm on Oct 10, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



There's another thread here somewhere, but it's happening to me too.

roodle

3:45 pm on Oct 10, 2005 (gmt 0)

10+ Year Member



I submitted one a week or so ago. It seems to be picked up reasonably frequently (daily?), but I'm not checking it too often. I get the impression G's crawling frequency isn't influenced by submitting a sitemap. It hasn't had any effect on crawled pages or rankings (yet..).

hutchmeister

4:22 pm on Oct 11, 2005 (gmt 0)

10+ Year Member



We have 5 maps at present, 2 went on a couple of hours ago so they are still 'pending'.
The other 3 have been grabbed every 3 or 4 hours since they were submitted and verified over the last few days.

I'm still not sure how this is going to affect the bot activity in the long run, and now much of the sites it peeks at each time, but so far were getting MUCH more regular crawling, with less content perused.

As with everthing from G we'll watch it, make postulations, try to second guess it, then fall flat on our faces.... yey!

*edit* - sorry, just looked a bit more closely. One site is getting it every 3-4 hours, the other 2 are getting grabbed a couple of times a day.

Website Manager

9:01 am on Oct 13, 2005 (gmt 0)



I have a couple of clients sitemaps that are being pulled roughly every 12 hours and have been for the last 2 months or so - is there any indication that the URL's are actually finding their way into Google's index?

I tagged a few of the pages I know are already listed with a small query string but this isn't showing up in the results pages yet.

anallawalla

2:12 am on Oct 14, 2005 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



is there any indication that the URL's are actually finding their way into Google's index?

Definitely. Doesn't mean the pages rank anywhere, but it's a start.

merlin78

6:21 am on Oct 14, 2005 (gmt 0)

10+ Year Member



I have also found my site maps are being uploaded alot more frequently of late. However on a couple of my sites I have noticed that Google is not grabbing the title and descriptions of the pages. These are in relation to some domains that have had some big changes over the last few months. The strange thing is that Google is indexing the title and description of pages for the one domain that I didn't create a sitemap for. The pages (and sites) in question are dynamically generated ColdFusion (cfm) pages but I don't think that should be causing any problem. Any ideas? Could Google be crawling the pages too quickly to index the titles and descriptions?

stinkfoot

11:12 am on Oct 14, 2005 (gmt 0)