Forum Moderators: Robert Charlton & goodroi
Now my site changed, it became a very dynamic sort of web 2.0 site.
I have approximately 1000 new pages added each day.
What should I do about sitemap?
Should I just delete it or?
I couldn't possibly add all relevant links to it : (
And Google crawls our site great, almost all articles end up high in results.
What would happen if I abandon sitemap? Would Google change its attitude towards my site?
However, it sounds like you are now in a situation where a sitemap might be a better thing to implement than it used to be. You can automate the sitempa generation process on your server, creating and submitting a new sitemap automatically up to once per hour. I'd look into doing that.
Here's one reference from Google about a recurring sitemap script:
[google.com...]
I already have some scripts running in a sort of automated mode, I believe I could create a creation of sitemap along with other actions.
Just a few more questions:
1. should I include all links in my sitemaps? This might grow very large very fast. Will it choke bots? For example, what happens if your sitemap file is over 1 MB?
Should I in such case just "show" bots the way to the main categories and keyword tag sections? (my categories are constructed something as www.example.com/category/tag/)
2. I'm not very knowledgable with scripting options google reccomends for creating a dynamic sitemap, but I could easily do it with PHP. Where could I find a simple outline and what actually needs to go inside sitemap.xml file?
Should I include only basic links or additional parameters? I saw somewhere that sitemaps can include stuff like crawl rate etc..
From these numbers we can see that my sitemap will actually grow by 1 MB every month.
I'm not sure how will Google like this.. What is the procedure in such occasions?
I've read some articles at Google but didn't find this topic addressed.
Do I need to break up my sitemap into pieces or?
If you want to list more than 50,000 URLs, you must create multiple Sitemap files. If you anticipate your Sitemap growing beyond 50,000 URLs or 10MB, you should consider creating multiple Sitemap files. If you do provide multiple Sitemaps, you can list them in a Sitemap index file. Sitemap index files may not list more than 1,000 Sitemaps.[google.com...]