Welcome to WebmasterWorld Guest from 54.198.134.104
Forum Moderators: Robert Charlton & goodroi
I have done optimization for my one website.
But in that site i don't know why Google taking so much time for crawling my webpages. Google crawling my Home page every 10 to 12 day and all inner pages crawling every 15 to 20 days.
Even I have already done deeplink submission for all my inner pages and also i am tweaking my pages every 3 days.
My site is one year old. My home page size is 38 KB and all inner pages size are 20KB to 30KB. What do you think? 38KB page size is difficult to crawl for Google. It creates problem in crawling my webpages?
I also checked in webmaster tools that my craw rate is normal.
Please suggest me your ideas that how i can increase my crawl rate in Google.
Make sure that you have no technical errors creating more than one URL for the same content. Googlebot crawls with a budget, and if it spends part of that budget downloading duplicate content from different URLs on your site, then it has less budget for crawling the rest of your URLs.
1. incorrect query strings
2. 404 pages (at both the server level and the platform level)
3. https protocol requests
4. different order of the parameters or virtual folders
5. index.html vs. directory root
6. mixed up letter cases
7. double slashes in the file path
If so, you have a rare creation. I have had many sites come to me for help, and almost all of them have at least one of these issues.
The best way to increase crawl rate, besides telling Google that you want it with your Webmaster Tools account and within your sitemap, is to have a very prominent site, well respected in your field and widely linked to. The more important Google sees your site, the better your crawl will be.
Also, make sure your server is handling if-modified-since requests properly, has not the least little DNS problem, no major server delays, no load-balancing issues, makes proper use of E-tags, and uses compression appropriately especially on text files.
If you've got all that handled on your side, then the ball is in Google's court. You cannot force a more frequent spidering, you can only invite it by building a technically solid site with truly world-class content.
In webmaster tools the radio button of high crawl rate is disable to so no it is not possible.
and Server side my site is OK right now there is no any issue