Msg#: 3934461 posted 4:33 pm on Jun 16, 2009 (gmt 0)
Google used to crawl my web site very diligently before I opened a webmaster account with Google, submitting my first XML sitemap around May 24th (not sure about the exact date, could March 20). Sometimes several times a day.
Since then, new pages that I added to my web site simply don't show up in Google Search (even when I force site:<mydomain>.com).
Old pages (prior to May 24th) show up fine, just as they used to.
What could have I done wrong?
Is it possible that an XML sitemap submission to Google actually hurts?
Msg#: 3934461 posted 5:46 pm on Jun 16, 2009 (gmt 0)
Does your sitemap and file location comply with: [sitemaps.org...] ?
Absolutely. I know that the path [<mydomain>.com...] looks problematic but I (think that I) made sure that everything complies as follows:
In ~/public_html's .htaccess there is a redirect 301 to [<mydomain>.com...]
Thus everything in my website is currently located under [<mydomain>.com...] - including the newly added pages that Google ignores for some reason.
In addition, I made sure that robots.txt (located in ~/public_html, i.e. [<mydomain>.com...] is world-readable and contains a line that tells where the site's XML sitemap file is located (see the message that started this thread).
Msg#: 3934461 posted 5:00 pm on Jun 19, 2009 (gmt 0)
Hmmm... even after implementing the above suggested change in robots.txt, it seems that Google is insisting on ignoring the last product description page that I added to my web site on June 8, 2009, 21:05.
Interestingly enough, a product that I added on June 4, 2009, 06:58 is found by Google search.
Is there a minimum 2-week delay for Google to add website pages to its results?
I did specify <changefreq> for product pages to change daily but I read somewhere that this parameter is treated as a suggestion only? I have no explanation for this weird behavior.